Navigation method in three-dimensional reconstruction operation based on accurate positioning and segmentation of kidney tumor

Abstract

The invention provides a navigation method in a kidney tumor three-dimensional reconstruction operation based on accurate positioning and segmentation, and the method aims at firstly carrying out the resampling, CT value truncation and z-score standardization operation of an obtained CT image of a patient to obtain a clearer organ feature boundary; and then accurately segmenting the kidney tumor in the CT image by using a designed kidney tumor segmentation network TransRFE-U2-Net so as to accurately reconstruct a three-dimensional model of the kidney tumor and the kidney. The network is improved by referring to a U-Net network structure, and a dimension raising U2-Net is provided to realize segmentation of a three-dimensional image. Besides, the network utilizes a residual feature enhancement module and a multi-head self-attention module based on global features to improve the ability of the network to capture global information and context information, and improves the accuracy of kidney tumor segmentation by the model, so that doctors can cut kidney tumors according to the accurate position and size of the tumors in a simulation operation, and the accuracy of kidney tumor segmentation is improved. Resection of normal kidney tissue is reduced, and the kidney function is protected.

Description

Technical Field

[0001] The invention relates to the technical field of computer vision and medical image processing, and in particular to an intraoperative navigation method based on accurate positioning and segmentation of kidney tumor three-dimensional reconstruction. Background Art

[0002] Renal cancer is one of the most common malignant tumors of the urinary system. Its incidence rate increases with age, and it has surpassed bladder cancer to rank first in the mortality rate related to urinary system tumors, posing a serious threat to the quality of life and life safety of patients. With the development of modern medical technology, the early diagnosis rate of renal cancer has increased year by year. Partial nephrectomy has been recommended as a surgical treatment by various guidelines because of its ideal tumor control effect and its ability to reduce the incidence and progression risk of chronic kidney disease. Compared with open kidney-saving surgery, laparoscopic partial nephrectomy (including traditional laparoscopy and robot) can reduce surgical trauma to patients and shorten postoperative recovery time. These advantages have become an important reason for the continued promotion of laparoscopic partial nephrectomy.

[0003] However, the preoperative planning of laparoscopic partial nephrectomy is still mainly based on two-dimensional images of CT or MRI. However, two-dimensional images cannot provide a sense of perspective, and cannot conceive the complex spatial structure and distribution of tumors, as well as the three-dimensional adjacent relationship between different structures. The value of preoperative planning based on two-dimensional images is limited. The two-dimensional information of preoperative sections lacks a sense of integrity and three-dimensional stereoscopic sense, which can easily lead to misjudgment. The surgeon often needs to temporarily plan the surgical method during the operation, which will increase the intraoperative decision-making time. In addition, in order to ensure the first requirement of the operation, the tumor must be completely removed without any residue. In order to achieve complete surgical resection, more normal renal tissue will inevitably be cut, affecting the quality of the operation. In addition, preoperative planning based on two-dimensional images may cause some important anatomical information to be ignored, resulting in intraoperative errors. Therefore, how to improve the safety of laparoscopic renal hilar tumor surgery and maximize the preservation of renal units is a major problem faced by clinical urologists.

[0004] In recent years, the continuous development of imaging technology and image-guided precision surgical treatment of cancer have provided new ideas and methods for laparoscopic surgery to treat renal hilar tumors. If the organs can be reconstructed in three-dimensional visualization before surgery to show the adjacent relationship between the renal tumor and the kidney as a whole, surgeons will have a more comprehensive understanding of the anatomical structure of the tumor before surgery, thereby helping them plan the path and range of tumor resection, on the one hand to completely remove renal cancer tissue, on the other hand to remove less normal renal tissue and protect renal function.

[0005] In the simulated three-dimensional space, surgeons can make preoperative plans based on the reconstructed model, so as to have a clear understanding of the operation, predict possible emergencies during the operation, and prevent panic and mistakes during the operation. Therefore, the present invention has important significance in the fields of medicine and computer vision. Summary of the invention

[0006] In view of the above-mentioned problems existing in the prior art, the purpose of the present invention is to provide an intraoperative navigation method based on precise positioning and segmentation of renal tumor three-dimensional reconstruction, which uses a designed deep learning network to perform three-dimensional renal tumor segmentation on CT images, reconstruct the size and position of the tumor, and then provide tumor positioning before surgery, so as to more effectively enable surgeons to establish preoperative planning, ensure the safety of surgery and reduce the resection of normal renal tissue.

[0007] The present invention provides the following technical solution: a navigation method for intraoperative three-dimensional reconstruction of renal tumor based on accurate positioning and segmentation, comprising the following steps:

[0008] (1) Obtaining original CT scan images from the CT scan database of the patient's preoperative physical examination;

[0009] (2) resampling, CT value truncation, and z-score normalization are performed on the acquired original CT data to make the characteristic boundaries between organs in the original image clearer;

[0010] (3) Using TransRFE-U 2 -Net performs renal tumor segmentation on the preprocessed CT images, TransRFE-U 2 -Net network is a 3D segmentation network based on global feature multi-head self-attention and residual feature enhancement;

[0011] (4) using the segmentation results in step (3) to perform three-dimensional reconstruction of the image to obtain a 3D model of the tumor and kidney positions;

[0012] (5) The doctor determines the size and location of the tumor based on the 3D reconstruction model and conducts preoperative discussions to formulate a surgical plan;

[0013] (6) Determine the optimal cutting route through simulated surgery.

[0014] Furthermore, in step (2), the specific process of resampling the acquired original CT data is as follows:

[0015] In data preprocessing, the layer thickness in the z direction and the layer spacing in the x and y directions of all samples were adjusted, and the spatial resolution of all samples in the segmentation network model remained unchanged.

[0016] Furthermore, in step (2), the specific process of CT value truncation is as follows:

[0017] According to the imaging principle of CT, different tissues have different attenuation coefficients for X-rays. The ratio of the attenuation coefficient of different tissues to the attenuation coefficient of water is used as the CT value. The following is the calculation process of the CT value:

[0018]

[0019] CT M Indicates the CT value of a certain tissue. The unit of value is HU, μ M It indicates the attenuation coefficient of a certain tissue. Different tissues have different attenuation coefficients and CT values. Represents the attenuation coefficient of water. When segmenting the kidney and renal tumor, the kidney is distinguished from surrounding organ tissues based on the fact that the greater the density of the tissue, the greater the corresponding attenuation coefficient and CT value;

[0020] The CT scan images obtained contain other tissues and organs besides the kidneys, so the threshold method is used to eliminate the interference of other tissues. The specific operation is as follows:

[0021]

[0022] Among them, V in represents the original CT image, V out Represents the image after truncating after thresholding.

[0023] Furthermore, in step (2), the specific process of z-score standardization is as follows:

[0024] When the convolutional neural network segments kidney CT images, different feature information is distinguished by CT values; the CT value range of the same organ is large and there are errors. When some of the same feature information is represented by CT values ​​with adjacent values, the network layer recognizes different feature information; using the z-score standardization method, the distribution of CT values ​​is mapped to a smaller range, but the relationship between the values ​​is the same as before processing and has not changed. The specific operation is as follows:

[0025]

[0026] Among them, V i is the CT value of a voxel in a single sample, V m is the mean CT value of all voxels in this sample, V sd is the standard deviation of the CT values ​​of all voxels in this sample, V i ′ is the corresponding value after z-score standardization.

[0027] Furthermore, in step (3), TransRFE-U 2 -Net network is in U 2 -Net is used for dimensionality upgrade. After the input data is encoded and features are extracted, the global dependencies are extracted through the multi-head self-attention module Trans-block of the global features. Then, the feature map size is restored and renal tumor segmentation is performed through the decoding module combined with the residual feature enhancement module.

[0028] Furthermore, U 2 After the -Net is upgraded to three dimensions, each layer under each encoding and decoding path is a structural block, and each structural block is nested with a sub-U-Net structure, that is, the entire model is a two-level nested U-shaped structure, and each structural block is called a Residual U-block; three-dimensional U 2 -Net encoding path has four downsamplings, with a total of five encoders, and in the decoding path there are four corresponding upsamplings, with a total of four decoders; in the encoding stage, after each feature map passes through an RSU, the maximum pooling is used to achieve downsampling, and the size of the feature map after downsampling is reduced to 1 / 2 of the original feature map; in the decoding stage corresponding to each encoding stage, after each image passes through an RSU, linear interpolation is used to achieve upsampling, and the size of the feature map after upsampling is restored to twice the original feature map.

[0029] Furthermore, in the step (3), by adding a residual branch, when the feature map enters the branch, it is respectively processed by the feature map based on three dimensions, and the linear perception unit MLP is fully connected, and the residual connection is performed after activation with the Sigmoid function, and the same upsampling operation as the original decoding module is performed; in order to enhance the expressive ability of the feature map in the spatial domain, a feature enhancement module in the spatial domain is introduced, and the upsampled feature map is respectively passed through a channel-based global pooling module to obtain two feature maps, which are concatenated and reduced to one through a convolutional layer, activated with a Sigmoid function and multiplied with the residual output of the second stage to obtain the final branch enhanced feature map, and the spatial domain enhancement module aggregates the information of all channels, and enhances the attention of the proposed network to the prospect information of the kidney and renal tumor; the feature enhancement module is used for each level of decoding module in the decoding path.

[0030] Furthermore, in step (3), a multi-head self-attention module based on global features is added to RFEU 2 -Net encoding end, the specific process is as follows:

[0031] S3.1. Flatten the entire 3D feature map obtained at the end of the encoding into a one-dimensional sequence. Specifically, the 3D feature map is divided into N image blocks, and each image block is flattened into a sequence Xp. Each sequence is mapped to a C-dimensional embedding space through a linear projection E, and the position encoding information E is added. pos , all image sequences after embedding processing ) can be expressed as:

[0032]

[0033] Where N is the number of blocks obtained after the image is sliced, that is, the sequence length; C is the number of channels; l-1 is the level number;

[0034] S3.2. Sequence Perform self-attention calculation to obtain the contextual correlation between each sub-image block in the original feature map, and use multiple groups of W to be learned q , W k , W v The matrix is ​​multiplied by a single sequence to obtain multiple sets of Q, K, and V vectors of the sequence; the attention coefficient matrix Attention(Q, K, V) can be expressed as:

[0035]

[0036] where d k It is the sequence dimension, K and V come from the same sequence, and Q comes from another sequence;

[0037] S3.3, calculate the attention coefficient matrices of multiple groups of remaining sequences to the current sequence through the formula in (S3.2), and linearly represent the current sequence with the remaining sequences. The current sequence contains global information, and the multi-head self-attention module will calculate all sequences;

[0038] S3.4. Flatten the encoded end feature map into a sequence for self-attention calculation, and reshape the data to the original feature map size.

[0039] By adopting the above technology, compared with the prior art, the beneficial effects of the present invention are as follows:

[0040] The present invention can effectively enable surgeons to have a more comprehensive understanding of the anatomical structure of the tumor before surgery, thereby helping them plan the path and range of tumor resection, on the one hand to completely remove renal cancer tissue, and on the other hand to remove less normal renal tissue and protect renal function; the method is highly versatile and can be applied to preoperative navigation of other tumor diseases after modifying and training the data set. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 This is the general process of navigation during tumor localization and segmentation surgery;

[0042] Figure 2 It is the CT image after resampling, CT value truncation, and z-score standardization of the original CT image in the exemplary case;

[0043] Figure 3 TransRFE-U 2 -Network diagram;

[0044] Figure 4 TransRFE-U 2 -Net Figure 2 An exemplary segmentation result of segmenting a CT image;

[0045] Figure 5 is based on Figure 4 An exemplary three-dimensional reconstruction model generated by the segmentation result;

[0046] Figure 6 It is a schematic diagram of the residual feature enhancement decoding module;

[0047] Figure 7 It is a schematic diagram of a multi-head self-attention module;

[0048] Figure 8 It is the multi-head self-attention data processing step. DETAILED DESCRIPTION

[0049] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments of the specification. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0050] On the contrary, the present invention covers any substitution, modification, equivalent method and scheme made on the essence and scope of the present invention as defined by the claims. Further, in order to make the public have a better understanding of the present invention, some specific details are described in detail in the detailed description of the present invention below. Those skilled in the art can fully understand the present invention without the description of these details.

[0051] See also Figure 1-Figure 8 , a method for intraoperative navigation based on accurate positioning and segmentation of renal tumor three-dimensional reconstruction, the specific process is as follows:

[0052] Step (1), obtaining an original CT scan image from a CT scan database of a patient's preoperative physical examination;

[0053] Step (2) resamples the acquired original CT data, truncates the CT value, and performs z-score normalization to make the boundaries between organs in the original image clearer. Figure 2The processed CT images are shown.

[0054] a. Resampling: The voxel spacing of three-dimensional data is very important for the learning of deep convolutional neural networks. The purpose of resampling is to unify the voxel spacing of sample data into the same spacing, so that it is the same as the voxel spacing of the target sample, so that the convolution operation in the deep neural network can extract similar features in the image. In the data preprocessing, the present invention adjusts the layer thickness distance in the z direction and the layer spacing in the x and y directions of all samples to (3.22mm, 1.62mm, 1.62mm), and the spatial resolution of all samples in the segmentation network model remains unchanged.

[0055] b. CT value cutoff: According to the imaging principle of CT, different tissues have different attenuation coefficients for X-rays. In clinical practice, the ratio of the attenuation coefficient of different tissues to the attenuation coefficient of water is used as the CT value. The following is the calculation process of the CT value of a certain tissue:

[0056]

[0057] CT M Indicates the CT value of a certain tissue, and the unit of value is HU. represents the attenuation coefficient of water, μ M Indicates the attenuation coefficient of a certain tissue. Different tissues have different attenuation coefficients and CT values. The greater the density of the tissue, the greater the corresponding attenuation coefficient and CT value. When segmenting the kidney and renal tumor, this feature can be used to distinguish the kidney from the surrounding organs and tissues with high similarity. By comparison, it is found that the values ​​of the kidney and renal tumor are roughly distributed between -200 and 400. In addition to the kidney, there are other tissues and organs in the CT scan image obtained, so the threshold method is used to exclude the interference of other tissues.

[0058] The specific operations are as follows:

[0059]

[0060] Among them, V in represents the original CT image, V out It represents the image after truncation after the threshold. After processing, the CT value of the entire sample is truncated between -79 and 304. The contrast of the entire CT image after threshold truncation is greatly improved;

[0061] cz-score normalization: When the convolutional neural network is used to segment kidney CT images, different feature information is distinguished by CT values. The range of CT values ​​of the same organ is relatively large, and there is a certain error.

[0062] When feature information is represented by CT values ​​with adjacent values, the network layer recognizes different feature information. Using the z-score normalization method, the distribution of CT values ​​is mapped to a smaller range, but the relationship between the values ​​remains the same as before processing and has not changed. The specific operation is as follows:

[0064]

[0065] Among them, is the CT value of a voxel in a single sample, V m is the mean CT value of all voxels in this sample, V sd is the standard deviation of the CT values ​​of all voxels in this sample, V i ′ is the corresponding value after z-score standardization;

[0066] Step (3): The CT image after resampling, CT value truncation, and z-score normalization is used as the trained TransRFE-U 2 -Net network is used to segment kidney tumors. Figure 3 Demonstrated TransRFE-U 2 -Net’s overall framework, which is based on U 2 -Net is used for dimensionality upgrade. After the input data is encoded and features are extracted, the global dependency is extracted through the multi-head self-attention module Trans-block of the global features. Then, the feature map size is restored and kidney tumor segmentation is performed through the decoding module combined with the residual feature enhancement module. Figure 4 The segmentation results of CT images are shown.

[0067] a. Upgrading Dimension U 2 -Net: Up-dimensional U 2 The overall structure of -Net is similar to the U-Net encoder-decoder structure. Each layer under each encoding and decoding path is a structural block. A sub-U-Net structure is nested in each structural block. That is, the entire model is a two-level nested U-shaped structure. Each structural block is called a Residual U-block (RSU). 2 -Net encoding path has 4 downsamplings, with a total of 5 encoders (En_1 to En_5), and in the decoding path there are 4 upsamplings, with a total of 4 decoders (De_1 to De_4). In the encoding stage, after each feature map passes through an RSU, the maximum pooling is used to achieve downsampling, and the size of the feature map after downsampling is reduced to 1 / 2 of the original feature map. In the decoding stage corresponding to each encoding stage, after each image passes through an RSU, linear interpolation is used to achieve upsampling, and the size of the feature map after upsampling is restored to twice the original feature map.

[0068] b. Residual feature enhancement decoding module: The residual feature enhancement module is combined with the improved 3D U 2 -Net, strengthens the ability to express features in the decoding path, improves the ability of the network to focus on areas of interest, and further improves the segmentation performance of the model. The specific structure is as follows Figure 5 As shown, the feature map X∈R B×C×D×H×W (B is the batch size, C is the number of channels, D, H, W are the depth, height and width of the feature map respectively), after being processed by the encoder, it enters the decoder De_4. The original decoding module fuses the input feature map with the feature map of the skip layer connection, and then expands the image size and reduces the number of channels through trilinear interpolation and a 3×3×3 convolution. The improved network adds a residual branch on the basis of the decoding module. When the feature map enters the branch, it is fully connected through the linear perception unit (MLP) through the feature map of the global maximum pooling module of B×C×1×1×1 based on three dimensions. At this time, the feature map has adjusted and corrected the weights according to the global pooling standard in three dimensions. After restoring the size, the proposed network will be easier to focus on the area of ​​interest. After activation with the Sigmoid function, the residual connection is made and the same upsampling operation as the original decoding module is performed. In order to enhance the expressive power of feature maps in the spatial domain, a feature enhancement module in the spatial domain is introduced. The upsampled feature maps are respectively passed through a channel-based global pooling module to obtain two feature maps of size B×1×2D×2H×2W. After the dimensions are concatenated, they are reduced to 1 through a convolutional layer. The Sigmoid function is used for activation and multiplied with the residual output of the second stage to obtain the final branch enhanced feature map. The spatial domain enhancement module aggregates the information of all channels and enhances the attention of the proposed network to the prospect information of kidneys and renal tumors. The feature enhancement module is used for each level of decoding module in the decoding path. The original decoding module has 4 levels, so the feature enhancement module also has 4 levels.

[0069] c. Multi-head self-attention module based on global features: This module is added to RFEU 2 -Net encoding end, its specific structure is as follows Figure 6 As shown. The entire 3D feature map obtained at the end of the encoding is flattened into a one-dimensional sequence. The specific implementation is to divide the 3D feature map into N image blocks (patch), then flatten each image block into a sequence Xp, map each sequence to a C-dimensional embedding space through linear projection E, and add position encoding information E pos , all image sequences after embedding processing ) can be expressed as:

[0070]

[0071] Where N is the number of blocks obtained after the image is sliced, that is, the sequence length (the deep feature map is divided into image blocks of 1×1×1 size, and the size of N is the product of the size of the deep feature map); C is the number of channels; l-1 is Figure 6 The level number in . Figure 6 GF-MSA sequence Perform self-attention calculation to obtain the contextual correlation between each sub-image block in the original feature map, and use multiple groups of W to be learned q , W k , W v The matrix is ​​multiplied by a single sequence to obtain multiple sets of Q, K, and V vectors of the sequence. The attention coefficient matrix Attention(Q, K, V) can be expressed as:

[0072]

[0073] where d k is the sequence dimension, K and V come from the same sequence, and Q comes from another sequence. The above formula is used to calculate the attention coefficient matrix of multiple groups of other sequences to the current sequence, and the current sequence is linearly represented by the other sequences. The current sequence contains global information, and the multi-head self-attention module will perform the above calculation on all sequences. Figure 6 In the example, MLP contains two fully connected layers, two dropout layers, and an activation function layer GELU. It performs global relationship processing on the sequence data after the self-attention calculation. Each GF-MSA and MLP use residual connection and perform layer normalization. The output of the first-level GF-MSA and MLP is used as the input of the second level. The matrix to be learned at each level W q , W k , W v In order to reflect the input-output relationship and parameter relationship of GF-MSA, the two-level GF-MSA is combined into a Transformer Layer. The final multi-head self-attention calculation formula can be expressed as:

[0074]

[0075] Flatten the feature map at the end of the encoding into a sequence for self-attention calculation, and reshape the data to the original feature map size. The processing steps are as follows Figure 7 As shown in the figure, it is named Trans-block, whose input is the output of the last encoder layer, and the output is connected to the decoder based on the residual feature enhancement module. Four layers of Transformer Layers (a total of 8 levels of GF-MSA) are selected and the input sequence channels are adjusted to 128 through 1×1×1 convolution. After the attention calculation is completed, it is adjusted to the number of channels of the original feature map;

[0076] Step (4), three-dimensionally reconstructing the segmentation result image, and obtaining a 3D model with clear tumor and kidney structures, and accurate size and position. Figure 5 The reconstruction results of an exemplary case are shown;

[0077] Step (5): The surgeon can determine the size and location of the tumor based on the 3D model and conduct a preoperative discussion on the tumor. By observing the model from multiple angles, the surgeon can determine the location and distribution of the tumor in the kidney from a morphological perspective, plan the location of the kidney defect, and formulate a feasible cutting plan.

[0078] Step (6) is to conduct a simulated surgery to test the feasibility of the surgical plan, assume the complex situations that may arise during the operation and prepare countermeasures, and finally determine the best surgical plan for tumor resection.

[0079] For example, Figure 5 An exemplary three-dimensional reconstruction model was displayed. During the simulated surgery, the patient was in the lateral position, and three punctures were placed in the affected lumbar region as the operating channel to enter the retroperitoneal space and open the perirenal fascia and fat capsule. The 3D model was used to locate the location of the tumor: such as the ventral / dorsal side, renal hilum / lateral edge, upper / lower pole, and then the surgical field was compared to accurately locate and find the tumor, and simply free the perirenal fat near the tumor. The 3D model intuitively shows the structural relationship between renal tumors and other renal tissues, determines the best path for cutting the tumor, and avoids excessive cutting of normal tissues during the cutting process, which may cause damage to other systems. When suturing the deep layer, single-layer suturing or double-layer suturing is selected based on the comparison of preoperative planning, the depth of the kidney defect, and the damage to the collecting system.

[0080] The present invention can be implemented in other specific embodiments within the scope of the idea and basic features of the present invention. Therefore, the above detailed description is only for illustration and does not limit the present invention at all levels. The scope of the present invention should be determined based on the reasonable interpretation of the additional invention claims, and all changes within the equivalent scope of the present invention belong to the scope of the present invention. The present invention is not limited to the embodiments described herein, but is given the widest scope that is the same as the principles and new features disclosed herein. In addition, an embodiment can be constructed by combining the invention claims that do not have a clear reference relationship in the invention claims, or by modifying the new invention claims after the application.

Claims

1. A method for intraoperative navigation based on accurate positioning and segmentation of renal tumor three-dimensional reconstruction, characterized in that: Includes the following step: (1) Obtaining original CT scan images from the CT scan database of the patient's preoperative physical examination; (2) resampling, CT value truncation, and z-score normalization are performed on the acquired original CT data to make the characteristic boundaries between organs in the original image clearer; (3) Using TransRFE-U 2 -Net performs renal tumor segmentation on the preprocessed CT images, TransRFE-U 2 -Net network is a 3D segmentation network based on global feature multi-head self-attention and residual feature enhancement; (4) using the segmentation result in step (3) to perform three-dimensional reconstruction on the image to obtain a 3D model of the tumor and kidney positions; (5) The doctor determines the size and location of the tumor based on the 3D reconstruction model and conducts preoperative discussions to formulate a surgical plan; (6) Determine the optimal cutting route through simulated surgery.

2. The intraoperative navigation method based on accurate positioning and segmentation of renal tumor three-dimensional reconstruction according to claim 1 is characterized in that In step (2), the specific process of resampling the acquired original CT data is as follows: In data preprocessing, the layer thickness in the z direction and the layer spacing in the x and y directions of all samples were adjusted, and the spatial resolution of all samples in the segmentation network model remained unchanged.

3. The intraoperative navigation method based on accurate positioning and segmentation of renal tumor three-dimensional reconstruction according to claim 2 is characterized in that In step (2), the specific process of CT value truncation is as follows: According to the imaging principle of CT, different tissues have different attenuation coefficients for X-rays. The ratio of the attenuation coefficients of different tissues to the attenuation coefficient of water is used as the CT value. The following is the CT value calculation process: CT M Indicates the CT value of a certain tissue. The unit of value is HU, μ M It indicates the attenuation coefficient of a certain tissue. Different tissues have different attenuation coefficients and CT values. Represents the attenuation coefficient of water. When segmenting the kidney and renal tumor, the kidney is distinguished from surrounding organ tissues based on the fact that the greater the density of the tissue, the greater the corresponding attenuation coefficient and CT value; The CT scan images obtained contain other tissues and organs besides the kidneys, so the threshold method is used to eliminate the interference of other tissues. The specific operation is as follows: Among them, V in represents the original CT image, V out Represents the image after truncating after thresholding.

4. The intraoperative navigation method based on accurate positioning and segmentation of renal tumor three-dimensional reconstruction according to claim 1 is characterized in that In step (2), the specific process of z-score standardization is as follows: When the convolutional neural network segments kidney CT images, different feature information is distinguished by CT values. The CT value range of the same organ is large and there are errors. When some of the same feature information is represented by CT values ​​with adjacent values, the network layer recognizes different feature information. Using the z-score standardization method, the distribution of CT values ​​is mapped to a smaller range, but the relationship between the values ​​is the same as before processing and has not changed. The specific operation is as follows: Among them, V i is the CT value of a voxel in a single sample, V m is the mean CT value of all voxels in this sample, V sd is the standard deviation of the CT values ​​of all voxels in this sample, V i ′ is the corresponding value after z-score standardization.

5. The intraoperative navigation method based on accurate positioning and segmentation of renal tumor three-dimensional reconstruction according to claim 1 is characterized in that In step (3), TransRFE-U 2 -Net network is in U 2 -Net is used for dimensionality upgrade. After the input data is encoded and features are extracted, the global dependencies are extracted through the multi-head self-attention module Trans-block of the global features. Then, the feature map size is restored and renal tumor segmentation is performed through the decoding module combined with the residual feature enhancement module.

6. The intraoperative navigation method based on accurate positioning and segmentation of renal tumor three-dimensional reconstruction according to claim 5 is characterized in that: Will U 2 After the -Net is upgraded to three dimensions, each layer under each encoding and decoding path is a structural block, and each structural block is nested with a sub-U-Net structure, that is, the entire model is a two-level nested U-shaped structure, and each structural block is called a Residual U-block; three-dimensional U 2 -Net encoding path has four downsamplings, with a total of five encoders, and in the decoding path there are four corresponding upsamplings, with a total of four decoders; in the encoding stage, after each feature map passes through an RSU, the maximum pooling is used to achieve downsampling, and the size of the feature map after downsampling is reduced to 1 / 2 of the original feature map; in the decoding stage corresponding to each encoding stage, after each image passes through an RSU, linear interpolation is used to achieve upsampling, and the size of the feature map after upsampling is restored to twice the original feature map.

7. The intraoperative navigation method based on accurate positioning and segmentation of renal tumor three-dimensional reconstruction according to claim 5 is characterized in that In the step (3), by adding a residual branch, when the feature map enters the branch, it is fully connected through the feature map based on three dimensions through the linear perception unit MLP, and is activated by the Sigmoid function to make a residual connection and perform the same upsampling operation as the original decoding module; in order to enhance the expressiveness of the feature map in the spatial domain, a feature enhancement module in the spatial domain is introduced, and the upsampled feature map is passed through a channel-based global pooling module to obtain two feature maps, which are concatenated and reduced to one through a convolutional layer, activated by the Sigmoid function and multiplied with the residual output of the second stage to obtain the final branch enhanced feature map. The spatial domain enhancement module aggregates the information of all channels and enhances the attention of the proposed network to the prospect information of the kidney and renal tumor; the feature enhancement module is used for each level of decoding module in the decoding path.

8. The method for intraoperative navigation based on accurate positioning and segmentation of renal tumor three-dimensional reconstruction according to claim 5 is characterized in that In step (3), a multi-head self-attention module based on global features is added to RFEU 2 -Net encoding end, the specific process is as follows: S3.

1. Flatten the entire 3D feature map obtained at the end of the encoding into a one-dimensional sequence. Specifically, the 3D feature map is divided into N image blocks, and each image block is flattened into a sequence Xp. Each sequence is mapped to a C-dimensional embedding space through a linear projection E, and the position encoding information E is added. pos , all image sequences after embedding processing ) can be expressed as: Where N is the number of blocks obtained after the image is sliced, that is, the sequence length; C is the number of channels; l-1 is the level number; S3.

2. Sequence Perform self-attention calculation to obtain the contextual correlation between each sub-image block in the original feature map, and use multiple groups of W to be learned q , W k , W v The matrix is ​​multiplied by a single sequence to obtain multiple sets of Q, K, and V vectors of the sequence; the attention coefficient matrix Attention(Q, K, V) can be expressed as: where d k It is the sequence dimension, K and V come from the same sequence, and Q comes from another sequence; S3.3, calculate the attention coefficient matrices of multiple groups of remaining sequences to the current sequence through the formula in (S3.2), and linearly represent the current sequence with the remaining sequences. The current sequence contains global information, and the multi-head self-attention module will calculate all sequences; S3.

4. Flatten the encoded end feature map into a sequence for self-attention calculation, and reshape the data to the original feature map size.

Images