Method and device for evaluating invasion degree of tumor on nerve and blood vessel bundles

Abstract

The invention discloses a method and a device for evaluating the degree of invasion of a nerve and blood vessel bundle by a tumor, and the method comprises the steps: obtaining two-parameter resonance imaging of a patient, and extracting at least two modal data in the two-parameter resonance imaging; performing data alignment on the at least two types of modal data; inputting the at least two aligned modal data into a trained segmentation model, and outputting mask data including tumor and vascular nerve tract areas; converting the mask data into point cloud data; and calculating the geometrical relationship between the tumor and the vascular nerve tract region in the point cloud data, and judging the invasion degree between the tumor and the vascular nerve tract region based on the geometrical relationship between the tumor and the vascular nerve tract region and a preset invasion judgment rule. According to the invention, high-precision automatic segmentation of the tumor and vascular nerve tract area of the prostate cancer can be realized.

Description

Technical Field

[0001] The present invention relates to the technical field of tumor identification, and in particular to a method and a device for evaluating the degree of tumor invasion of neurovascular bundles. Background Art

[0002] Prostate cancer is one of the most common solid tumors in men. Radical surgery, such as laparoscopic radical prostatectomy and robot-assisted radical prostatectomy, has become the standard treatment for localized and locally advanced prostate cancer. Urinary incontinence is one of the most common complications after radical prostatectomy, which seriously affects the quality of life of patients. However, whether the vascular nerve bundle is preserved during surgery is crucial for postoperative urinary control function. Nerve-preserving "intrafascial resection" is a technique that preserves the vascular nerve bundles around the prostate while removing the prostate. It is suitable for tumors confined to the prostate capsule (clinical stage T2 and below). This approach can minimize damage to nerves and blood vessels, which is crucial for maintaining postoperative erectile function and urinary control function. In contrast, extrafascial resection is a complete removal of the prostate and its surrounding tissues. It is usually suitable for more aggressive advanced tumors, but may lead to a higher incidence of postoperative complications such as urinary incontinence and erectile dysfunction.

[0003] Prostate MRI images have high soft tissue resolution. Biparametric MRI (bpMRI) combines multiple imaging sequences, including T2-weighted imaging (T2WI), diffusion-weighted imaging (DWI), and apparent diffusion coefficient imaging (ADC), to enhance the detection of prostate cancer. Artificial intelligence algorithms are widely used in the field of semantic segmentation of magnetic resonance images. They mainly use deep learning technology based on convolutional neural networks to automatically learn image features and achieve accurate recognition and segmentation of different tissues or structures in MRI. In prostate cancer, these algorithms can help doctors accurately identify tumor boundaries and their relationship with neurovascular bundles, providing important basis for surgical decision-making.

[0004] Current clinical guidelines distinguish between intrafascial and extrafascial surgical approaches based on the TNM stage of the tumor and the spatial relationship between the tumor and NVB. The appropriate surgical approach is determined by evaluating the size, location, and involvement of key structures of the tumor. However, manual film reading is a labor-intensive task, and it is difficult to achieve three-dimensional perspective analysis. Therefore, there is an urgent need for an accurate and reliable automated quantitative analysis technology solution to improve the accuracy of surgical decision-making. Summary of the invention

[0005] The present invention provides a method and device for evaluating the degree of tumor invasion of neurovascular bundles, which are used to improve the recognition accuracy of tumors and neural structures.

[0006] According to one aspect of the present invention, a method for evaluating the extent of tumor invasion of neurovascular bundles is provided, comprising:

[0007] Acquire dual-parameter resonance imaging of the patient, and extract at least two modality data from the dual-parameter resonance imaging;

[0008] Performing data alignment on at least two of the modal data;

[0009] Inputting the aligned at least two modality data into a trained segmentation model, and outputting mask data including the tumor and the vascular nerve bundle area;

[0010] Converting the mask data into point cloud data;

[0011] The geometric relationship between the tumor and the vascular nerve bundle region in the point cloud data is calculated, and the degree of invasion between the tumor and the vascular nerve bundle region is determined based on the geometric relationship between the tumor and the vascular nerve bundle region and a preset invasion determination rule.

[0012] Optionally, before inputting the aligned at least two modality data into the trained segmentation model, the method further includes:

[0013] Acquire multiple dual-parameter resonance images;

[0014] extracting at least two modality data of a plurality of dual-parameter resonance imaging respectively;

[0015] Performing data alignment on at least two of the modal data;

[0016] Annotate the tumor area and the blood vessel nerve bundle area in at least two of the modal data, and construct a data set by taking at least two of the modal data of dual-parameter resonance imaging as a sample;

[0017] The segmentation model constructed based on the data set training is terminated until the number of iterations reaches a preset iteration preset or the model parameters converge, thereby obtaining the trained segmentation model.

[0018] Optionally, the at least two modality data include any two or three of T2-weighted images, diffusion-weighted images and apparent diffusion coefficient images.

[0019] Optionally, it also includes:

[0020] The sample data in the data set are preprocessed, and the preprocessing at least includes clipping the sample data to obtain data in a non-zero area, resampling the sample data, and normalizing the sample data.

[0021] Optionally, the segmentation model is a three-dimensional full-resolution image model, i.e., a 3D U-Net network;

[0022] The 3D U-Net network includes multiple different stages to gradually extract features of sample data.

[0023] Optionally, the 3D U-Net network includes 7 different stages, each stage is responsible for extracting features at different levels, gradually from low-level features to high-level features; the number of feature channels in each stage is 32, 64, 128, 256, 320, 320, 320 respectively;

[0024] The convolution kernel sizes of each stage are: the convolution kernel size of the first stage is 1, 3, 3; the convolution kernel size of the second stage is 1, 3, 3; the convolution kernel size of the third to seventh stages is 3, 3, 3; the stride of each stage is set as follows: the stride of the first stage is 1, 1, 1; the stride of the second stage is 1, 2, 2; the stride of the third stage is 1, 2, 2; the stride of the fourth stage is 2, 2, 2; the stride of the fifth stage is 2, 2, 2; the stride of the sixth stage is 1, 2, 2; the stride of the seventh stage is 1, 2, 2;

[0025] The encoder and decoder stages have 2 convolutional layers each, no Dropout layers are used, and the non-linear activation function used is specified.

[0026] Optionally, it also includes:

[0027] The probability map generated by the segmentation model is converted into a binary mask through a threshold, and is used to remove artifacts and noise in the binary mask based on dilation and erosion operations, thereby obtaining optimized mask data.

[0028] Optionally, converting the mask data into point cloud data comprises:

[0029] Extract the 3D surface mesh of the non-zero area from the mask data, including vertices, faces, and normal vectors, specifically:

[0030] Based on the three-dimensional mask data V(x, y, z) obtained by segmentation, the contour threshold T is defined, the contour surface is constructed, and the three-dimensional mask is segmented into n cubes Ci, each cube consists of 8 fixed points;

[0031] C i = {V(x i ,y i ,zi ),V(x i+1 ,y i ,z i ),V(x i ,y i+1 ,z i ),V(x i+1 ,y i ,z i+1 ),V(x i+1 ,y i+1 ,z i ),V(x i ,y i+1 ,z i+1 )V(x i+1 ,y i+1 ,z i+1 )}

[0032] Among them, x i ,y i , z i Represents the x-axis, y-axis, and z-axis coordinates of the midpoint of the cube; for each vertex V of the cube i ∈C i , determine whether it is greater than the contour threshold T:

[0033]

[0034] Generate binary state index: S = S 0 S 1 S 2 S 3 S 4 S 5 S 6 S 7

[0035] Using a lookup table, determine the boundary connectivity scheme in the cube based on the state index S:

[0036] B(S)=LookupTable[S]

[0037] For each edge e connecting two vertices k , perform linear interpolation to obtain points on the isosurface:

[0038]

[0039] Where V a , V b is with edge e k The values ​​of the two related vertices will be interpolated to get the point P k Connect to form a triangular surface mesh model;

[0040] The number of triangles in all triangular surface mesh models is obtained, and the corresponding number of point clouds is calculated based on the number of triangles. The point cloud is extracted from the triangular mesh using the Poisson sampling method to obtain point cloud data.

[0041] Optionally, the geometric relationship between the tumor and the vascular nerve bundle region in the calculated point cloud data is determined based on the geometric relationship between the tumor and the vascular nerve bundle region and a preset invasion determination rule to determine the invasion degree between the tumor and the vascular nerve bundle region, including:

[0042] Calculating, based at least on the point cloud data, the geometric centers of the tumor and the vascular nerve bundle region, the closest distance between the surface of the tumor and the surface of the vascular nerve bundle region, and the closest geometric center distance between the tumor and the vascular nerve bundle region;

[0043] The degree of invasion between the tumor and the vascular nerve bundle area is judged at least based on the comparison result of the closest distance from the surface of the tumor to the surface of the vascular nerve bundle area and a preset closest distance threshold, and the comparison result between the closest geometric center distance between the tumor and the vascular nerve bundle area and a preset distance threshold.

[0044] According to another aspect of the present invention, there is provided a device for evaluating the extent of tumor invasion of neurovascular bundles, comprising:

[0045] A data extraction unit, used to obtain dual-parameter resonance imaging of the patient and extract at least two modality data in the dual-parameter resonance imaging;

[0046] A data alignment unit, used for aligning at least two types of modal data;

[0047] A segmentation unit, used for inputting the aligned at least two modality data into a trained segmentation model, and outputting mask data including the tumor and the vascular nerve bundle area;

[0048] A point cloud conversion unit, used for converting the mask data into point cloud data;

[0049] A judgment unit is used to calculate the geometric relationship between the tumor and the vascular nerve bundle area in the point cloud data, and judge the degree of invasion between the tumor and the vascular nerve bundle area based on the geometric relationship between the tumor and the vascular nerve bundle area and a preset invasion judgment rule.

[0050] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0051] at least one processor; and

[0052] a memory communicatively connected to the at least one processor; wherein,

[0053] The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can perform the method for assessing the degree of tumor invasion of the neurovascular bundle as described in any embodiment of the present invention.

[0054] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions for enabling a processor to implement the method for assessing the degree of tumor invasion of neurovascular bundles as described in any embodiment of the present invention when the computer instructions are executed.

[0055] The technical solution of the embodiment of the present invention, by combining preoperative dual-parameter resonance imaging of prostate cancer with deep learning technology, can achieve high-precision automatic segmentation of the tumor and vascular nerve bundle area of ​​prostate cancer, thereby providing reliable preoperative guidance for surgeons to select radical prostatectomy plans. This method not only improves the recognition accuracy of tumors and nerve structures, but also optimizes surgical planning, reduces the risk of nerve damage, and effectively improves the patient's postoperative urinary control function and quality of life. In addition, through three-dimensional reconstruction and point cloud data analysis, the spatial relationship between the tumor and related anatomical structures can be more intuitively displayed, assisting in guiding surgical methods, and providing an important basis for personalized treatment of prostate cancer patients.

[0056] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0058] Figure 1 is a flow chart of a method for evaluating the degree of tumor invasion of neurovascular bundles provided according to Embodiment 1 of the present invention;

[0059] Figure 2 is a flow chart of a method for evaluating the degree of tumor invasion of neurovascular bundles provided according to Embodiment 2 of the present invention;

[0060] Figure 3 is a structural diagram of a device for evaluating the degree of tumor invasion of neurovascular bundles provided according to Embodiment 2 of the present invention;

[0061] Figure 4 It is a schematic diagram of the structure of an electronic device for implementing the method for evaluating the degree of tumor invasion of neurovascular bundles according to an embodiment of the present invention. DETAILED DESCRIPTION

[0062] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0063] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0064] Embodiment 1

[0065] Figure 1 A flowchart of a method, device, electronic device, and medium for evaluating the degree of tumor invasion of neurovascular bundles is provided for the first embodiment of the present invention. Figure 1 As shown, the method includes:

[0066] S101 . Obtain dual-parameter resonance imaging of a patient, and extract at least two modality data from the dual-parameter resonance imaging.

[0067] The current patient's prostate magnetic resonance scan DICOM format image data can be obtained. In this process, the dual-parameter imaging data bpMRI is manually cleaned to ensure its quality and reliability. In particular, three key imaging modality data are extracted from the acquired dual-parameter imaging data, including: T2-weighted image (T2WI), diffusion-weighted image (DWI) and apparent diffusion coefficient image (ADC).

[0068] S102, aligning at least two types of modal data.

[0069] Spatial data alignment is performed on at least two types of data among T2-weighted image (T2WI), diffusion-weighted image (DWI) and apparent diffusion coefficient image, specifically including:

[0070] First, create a resampling filter to resample the image data of the three sequences; secondly, the T2-weighted image can be used as a reference for the diffusion-weighted image and the apparent diffusion coefficient image, so that the diffusion-weighted image and the apparent diffusion coefficient image are aligned with the T2-weighted image in space. Then, the diffusion-weighted image and the apparent diffusion coefficient image are interpolated to obtain a smooth result and reduce possible image distortion; the pixel interval of the diffusion-weighted image and the apparent diffusion coefficient image is set to be consistent with that of the T2-weighted image to ensure consistent image resolution; the direction matrix of the diffusion-weighted image and the apparent diffusion coefficient image is set to be consistent with that of the T2-weighted image to ensure consistent positioning of the multi-sequence images in space; the origin of the diffusion-weighted image and the apparent diffusion coefficient image is set to be consistent with that of the T2-weighted image to ensure spatial alignment; the size of the diffusion-weighted image and the apparent diffusion coefficient image is set to be consistent with that of the T2-weighted image to ensure consistent dimensions.

[0071] S103, inputting the aligned at least two modality data into a trained segmentation model, and outputting mask data including the tumor and blood vessel nerve bundle area.

[0072] The segmentation model is used to segment the image, and in this embodiment is used to segment the mask data of the tumor and the blood vessel nerve bundle region in at least two modal data.

[0073] S104: Convert the mask data into point cloud data.

[0074] This embodiment can convert the mask data of the segmented tumor and blood vessel nerve bundle area into point cloud data, and can also extract the spatial resolution information of the point cloud data and the three-dimensional surface network, including vertices, faces and normal vectors.

[0075] S105, calculating the geometric relationship between the tumor and the vascular nerve bundle region in the point cloud data, and judging the degree of invasion between the tumor and the vascular nerve bundle region based on the geometric relationship between the tumor and the vascular nerve bundle region and a preset invasion judgment rule.

[0076] The geometric relationship between the tumor and the vascular nerve bundle area can be calculated based on the spatial resolution information of the point cloud data and the three-dimensional surface network, and the degree of invasion between the tumor and the vascular nerve bundle area can be determined based on the geometric relationship between the tumor and the vascular nerve bundle area and the preset invasion judgment rules.

[0077] The technical solution of the embodiment of the present invention, by combining preoperative dual-parameter resonance imaging of prostate cancer with deep learning technology, can achieve high-precision automatic segmentation of the tumor and vascular nerve bundle area of ​​prostate cancer, thereby providing reliable preoperative guidance for surgeons to select radical prostatectomy plans. This method not only improves the recognition accuracy of tumors and nerve structures, but also optimizes surgical planning, reduces the risk of nerve damage, and effectively improves the patient's postoperative urinary control function and quality of life. In addition, through three-dimensional reconstruction and point cloud data analysis, the spatial relationship between the tumor and related anatomical structures can be more intuitively displayed, assisting in guiding surgical methods, and providing an important basis for personalized treatment of prostate cancer patients.

[0078] Embodiment 2

[0079] Figure 2 A flowchart of a method for evaluating the degree of tumor invasion of neurovascular bundles provided in Example 2 of the present invention is shown in FIG. Figure 2 As shown, the method includes:

[0080] S201, acquiring multiple dual-parameter resonance images.

[0081] It should be noted that multiple dual-parameter resonance imaging data of prostate magnetic resonance scans in DICOM format can be obtained from the database. At least two key imaging modalities can be extracted from the acquired dual-parameter resonance imaging data, which may include: T2-weighted image (T2WI), diffusion-weighted image (DWI) and apparent diffusion coefficient image (ADC). And all the original DICOM format data are converted into a unified format, such as NIfTI (.nii.gz) format, that is, the medical image data is standardized to facilitate subsequent processing. After the conversion is completed, the three types of image data are read for subsequent analysis.

[0082] S202: extract at least two modality data of a plurality of dual-parameter resonance imaging respectively.

[0083] In one embodiment, the at least two modality data include any two or three of T2-weighted images, diffusion-weighted images, and apparent diffusion coefficient images.

[0084] S203, preprocessing the sample data in the data set, wherein the preprocessing at least includes clipping the sample data to obtain data in a non-zero area, resampling the sample data, and normalizing the sample data.

[0085] Resampling filters can be created to resample three types of data: T2-weighted images, diffusion-weighted images, and apparent diffusion coefficient images; the intervals of all images are normalized to the median interval of the dataset.

[0086] In addition, data enhancement techniques can be used to enhance the sample data in the data set, including random rotation, random scaling, random elastic transformation, gamma correction and mirroring, to improve the generalization ability of the model. And the data in the non-zero area can also be cropped to reduce the consumption of computing resources.

[0087] S204: align at least two types of modal data.

[0088] The T2-weighted image can be used as a reference for the diffusion-weighted image and the apparent diffusion coefficient image so that the diffusion-weighted image and the apparent diffusion coefficient image are aligned with the T2-weighted image in space; the diffusion-weighted image and the apparent diffusion coefficient image are interpolated to obtain a smooth result and reduce possible image distortion; the pixel interval of the diffusion-weighted image and the apparent diffusion coefficient image is set to be consistent with that of the T2-weighted image to ensure consistent image resolution; the direction matrix of the diffusion-weighted image and the apparent diffusion coefficient image can be set to be consistent with that of the T2-weighted image to ensure consistent spatial positioning of multiple sequence images; the origin of the diffusion-weighted image and the apparent diffusion coefficient image is set to be consistent with that of the T2-weighted image to ensure spatial alignment; the size of the diffusion-weighted image and the apparent diffusion coefficient image is set to be consistent with that of the T2-weighted image to ensure consistent dimensions.

[0089] S205 , annotating the tumor region and the blood vessel nerve bundle region in at least two of the modal data, and constructing a data set by taking at least two of the modal data of dual-parameter resonance imaging as a sample.

[0090] The tumor area and the vascular nerve bundle area can be annotated layer by layer, for example, the tumor area and the vascular nerve bundle area in the three modal data can be annotated, and the aligned three modal data can be used as a sample to construct a data set with multiple sample data.

[0091] S206, training the constructed segmentation model based on the data set until the number of iterations reaches a preset iteration preset or the model parameters converge, and then terminating the training to obtain the trained segmentation model.

[0092] In one embodiment, the 3D U-Net network includes a plurality of different stages to gradually extract features of the sample data.

[0093] The 3D U-Net network consists of 7 different stages, each of which is responsible for extracting features at different levels, gradually from low-level features to high-level features; the number of feature channels in each stage is 32, 64, 128, 256, 320, 320, 320 respectively;

[0094] The convolution kernel sizes of each stage are: the convolution kernel size of the first stage is 1, 3, 3; the convolution kernel size of the second stage is 1, 3, 3; the convolution kernel size of the third to seventh stages is 3, 3, 3; the stride of each stage is set as follows: the stride of the first stage is 1, 1, 1; the stride of the second stage is 1, 2, 2; the stride of the third stage is 1, 2, 2; the stride of the fourth stage is 2, 2, 2; the stride of the fifth stage is 2, 2, 2; the stride of the sixth stage is 1, 2, 2; the stride of the seventh stage is 1, 2, 2;

[0095] The encoder and decoder stages have 2 convolutional layers each, no Dropout layers are used, and the non-linear activation function used is specified.

[0096] During the training process, the aligned three modal data are used as the input of the segmentation model, and the annotated mask is considered as the label. The data set is divided into training set and test set according to the preset ratio, and a five-fold cross validation can be used. The parameters are initialized and the number of training epochs is set to 1000. The training batch size is automatically allocated, batch size: 2, patchsize: [16, 320, 320]. The loss function combines the Dice loss Loss dice and cross entropy loss Loss CE , defined as follows:

[0097] Loss total =Loss dice +Loss CE

[0098]

[0099] Loss CE =-Σ(Y log(P)+(1-Y)log(1-P))

[0100] Among them, u is the Softmax probability output, v is the hard-coded ground truth, y represents the true label, and p represents the predicted probability of the model.

[0101] It should be noted that in order to increase the stability of the network, when sampling patches, it is ensured that more than 1 / 3 of the pixels in a batch are foreground pixels.

[0102] Alternatively, you can use the Adam optimizer with an initial learning rate of 0.01 and a weight decay (L2 regularization) of 0.00003 (3e -5 Each training cycle (epoch) is performed for 250 iterations, each validation cycle is performed for 50 iterations, and a total of 1000 epochs are trained.

[0103]

[0104] Among them, θ t+1 represents the updated weight parameter; θ t Represents the convolution weight parameter before updating; lr is the current learning rate; represents the bias-corrected first-order moment estimate; is the bias-corrected second moment estimate.

[0105] When the training epoch limit is reached, or when the exponential moving average loss of the validation set decreases by no more than 5e within 60 epochs -3 , or the learning rate is reduced to 1e -6 , the training will stop.

[0106] S207 , obtaining dual-parameter resonance imaging of the patient, and extracting at least two modality data from the dual-parameter resonance imaging.

[0107] The current patient's prostate magnetic resonance scan DICOM format image data can be obtained. In this process, the dual-parameter imaging data bpMRI is manually cleaned to ensure its quality and reliability. In particular, three key imaging modality data are extracted from the acquired dual-parameter imaging data, including: T2-weighted image (T2WI), diffusion-weighted image (DWI) and apparent diffusion coefficient image (ADC).

[0108] S208, preprocessing the sample data in the data set, wherein the preprocessing at least includes clipping the sample data to obtain data in a non-zero area, resampling the sample data, and normalizing the sample data.

[0109] Resampling filters can be created to resample three types of data: T2-weighted images, diffusion-weighted images, and apparent diffusion coefficient images; the intervals of all images are normalized to the median interval of the dataset.

[0110] In addition, data enhancement techniques can be used to enhance the sample data in the data set, including random rotation, random scaling, random elastic transformation, gamma correction and mirroring, to improve the generalization ability of the model. And the data in the non-zero area can also be cropped to reduce the consumption of computing resources.

[0111] S209: align at least two types of modal data.

[0112] Spatial data alignment is performed on at least two types of data among T2-weighted image (T2WI), diffusion-weighted image (DWI) and apparent diffusion coefficient image, specifically including:

[0113] First, create a resampling filter to resample the image data of the three sequences; secondly, the T2-weighted image can be used as a reference for the diffusion-weighted image and the apparent diffusion coefficient image, so that the diffusion-weighted image and the apparent diffusion coefficient image are aligned with the T2-weighted image in space. Then, interpolate the diffusion-weighted image and the apparent diffusion coefficient image to obtain a smooth result and reduce possible image distortion; set the pixel interval of the diffusion-weighted image and the apparent diffusion coefficient image to be consistent with that of the T2-weighted image to ensure consistent image resolution; set the direction matrix of the diffusion-weighted image and the apparent diffusion coefficient image to be consistent with that of the T2-weighted image to ensure consistent spatial positioning of the multi-sequence images;

[0114] The origin of the diffusion-weighted image and the apparent diffusion coefficient image was set to be consistent with the origin of the T2-weighted image to ensure spatial alignment; the size of the diffusion-weighted image and the apparent diffusion coefficient image was set to be consistent with the size of the T2-weighted image to ensure dimensional consistency.

[0115] S210, inputting the aligned at least two modality data into a trained segmentation model, and outputting mask data including the tumor and blood vessel nerve bundle area.

[0116] In one embodiment, the method further includes converting the probability map generated by the segmentation model into a binary mask through a threshold, and removing artifacts and noise in the binary mask based on dilation and erosion operations, thereby obtaining optimized mask data.

[0117] Specifically, the generated probability map can be converted into a binary mask (i.e., the segmentation of background and foreground) through a threshold, and used to remove small artifacts and noise through dilation and erosion operations. The dilation operation can connect separated areas, while the erosion operation can remove small objects. Analyze the connected components in the segmentation results, and remove connected areas that are smaller than the set threshold (remove areas smaller than a certain volume). Smooth the segmentation results.

[0118] S211, converting the mask data into point cloud data.

[0119] In one embodiment, specifically, extracting a three-dimensional surface mesh of a non-zero area from mask data, including vertices, faces, and normal vectors, specifically:

[0120] Based on the three-dimensional mask data V(x,y,z) obtained by segmentation, the contour threshold T is defined and the contour surface is constructed.

[0121] Divide the three-dimensional mask into n cubes Ci, each cube consists of 8 fixed points;

[0122] C i = {V(x i ,yi ,z i ),V(x i+1 ,y i ,z i ),V(x i ,y i+1 ,z i ),V(x i+1 ,y i ,z i+1 ),V(x i+1 ,y i+1 ,z i ),V(x i ,y i+1 ,z i+1 )V(x i+1 ,y i+1 ,z i+1 )}

[0123] Among them, x i ,y i , z i Represents the x-axis, y-axis, and z-axis coordinates of the midpoint of the cube; for each vertex V of the cube i ∈C i , determine whether it is greater than the contour threshold T:

[0124]

[0125] Generate binary state index: S = S 0 S 1 S 2 S 3 S 4 S 5 S 6 S 7

[0126] Using a lookup table, determine the boundary connectivity scheme in the cube based on the state index S:

[0127] B(S)=LookupTable[S]

[0128] For each edge e connecting two vertices k , perform linear interpolation to obtain points on the isosurface:

[0129]

[0130] Where V a , V b is with edge e k The values ​​of the two related vertices will be interpolated to get the point P k Connect to form a triangular surface mesh model; and export the STL model.

[0131] By reading the STL model file, you can get the number of triangles in all triangular surface mesh models, and calculate the corresponding number of point clouds based on the number of triangles. Use the Poisson sampling method to extract point clouds from the triangular mesh to get point cloud data, and save the result as a PCD format file.

[0132] S212, calculating the geometric relationship between the tumor and the vascular nerve bundle area in the point cloud data, and judging the degree of invasion between the tumor and the vascular nerve bundle area based on the geometric relationship between the tumor and the vascular nerve bundle area and preset invasion judgment rules.

[0133] In one embodiment, specifically: the geometric centers of the tumor and the vascular nerve bundle area, the closest distance between the surface of the tumor and the surface of the vascular nerve bundle area, and the closest geometric center distance between the tumor and the vascular nerve bundle area are calculated at least based on point cloud data.

[0134] The degree of invasion between the tumor and the vascular nerve bundle area is judged at least based on the comparison result of the closest distance from the surface of the tumor to the surface of the vascular nerve bundle area and a preset closest distance threshold, and the comparison result between the closest geometric center distance between the tumor and the vascular nerve bundle area and a preset distance threshold.

[0135] For example, when the minimum distance between a point on the tumor surface and a point on the surface of the vascular nerve bundle area is less than 1 mm, the point is considered to be a point where the tumor invades the vascular nerve bundle area. The conditions for intrafascial radical prostatectomy can be determined by determining whether the point in the tumor area overlaps with the point in the vascular nerve bundle area.

[0136] Embodiment 3

[0137] Figure 3 This is a schematic diagram of the structure of a device for evaluating the degree of tumor invasion of neurovascular bundles provided in Example 3 of the present invention. Figure 3 As shown, the device comprises:

[0138] A data extraction unit 301 is used to obtain dual-parameter resonance imaging of a patient and extract at least two modality data from the dual-parameter resonance imaging;

[0139] A data alignment unit 302, configured to align at least two types of modal data;

[0140] A segmentation unit 303, used for inputting the aligned at least two modality data into a trained segmentation model, and outputting mask data including the tumor and the blood vessel nerve bundle area;

[0141] A point cloud conversion unit 304, used to convert the mask data into point cloud data;

[0142] The judgment unit 305 is used to calculate the geometric relationship between the tumor and the vascular nerve bundle area in the point cloud data, and judge the degree of invasion between the tumor and the vascular nerve bundle area based on the geometric relationship between the tumor and the vascular nerve bundle area and the preset invasion judgment rules.

[0143] The device for assessing the degree of tumor invasion of neurovascular bundles provided in the embodiment of the present invention can execute the method for assessing the degree of tumor invasion of neurovascular bundles provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0144] Embodiment 4

[0145] Figure 4 A schematic diagram of the structure of an electronic device 10 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0146] like Figure 4 As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0147] A number of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0148] The processor 11 may be a variety of general and / or dedicated processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as a method for assessing the extent of tumor invasion of neurovascular bundles.

[0149] In some embodiments, a method for assessing the extent of tumor invasion of neurovascular bundles may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the method for assessing the extent of tumor invasion of neurovascular bundles described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to perform a method for assessing the extent of tumor invasion of neurovascular bundles in any other appropriate manner (e.g., by means of firmware).

[0150] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0151] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0152] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0153] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0154] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0155] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0156] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0157] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for evaluating the extent of tumor invasion of neurovascular bundles, characterized in that: include: Acquire dual-parameter resonance imaging of the patient, and extract at least two modality data from the dual-parameter resonance imaging; Performing data alignment on at least two of the modal data; Inputting the aligned at least two modality data into a trained segmentation model, and outputting mask data including the tumor and the vascular nerve bundle area; Converting the mask data into point cloud data; The geometric relationship between the tumor and the vascular nerve bundle region in the point cloud data is calculated, and the degree of invasion between the tumor and the vascular nerve bundle region is determined based on the geometric relationship between the tumor and the vascular nerve bundle region and a preset invasion determination rule.

2. The method for evaluating the degree of tumor invasion of neurovascular bundles according to claim 1, characterized in that: Before inputting the aligned at least two modality data into the trained segmentation model, the method further includes: Acquire multiple dual-parameter resonance images; extracting at least two modality data of a plurality of dual-parameter resonance imaging respectively; Performing data alignment on at least two of the modal data; Annotate the tumor area and the blood vessel nerve bundle area in at least two of the modal data, and construct a data set by taking at least two of the modal data of dual-parameter resonance imaging as a sample; The segmentation model constructed based on the data set training is terminated until the number of iterations reaches a preset iteration preset or the model parameters converge, thereby obtaining the trained segmentation model.

3. The method for evaluating the degree of tumor invasion of neurovascular bundles according to claim 2, characterized in that: The at least two modality data include any two or three of T2-weighted images, diffusion-weighted images, and apparent diffusion coefficient images.

4. The method for evaluating the degree of tumor invasion of neurovascular bundles according to claim 2, characterized in that: Also includes: The sample data in the data set are preprocessed, and the preprocessing at least includes clipping the sample data to obtain data in a non-zero area, resampling the sample data, and normalizing the sample data.

5. The method for evaluating the degree of tumor invasion of neurovascular bundles according to claim 2, characterized in that: The segmentation model is a three-dimensional full-resolution image model, i.e., a 3D U-Net network; The 3D U-Net network includes multiple different stages to gradually extract features of sample data.

6. The method for evaluating the degree of tumor invasion of neurovascular bundles according to claim 5, characterized in that: The 3DU-Net network consists of 7 different stages, each of which is responsible for extracting features at different levels, gradually from low-level features to high-level features; The number of feature channels in each stage is 32, 64, 128, 256, 320, 320, 320 respectively; The convolution kernel sizes of each stage are: the convolution kernel size of the first stage is 1, 3, 3; the convolution kernel size of the second stage is 1, 3, 3; the convolution kernel size of the third to seventh stages is 3, 3, 3; the stride of each stage is set as follows: the stride of the first stage is 1, 1, 1; the stride of the second stage is 1, 2, 2; the stride of the third stage is 1, 2, 2; the stride of the fourth stage is 2, 2, 2; the stride of the fifth stage is 2, 2, 2; the stride of the sixth stage is 1, 2, 2; the stride of the seventh stage is 1, 2, 2; The encoder and decoder stages have 2 convolutional layers each, no Dropout layers are used, and the non-linear activation function used is specified.

7. The method for evaluating the degree of tumor invasion of neurovascular bundles according to claim 2, characterized in that: Also includes: The probability map generated by the segmentation model is converted into a binary mask through a threshold, and is used to remove artifacts and noise in the binary mask based on dilation and erosion operations, thereby obtaining optimized mask data.

8. The method for evaluating the degree of tumor invasion of neurovascular bundles according to claim 1, characterized in that: The step of converting the mask data into point cloud data comprises: Extract the 3D surface mesh of the non-zero area from the mask data, including vertices, faces, and normal vectors, specifically: Based on the three-dimensional mask data V(x, y, z) obtained by segmentation, the contour threshold T is defined, the contour surface is constructed, and the three-dimensional mask is segmented into n cubes Ci, each cube consists of 8 fixed points; C i ={V(x i ,y i ,z i ),V(x i+1 ,y i ,z i ),V(x i ,y i+1 ,z i ),V(x i+1 ,y i ,z i+1 ),V(x i+1 ,y i+1 ,z i ),V(x i ,y i+1 ,z i+1 )V(x i+1 ,y i+1 ,z i+1 )} Among them, x i ,y i , z i Represents the x-axis, y-axis, and z-axis coordinates of the midpoint of the cube; for each vertex V of the cube i ∈C i , determine whether it is greater than the contour threshold T: Generate binary state index: S = S0S1S2S3S4S5S6S7 Using a lookup table, determine the boundary connectivity scheme in the cube based on the state index S: B(S)=LookupTable[S] For each edge e connecting two vertices k , perform linear interpolation to obtain points on the isosurface: Where V a , V b is with edge e k The values ​​of the two related vertices will be interpolated to get the point P k Connect to form a triangular surface mesh model; The number of triangles in all triangular surface mesh models is obtained, and the corresponding number of point clouds is calculated based on the number of triangles. The point cloud is extracted from the triangular mesh using the Poisson sampling method to obtain point cloud data.

9. The method for evaluating the degree of tumor invasion of neurovascular bundles according to claim 1, characterized in that: The step of calculating the geometric relationship between the tumor and the vascular nerve bundle region in the point cloud data, and judging the degree of invasion between the tumor and the vascular nerve bundle region based on the geometric relationship between the tumor and the vascular nerve bundle region and a preset invasion determination rule includes: Calculating, based at least on the point cloud data, the geometric centers of the tumor and the vascular nerve bundle region, the closest distance between the surface of the tumor and the surface of the vascular nerve bundle region, and the closest geometric center distance between the tumor and the vascular nerve bundle region; The degree of invasion between the tumor and the vascular nerve bundle area is judged at least based on the comparison result of the closest distance from the surface of the tumor to the surface of the vascular nerve bundle area and a preset closest distance threshold, and the comparison result between the closest geometric center distance between the tumor and the vascular nerve bundle area and a preset distance threshold.

10. A device for evaluating the extent of tumor invasion of neurovascular bundles, characterized in that: include: A data extraction unit, used to obtain dual-parameter resonance imaging of the patient and extract at least two modality data in the dual-parameter resonance imaging; A data alignment unit, used for aligning at least two types of modal data; A segmentation unit, used for inputting the aligned at least two modality data into a trained segmentation model, and outputting mask data including the tumor and the vascular nerve bundle area; A point cloud conversion unit, used for converting the mask data into point cloud data; A judgment unit is used to calculate the geometric relationship between the tumor and the vascular nerve bundle area in the point cloud data, and judge the degree of invasion between the tumor and the vascular nerve bundle area based on the geometric relationship between the tumor and the vascular nerve bundle area and a preset invasion judgment rule.

Images