A vascular reconstruction method based on small sample recognition

Abstract

The present invention discloses a vascular reconstruction method based on small-sample recognition, comprising: S1 obtaining CT data of a training sample vascular tree, the CT data including CT slices with vascular cross-sectional images and corresponding CT values; S2 an operator annotating blood vessels in some of the CT slices on a device to obtain a set of accurately annotated data; S3 based on the accurately annotated data obtained in S2, through neural network training, obtaining a primary recognition model; S4 performing iterative data augmentation training on the primary recognition model to finally obtain a recognition model capable of recognizing blood vessels in CT slices and outputting results; S5 based on the recognition model obtained in S4, recognizing the CT data of the vascular tree to be reconstructed to obtain the spatial information of the blood vessels; S6 connecting the blood vessels according to the spatial information obtained in S5 and through post-processing, obtaining a final three-dimensional model of the vascular tree. This method reduces the time-consuming of manual annotation while ensuring the recognition accuracy, and improves the recognition efficiency.

Description

Technical Field

[0001] The present invention relates to the technical field of medical image processing, and in particular to a blood vessel reconstruction method based on small sample recognition. Background Art

[0002] The recognition, segmentation, and reconstruction of pulmonary arteriovenous blood vessels play a crucial role in minimally invasive puncture surgery for the treatment of lung tumors. When the rigid needle body enters the patient's body during the puncture surgery, due to the body position changes caused by the patient's intraoperative breathing, the puncture needle may accidentally touch the vein, resulting in massive intraoperative bleeding. Therefore, accurate and clear intraoperative blood vessel imaging guidance is necessary.

[0003] Currently, tasks such as blood vessel segmentation mostly focus on parts such as the eyeball, heart, and liver. The existing technology uses the blood vessel tree as a training sample for end-to-end network learning. This method has problems such as high requirements for the amount of training data, long training time, and high dependence on samples and networks.

[0004] Patent document CN113674279A discloses a method and device for processing coronary CTA images based on deep learning. The method includes converting a CTA image sequence into a target NIFTI file for recognition, and converting the obtained NIFTI file with mask information into a target mask image sequence; removing the sternum region in the CTA image sequence according to the target mask image sequence to obtain a target image sequence; performing three-dimensional reconstruction of the image based on volume rendering on the target image sequence; extracting the blood vessel region from the three-dimensional model, and projecting each point in the blood vessel region onto a two-dimensional plane to obtain a reconstructed image after straightening the blood vessels. This method needs to first extract the spatial three-dimensional coordinates of the blood vessels from the entire three-dimensional modeling, and then perform modeling based on these three-dimensional coordinates. The preliminary preparation work takes too long and has certain requirements for the computing power of the device.

[0005] Patent document CN109063557B discloses a method for quickly constructing cardiac coronary blood vessel recognition data, including obtaining original pictures; performing pixel-level annotation on a very small number of original pictures according to a rough annotation map to form a refined annotation picture; converting the refined annotation picture from a three-channel image into a single-channel image; performing binary processing on the single-channel image and storing it as a binary picture; using the binary picture and its corresponding original picture as training data to train an initial network to obtain a first network; inputting all the original pictures into the above first network to obtain a binary result map, and generating a pseudo-refined annotation picture based on the binary result map and its corresponding rough annotation picture; establishing a database based on the original pictures, refined annotation pictures, and pseudo-annotation pictures to train the network. This method replaces manual annotation by building a pseudo-annotation map generation network, but this network can only recognize blood vessels in 2D images and cannot construct a 3D blood vessel tree, and cannot provide accurate and reliable spatial information for puncture. Summary of the Invention

[0006] To solve the above problems, the present invention provides a vascular reconstruction method based on small-sample recognition. This method only requires a small amount of accurately labeled data to be manually marked as the initial training material. Subsequently, the results output by the recognition model are used as alternative data, which are used as the secondary training material after manual correction. While ensuring the recognition accuracy, it reduces the time-consuming of manual annotation and improves the recognition efficiency.

[0007] A vascular reconstruction method based on small-sample recognition, comprising:

[0008] S1 Obtain the CT data of the training sample vascular tree, where the CT data includes CT slices with vascular cross-sectional images and corresponding CT values;

[0009] S2 The operator annotates the blood vessels in some CT slices on the device to obtain a set of accurately labeled data;

[0010] S3 Based on the accurately labeled data obtained in S2, through neural network training, obtain the initial recognition model;

[0011] S4 Perform iterative data augmentation training on the initial recognition model to finally obtain a recognition model that can recognize the blood vessels in the CT slices and output results;

[0012] S5 Based on the recognition model obtained in S4, recognize the CT data of the vascular tree to be reconstructed to obtain the spatial information of the blood vessels;

[0013] S6 Connect the blood vessels according to the spatial information obtained in S5, and through post-processing, obtain the final three-dimensional model of the vascular tree.

[0014] Preferably, the iterative data augmentation training in S4 is specifically:

[0015] S4.1 Select some unlabeled CT slices, put them into the initial recognition model for recognition, and obtain alternative data composed of the coordinate points recognized as blood vessels;

[0016] S4.2 The operator manually reviews the alternative data on the device to obtain a set of pseudo-accurately labeled data;

[0017] S4.3 Put the pseudo-accurately labeled data obtained in S4.2 into the initial recognition model for training;

[0018] S4.4 Repeat the above steps until the preset recognition accuracy is reached to obtain the final recognition model, where the preset recognition accuracy is equal to or greater than 80%.

[0019] Preferably, the annotation is to add a recognition identifier to the center point of the blood vessel cross-section in the CT slice and record its coordinates.

[0020] Preferably, in the manual review in S4.2, specifically, the operator determines the coordinate points in the alternative data, where the correctly recognized ones are used as positive samples, and the incorrectly recognized ones are used as negative samples. The positive samples and negative samples are integrated into pseudo-precise annotation data. Since the initial accuracy of the recognition model trained by small-sample recognition is not high, the proportion of positive samples in the output results of the initial recognition model is not high. Therefore, when the operator conducts a review in the alternative data, only new recognition identifiers need to be added to the coordinate points of the positive samples, so as to obtain pseudo-precise annotation data with higher accuracy. Further, using the pseudo-precise annotation data with high accuracy as the secondary training samples can make the training effect of the recognition model better.

[0021] Preferably, in S1, the gray value of the CT slice is normalized; by adjusting the pixels, the blood vessel image becomes more obvious and is convenient for recognition.

[0022] Preferably, the output result in S4 includes the CT slice with an identifier, and the identifier includes the two-dimensional coordinates of the CT slice and the CT value corresponding to the coordinate point.

[0023] Preferably, the spatial information in S5 includes the two-dimensional coordinates of the blood vessel center point and the corresponding CT slice serial number, where the serial number is used as the vertical coordinate.

[0024] Preferably, the specific process of S6 is as follows:

[0025] S6.1 By setting the CT value of each blood vessel center point as the standard for adaptive threshold setting, search for the differential coordinate points with obvious differences in CT value from the blood vessel center point to the surrounding;

[0026] S6.2 Based on the differential coordinate points, form a closed rectangle, and continue to identify all blood vessel points within the closed rectangle to finally obtain the blood vessel contour;

[0027] S6.3 According to the principle of proximity and the principle of consistent gradient, connect the blood vessel contours with adjacent CT slice serial numbers to obtain the blood vessel model;

[0028] S6.4 Integrate all blood vessel models and output to obtain the three-dimensional model of the blood vessel tree.

[0029] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0030] (1) Based on the two-dimensional images of small-sample blood vessels, the recognition model is trained in a closed loop, which can continuously improve the accuracy of the recognition model.

[0031] (2) During the sample annotation process, this technical solution only requires manual precise annotation of a small amount of data as training samples. Subsequently, only the correct coordinate points in the recognition results need to be judged and re-annotated as secondary training samples, thereby reducing the time-consuming of manual annotation and ensuring the accuracy of the training samples at the same time.

[0032] (3) The recognition model is based on two-dimensional images for recognition to generate three-dimensional data, which is different from the existing technology that directly recognizes three-dimensional images and has less computing pressure on the device. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 is a schematic flow chart of the vascular reconstruction method provided by the present invention;

[0034] Figure 2 is an effect diagram of the three-dimensional reconstruction model of the pulmonary vascular tree generated by the technical solution of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0035] As Figure 1 shown, a vascular reconstruction method based on small sample recognition has the following specific steps:

[0036] S1 Obtain the CT data of the training sample vascular tree, where the CT data includes CT slices with vascular cross-sectional images and corresponding CT values. Before training, window width and window level adjustments are made to the CT slices to make the blood vessels more obvious in the image, and the gray values are normalized to unify the image size to an appropriate resolution.

[0037] Among them, the window width and window level refer to, for example, the density difference of 2000 different gray levels that CT can recognize in the human body. However, the human eye can only distinguish 16 gray levels, that is, the CT value that the human eye can distinguish in the CT image should be 125 Hu (2000 / 16). That is, only when the CT values of different tissues in the human body differ by more than 125 Hu can they be recognized by the human eye. The CT values of human soft tissues mostly vary between 20 - 50 Hu, and the human eye cannot recognize them. Therefore, it is necessary to observe in segments to reflect the advantages of CT.

[0038] Among them, the observed CT value range is called the window width, and the central CT value of the observation is the window level or window center.

[0039] Therefore, the window width and window level adjustment can be operationally equivalent to setting upper and lower thresholds. Set the lower threshold to -150, and set the gray values less than -150 to -150. Set the upper threshold to 200, and set the gray values greater than 200 to 200.

[0040] The normalization process statistically obtains the maximum and minimum values of all pixels after adjusting the window width and window level, and then performs min-max normalization on each pixel to map the result values to the range of [0 - 1]. The min-max normalization conversion function is as follows:

[0041]

[0042] Where max is the maximum value of the sample data, min is the minimum value of the sample data, x is the pixel value before normalization processing, and y is the pixel value after normalization processing.

[0043] S2 The operator annotates the blood vessels in a certain number of randomly selected CT slices on the device to obtain a set of accurately annotated data. The operator only needs to add an identification mark to the center point of the cross-section of the blood vessels in the CT slices.

[0044] S3 Based on the accurately annotated data obtained in S2, through neural network training, an initial recognition model is obtained. This initial model can also recognize the blood vessels in any slice of CT data, but the recognition accuracy of this initial model is not high.

[0045] S4 Perform iterative data augmentation training on the initial blood vessel recognition model to finally obtain a recognition model that can recognize the blood vessels in CT slices and output results. The iterative data augmentation training is specifically as follows:

[0046] S4.1 Select some unannotated CT slices and put them into the initial recognition model for recognition to obtain alternative data composed of the coordinate points recognized as blood vessels.

[0047] S4.2 The operator manually reviews the alternative data on the device:

[0048] The operator determines the coordinate points in the alternative data. Among them, the correctly recognized ones are used as positive samples, and the wrongly recognized ones are used as negative samples. New identification marks are added to the coordinate points corresponding to the positive samples to obtain a set of pseudo-accurately annotated data.

[0049] S4.3 Put the pseudo-accurately annotated data obtained in S4.2 into the initial recognition model for training.

[0050] S4.4 Repeat the above steps until the preset recognition accuracy is reached to obtain the final recognition model. The iteration stops when the preset recognition accuracy is greater than 80%.

[0051] The output result of the recognition model includes the CT slice with marks, and the marks correspond to the two-dimensional coordinates of the blood vessel center point in the CT slice and the CT value corresponding to the blood vessel center point.

[0052] S5 Based on the recognition model obtained in S4, recognize the CT data of the blood vessel tree to be reconstructed to obtain the two-dimensional coordinates of the blood vessel center points and the corresponding CT slice serial numbers, where the serial numbers are used as the vertical coordinates.

[0053] S6 performs three-dimensional reconstruction of the vascular tree based on the two-dimensional coordinates of the vascular center points obtained in S5 and the corresponding CT slice serial numbers:

[0054] S6.1 By setting the CT value of each vascular center point as the standard for adaptive threshold setting, search for differential coordinate points with obvious differences in CT value from the vascular center point to the surrounding;

[0055] S6.2 Based on the differential coordinate points, form a closed rectangle, and continue to identify all vascular points within the closed rectangle to finally obtain the vascular contour;

[0056] S6.3 Connect the vascular contours with adjacent CT slice serial numbers according to the principle of proximity and the principle of consistent gradient to obtain the vascular model;

[0057] S6.4 Integrate and output all vascular models to obtain the three-dimensional model of the vascular tree, and its actual modeling effect is as shown in Figure 2 shown.

Claims

1. A vascular reconstruction method based on small-sample recognition, characterized in that, Including: S1 Obtain the CT data of the training sample vascular tree, where the CT data includes CT slices with vascular cross-sectional images and corresponding CT values; S2 The operator annotates the blood vessels in some of the CT slices on the device to obtain a set of accurately annotated data. The annotation is to add an identification mark to the center point of the blood vessel cross-section in the CT slice and record its coordinates; S3 Based on the accurately annotated data obtained in S2, through neural network training, obtain the initial recognition model; S4 Perform iterative data augmentation training on the initial recognition model to finally obtain a recognition model that can recognize the blood vessels in the CT slice and output the results. The iterative data augmentation training is specifically as follows: S4.1 Select some unannotated CT slices and put them into the initial recognition model for recognition to obtain alternative data composed of coordinate points recognized as blood vessels; S4.2 The operator performs manual review on the alternative data on the device to obtain a set of pseudo-accurately annotated data; S4.3 Put the pseudo-accurately annotated data obtained in S4.2 into the initial recognition model for training; S4.4 Repeat the above steps until the preset recognition accuracy is reached to obtain the final recognition model; S5 Based on the recognition model obtained in S4, recognize the CT data of the vascular tree to be reconstructed to obtain the spatial information of the blood vessels; S6 Connect the blood vessels according to the spatial information obtained in S5, and through post-processing, obtain the final three-dimensional model of the vascular tree. The specific process is as follows: S6.1 By setting the CT value of each blood vessel center point as the standard for adaptive threshold setting, extend and search around the blood vessel center point for different coordinate points with obvious differences in their CT values; S6.2 Based on the different coordinate points, form a closed rectangle, and continue to recognize all blood vessel points within the closed rectangle to finally obtain the blood vessel contour; S6.3 Connect the blood vessel contours with adjacent CT slice serial numbers according to the principle of proximity and the principle of consistent gradient to obtain the blood vessel model; S6.4 Integrate and output all blood vessel models to obtain the three-dimensional model of the vascular tree.

2. The vascular reconstruction method according to claim 1, characterized in that, The manual review in S4.2 is specifically that the operator determines the coordinate points in the alternative data and adds a new identification mark to the coordinate points of the correctly recognized blood vessels.

3. The vascular reconstruction method according to claim 1, characterized in that S1 performs normalization processing on the gray value of the CT slice.

4. The vascular reconstruction method according to claim 1, wherein The output result in S4 includes the CT slice with marks, and the marks are the two-dimensional coordinates of the CT slice and the CT value corresponding to the coordinate point.

5. The vascular reconstruction method according to claim 1, characterized in that, The spatial information in S5 includes the two-dimensional coordinates of the blood vessel center point and the serial number of the corresponding CT slice, where the serial number is used as the vertical coordinate.

Images