Blood vessel lumen extraction method, device, electronic device and storage medium
Through the deep learning-based vascular lumen extraction model, using multiple sets of historical vascular images and labeled images training, the problem of the automatic segmentation method of intravascular ultrasound images in the existing technology requires advanced knowledge and manual parameters adjustment, achieving rapid and accurate vascular lumen extraction effect.
Patent Information
- Application Number
- CN202011205786.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2020-11-02
- Publication Date
- 2024-11-29
- Estimated Expiration
- 2040-11-02
AI Technical Summary
The existing automatic segmentation method of intravascular ultrasound images requires advanced image algorithm knowledge, manual parameters are time-consuming and labor-intensive, and the result accuracy and robustness are insufficient.
A vascular lumen extraction model trained based on multiple sets of historical vascular images and historical labeled images is used to input the image to be extracted into the model, and a deep learning neural network is used to quickly and accurately extract the vascular lumen, reducing the burden of algorithm development and parameter adjustment.
Fast and accurate vascular lumen extraction is achieved, the extraction efficiency and accuracy are improved, the difficulty of algorithm development is reduced, and the inefficiency and accuracy problems caused by manual parameter adjustment are avoided.
Smart Images

Figure CN112308844B_ABST
Abstract
Description
Technical Field
[0001] The embodiments of the present invention relate to medical imaging technology, and in particular to a method, device, electronic device and storage medium for extracting a blood vessel lumen. Background Art
[0002] Cardiovascular diseases have a high mortality rate worldwide, and intravascular ultrasound technology is the mainstream diagnostic tool for cardiovascular diseases. It is an intravascular imaging technology that can obtain high-definition vascular tomographic images in real time by inserting a sensor probe into an artery. By outlining the inner and outer membranes in the image, the morphological characteristics and plaque conditions in the blood vessels can be specifically evaluated, such as lumen diameter, minimum lumen cross-sectional area, atherosclerotic plaque volume, etc. This information provides a quantitative basis for doctors to diagnose whether a stent needs to be placed or what stent to place.
[0003] At present, the automatic segmentation method of intravascular ultrasound images generally needs to consider three types of information: grayscale features, sequence time characteristics and prior knowledge of intravascular ultrasound images. First, the noise and artifact interference are reduced by the sequence time characteristics and prior knowledge to extract the initial vascular structure of the first frame image; then the initial structure is deformed by using the B-spline GVF snake that combines the gradient, grayscale variance and grayscale mean information to obtain the final structure of the first frame; finally, the structure of subsequent frames is extracted using the time characteristics of the sequence image.
[0004] The above method requires engineers to have a high level of knowledge of image algorithms, and there are many variable parameters that need to be manually adjusted based on experience, which is time-consuming and labor-intensive, and has poor results in terms of result accuracy and robustness. Summary of the invention
[0005] The embodiments of the present invention provide a method, device, electronic device and storage medium for extracting a blood vessel lumen, so as to achieve the effect of extracting the blood vessel lumen quickly and accurately.
[0006] In a first aspect, an embodiment of the present invention provides a method for extracting a blood vessel lumen, the method comprising:
[0007] Acquire an image with blood vessels to be extracted;
[0008] The image to be extracted is input into a trained vascular lumen extraction model to obtain the vascular lumen of the image to be extracted; wherein the trained vascular lumen extraction model is trained based on multiple groups of historical vascular images and historical annotated images corresponding to the historical vascular images.
[0009] In a second aspect, an embodiment of the present invention further provides a blood vessel lumen extraction device, the device comprising:
[0010] An image acquisition module to be extracted, used to acquire an image to be extracted having blood vessels;
[0011] The vascular lumen extraction module is used to input the image to be extracted into a trained vascular lumen extraction model to obtain the vascular lumen of the image to be extracted; wherein the trained vascular lumen extraction model is trained based on multiple groups of historical vascular images and historical annotated images corresponding to the historical vascular images.
[0012] In a third aspect, an embodiment of the present invention further provides an electronic device, the electronic device comprising:
[0013] one or more processors;
[0014] A storage device for storing one or more programs;
[0015] When the one or more programs are executed by the one or more processors, the one or more processors implement any of the blood vessel lumen extraction methods described in the embodiments of the present invention.
[0016] In a fourth aspect, an embodiment of the present invention further provides a storage medium comprising computer executable instructions, wherein the computer executable instructions, when executed by a computer processor, are used to execute any of the blood vessel lumen extraction methods described in the embodiments of the present invention.
[0017] The technical solution of the embodiment of the present invention directly obtains the vascular lumen of the image to be extracted by inputting the acquired image to be extracted into a trained vascular lumen extraction model. In this way, the vascular lumen can be extracted quickly and accurately for the acquired image to be extracted, which reduces the burden of algorithm development, reduces the difficulty of algorithm development, and avoids the problem of low efficiency and accuracy of vascular lumen extraction caused by manual adjustment of algorithm parameters in the prior art. BRIEF DESCRIPTION OF THE DRAWINGS
[0018] Figure 1 is a flow chart of the blood vessel lumen extraction method in Embodiment 1 of the present invention;
[0019] Figure 2 is a flow chart of a blood vessel lumen extraction method in Embodiment 2 of the present invention;
[0020] Figure 3 is a flow chart of a blood vessel lumen extraction method in Embodiment 3 of the present invention;
[0021] Figure 4 is a schematic diagram of the structure of a blood vessel lumen extraction device in a fourth embodiment of the present invention;
[0022] Figure 5 It is a structural schematic diagram of an electronic device in Embodiment 5 of the present invention. DETAILED DESCRIPTION
[0023] The present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the present invention, rather than to limit the present invention. It should also be noted that, for ease of description, only parts related to the present invention, rather than all structures, are shown in the accompanying drawings.
[0024] Embodiment 1
[0025] Figure 1 This is a flow chart of a blood vessel lumen extraction method provided in Embodiment 1 of the present invention. This embodiment is applicable to the case of quickly and accurately extracting blood vessel lumens. The method can be performed by a blood vessel lumen extraction device. The blood vessel lumen extraction device can be implemented by software and / or hardware. The blood vessel lumen extraction device can be configured on an electronic computing device. Specifically, the method includes the following steps:
[0026] S110 , obtaining an image with blood vessels to be extracted.
[0027] Exemplarily, the image to be extracted may be an image containing blood vessels, for example, an intravascular ultrasound (IVUS) image.
[0028] S120, inputting the image to be extracted into a trained vascular lumen extraction model to obtain the vascular lumen of the image to be extracted; wherein the trained vascular lumen extraction model is trained based on multiple groups of historical vascular images and historical annotated images corresponding to the historical vascular images.
[0029] Exemplarily, the vascular lumen extraction model may be a neural network model based on deep learning, such as a bas-net neural network model.
[0030] The historical blood vessel image may be an image containing blood vessels that has been scanned and acquired, for example, may be a historical intravascular ultrasound image.
[0031] The historical annotated image corresponding to the historical vascular image may be an image in which the blood vessels in the historical vascular image are annotated. For example, a physician may use annotation software such as 3D-Slice to outline the inner and outer walls of the blood vessel lumen on each historical vascular image as annotation information and save it as a picture in PNG or JPG format. This image is the historical annotated image corresponding to the historical vascular image.
[0032] The vascular lumen extraction model is trained using multiple groups of historical vascular images and historical annotated images corresponding to the historical vascular images to obtain a trained vascular lumen extraction model. The image to be extracted is input into the trained vascular lumen extraction model to directly obtain the vascular lumen of the image to be extracted. In this way, the vascular lumen can be quickly and accurately extracted for the acquired image to be extracted, which reduces the burden of algorithm development, reduces the difficulty of algorithm development, and avoids the problem of low efficiency and accuracy of vascular lumen extraction caused by manual adjustment of various algorithm parameters in the prior art.
[0033] The technical solution of the embodiment of the present invention directly obtains the vascular lumen of the image to be extracted by inputting the acquired image to be extracted into a trained vascular lumen extraction model. In this way, the vascular lumen can be extracted quickly and accurately for the acquired image to be extracted, which reduces the burden of algorithm development, reduces the difficulty of algorithm development, and avoids the problem of low efficiency and accuracy of vascular lumen extraction caused by manual adjustment of algorithm parameters in the prior art.
[0034] Embodiment 2
[0035] Figure 2 The flowchart of the vascular lumen extraction method provided in the second embodiment of the present invention can be combined with each optional scheme in the above embodiments. In the embodiment of the present invention, optionally, the training method of the vascular lumen extraction model includes: obtaining multiple groups of historical vascular images and historical annotated images corresponding to the historical vascular images, wherein the historical annotated images are images after outlining the inner and outer walls of the blood vessels in the historical vascular images; inputting the multiple groups of historical vascular images and the historical annotated images into the vascular lumen extraction model, iteratively training the vascular lumen extraction model, and obtaining a trained vascular lumen extraction model.
[0036] like Figure 2 As shown, the method of the embodiment of the present invention specifically includes the following steps:
[0037] S210 , acquiring multiple groups of historical vascular images and historical annotated images corresponding to the historical vascular images.
[0038] Exemplarily, when training the vascular lumen extraction model, multiple groups of historical vascular images and historical annotated images corresponding to the historical vascular images are first extracted.
[0039] It should be noted that the historical vascular images here may be historical intravascular ultrasound images of patients obtained from a hospital image database or other means. The corresponding historical annotated images may be images obtained by outlining and annotating the inner and outer walls of the blood vessels in each historical vascular image.
[0040] It should be noted that the specific method of acquiring the historical vascular images is not limited here, and any method that can acquire the historical vascular images belongs to the protection scope of the present invention. Similarly, the outlining and marking of the inner and outer walls of the blood vessels in the historical vascular images here is also limited to the marking software such as 3d-slice in the above embodiment, and any method that can outline and mark the inner and outer walls of the blood vessels in the historical vascular images belongs to the protection scope of the present invention.
[0041] S220 , inputting multiple groups of historical vascular images and historical annotated images into a vascular lumen extraction model, iteratively training the vascular lumen extraction model, and obtaining a trained vascular lumen extraction model.
[0042] Exemplarily, the acquired multiple groups of historical vascular images and corresponding historical annotated images are input into a vascular lumen extraction model to be trained, and the vascular lumen extraction model to be trained is iteratively trained to obtain a trained vascular lumen extraction model.
[0043] In this way, the vascular lumen extraction model to be trained is trained using multiple groups of historical vascular images and historical annotated images to obtain a trained vascular lumen extraction model, so that the vascular lumen can be quickly and accurately extracted from the image to be extracted based on the trained vascular lumen extraction model.
[0044] Optionally, the iterative training of the vascular lumen extraction model may specifically be: classifying historical vascular images based on the vascular lumen extraction model to obtain classified images of the inner and outer walls of the historical vascular images; comparing the classified images of the inner and outer walls of the historical vascular images with corresponding historical annotated images to determine the loss function of the vascular lumen extraction model; and when the loss function is less than a preset threshold, it is determined that the training of the vascular lumen extraction model is completed.
[0045] Exemplarily, the classified image of the inner and outer walls of the historical blood vessel image may be an image outlining the inner and outer walls of the blood vessel obtained after the blood vessel lumen extraction model to be trained classifies the historical blood vessel image.
[0046] The preset threshold may be a threshold of a pre-set loss function. When the loss function of the vascular lumen extraction model to be trained is less than the value, it is proved that the vascular lumen extraction model to be trained has been trained.
[0047] After acquiring the historical vascular images, each historical vascular image is input into the vascular lumen extraction model to be trained, and the parameters of each network structure in the vascular lumen extraction model to be trained are adjusted. The blood vessels and other contents contained in each historical vascular image can be classified to obtain each classified image.
[0048] Specifically, taking any historical vascular image as an example, the historical vascular image contains not only blood vessels but also other soft tissue components. The historical vascular image is input into the vascular lumen extraction model to be trained, and the vascular lumen extraction model to be trained will classify the contents contained in the historical vascular image, for example, outlining the inner and outer walls of the blood vessels in the historical vascular image to obtain a classified image of the blood vessels in the historical vascular image, and outlining the soft tissue part in the historical vascular image to obtain a classified image of the soft tissue part in the historical vascular image. In this way, the classification of the historical vascular image can be realized, and then a classified image of the inner and outer walls of the blood vessels in the historical vascular image is obtained by outlining the inner and outer walls of all blood vessels in the historical vascular image.
[0049] The obtained vascular inner and outer wall classified images of each historical vascular image are respectively compared with the historical annotated images of each historical vascular image to determine the loss function of the vascular lumen extraction model to be trained. When the loss function of the vascular lumen extraction model to be trained is less than a preset threshold, it is determined that the training of the vascular lumen extraction model to be trained is completed.
[0050] Optionally, the loss function of the vascular lumen extraction model may be determined based on the following formula (1):
[0051]
[0052] For any sample (x, y), y is the true classification result corresponding to the sample, and the predicted classification result is the set of all classification results. Assuming there are k label values, the probability that the i-th sample is predicted to be the K-th label is p i,k , N is the total number of samples, L log It is the loss function of the preset classification results and the actual classification results of the historical vascular images.
[0053] Exemplarily, the loss function of the vascular lumen extraction model to be trained is determined according to the above formula (1). Specifically, for any historical vascular image, after it is input into the vascular lumen extraction model to be trained, the probability of whether the classified image belongs to a vascular classification image or a soft tissue classification image (i.e., the predicted classification result) is determined according to the input classification image, and the predicted classification result is calculated with the historical annotated image (true classification result) corresponding to the historical vascular image using the above formula to obtain the loss function of the vascular lumen extraction model to be trained.
[0054] It should be noted that the above formula (1) is a multi-classification cross entropy loss function that is an improvement on the binary cross entropy loss function in the prior art. In the prior art, the loss function of the neural network model for classification problems all adopts the binary cross entropy loss function (such as formula (2)). For a sample (x, y), x is the sample and y is the true label corresponding to the sample. In a binary classification problem, the set of its values may be {0, 1}. Assume that the true label of a sample is y. t , the y of the sample t =1 with a probability of y p , then the loss function of this sample is:
[0055] log(y t |y p )=-(y t *log(y p )+(1-y t )log(1-y p )) (2)
[0056] It should be noted that if the model is used on the entire data set: its loss function is the average of the loss functions of all sample points.
[0057] However, in the vascular lumen extraction model of the embodiment of the present invention, it is not a binary classification task, but a ternary classification task. Usually, the classification number of categories greater than or equal to 3 is called multi-classification, so it is not suitable to use the cross entropy of binary classification. Therefore, in the embodiment of the present invention, it is replaced by the multi-classification cross entropy function of the above formula (1).
[0058] In this way, the binary cross entropy loss function in the prior art is improved to obtain the multi-classification cross entropy loss function in the embodiment of the present invention. The improved multi-classification cross entropy loss function is used to calculate the loss function of the vascular lumen extraction model to be trained in the embodiment of the present invention, so that a more accurate vascular lumen extraction model can be obtained.
[0059] It should be noted that when the vascular lumen extraction model to be trained is iteratively trained, the corresponding historical vascular images with large loss function values are deleted according to the obtained multi-classification cross entropy loss function, and then the historical vascular images with small loss function values and the corresponding historical annotated images are used to continue to iterate the vascular lumen extraction model to be trained until the value of the loss function is less than the preset threshold.
[0060] S230: Acquire an image with blood vessels to be extracted.
[0061] S240, inputting the image to be extracted into a trained vascular lumen extraction model to obtain the vascular lumen of the image to be extracted; wherein the trained vascular lumen extraction model is trained based on multiple groups of historical vascular images and historical annotated images corresponding to the historical vascular images.
[0062] The technical solution of the embodiment of the present invention is to input multiple sets of historical vascular images and historical annotated images corresponding to the historical vascular images into the vascular lumen extraction model to be trained, and iteratively train the vascular lumen extraction model to be trained to obtain a trained vascular lumen extraction model. In this way, the vascular lumen of the image to be extracted can be quickly and accurately extracted based on the trained vascular lumen extraction model.
[0063] Embodiment 3
[0064] Figure 3 The flowchart of the vascular lumen extraction method provided in the third embodiment of the present invention can be combined with each optional scheme in the above embodiment. In the embodiment of the present invention, optionally, before inputting the multiple groups of historical vascular images and historical annotated images into the vascular lumen extraction model, the method further includes: based on a preset screening rule, screening the historical annotated images, and eliminating unqualified historical annotated images; wherein the unqualified historical annotated images include: historical annotated images with incorrect delineation of the inner and outer walls of the blood vessels and / or historical annotated images with missed delineation of the inner and outer walls of the blood vessels; correspondingly, after screening the historical annotated images, the historical vascular images corresponding to the eliminated historical annotated images are correspondingly eliminated. Before inputting the multiple groups of historical vascular images and historical annotated images into the vascular lumen extraction model, the method further includes: performing image enhancement according to preset enhancement rules on the screened historical vascular images and the screened historical annotated images, respectively, to obtain a target historical vascular image and a target historical annotated image corresponding to the target historical vascular image; correspondingly, inputting the multiple groups of historical vascular images and historical annotated images into the vascular lumen extraction model includes: inputting the multiple groups of target historical vascular images and the target historical annotated images into the vascular lumen extraction model.
[0065] like Figure 3 As shown, the method of the embodiment of the present invention specifically includes the following steps:
[0066] S310 , acquiring multiple groups of historical vascular images and historical annotated images corresponding to the historical vascular images.
[0067] S320 , screening the historically annotated images based on preset screening rules, removing unqualified historically annotated images, and removing historical vascular images corresponding to the removed historically annotated images.
[0068] Exemplarily, the preset screening rule may be a pre-set rule for screening historically annotated images, for example, it may be a rule for deleting historically annotated images in which the inner and outer walls of blood vessels are incorrectly outlined and / or historically annotated images in which the inner and outer walls of blood vessels are not outlined.
[0069] The unqualified historically annotated images may be unqualified images such as historically annotated images in which the inner and outer walls of the blood vessels are incorrectly delineated and / or historically annotated images in which the inner and outer walls of the blood vessels are not delineated.
[0070] After deleting the historical annotated images with incorrect delineation of the inner and outer walls of the blood vessels and / or the historical annotated images with missed delineation of the inner and outer walls of the blood vessels through the above preset screening rules, the historical blood vessel images corresponding to the deleted historical annotated images are also deleted.
[0071] Specifically, the unqualified historical annotated images can be deleted by using the preset screening rules, which can be to train a model using all the historical annotated images and historical vascular images. Specifically, the historical vascular images can be input into the model to be trained to obtain the predicted annotated images corresponding to the historical vascular images, and the predicted annotated images can be compared with the historical annotated images corresponding to the historical vascular images. For example, the intersection-over-union ratio or cross-entropy loss function can be calculated to compare the predicted annotated images with the historical annotated images corresponding to the historical vascular images. When the value of the intersection-over-union ratio or the cross-entropy loss function is less than the preset value, it is proved that the training of the model to be trained is completed, that is, the preset annotated images can be used and do not need to be eliminated. The historical annotated images with large values of the intersection-over-union ratio or the cross-entropy loss function are eliminated. The historical annotated images with large values of the intersection-over-union ratio or the cross-entropy loss function are historical annotated images with incorrect delineation of the inner and outer walls of the blood vessels and / or historical annotated images with missed delineation of the inner and outer walls of the blood vessels. The remaining historical annotated images are subjected to subsequent calculations.
[0072] It should be noted that, in addition to the calculation of the intersection-over-union ratio or cross entropy loss function described above, the calculation of the signal-to-noise ratio can also be used to delete unqualified annotated image data. Specifically, the image noise of the preset annotated image and the image noise of the historical annotated image can be calculated, and the image noise of the two can be compared to calculate the signal-to-noise ratio of the two.
[0073] It should be noted that the above specific use of intersection-over-union ratio, signal-to-noise ratio or cross entropy loss function can be selected according to user needs, and at least two of the above three methods can be selected to increase accuracy. Of course, methods other than the above three methods can also be used to clean and delete historical annotated images, and any method that can clean and delete historical annotated images belongs to the protection scope of the embodiments of the present invention.
[0074] In this way, unqualified historical annotated images and corresponding historical vascular images are deleted to prevent these unqualified images from affecting the subsequent training of the vascular lumen extraction model to be trained, thereby affecting the training accuracy of the vascular lumen extraction model to be trained.
[0075] S330 , performing image enhancement according to preset enhancement rules on the filtered historical vascular image and the filtered historical annotated image to obtain a target historical vascular image and a target historical annotated image corresponding to the target historical vascular image.
[0076] Exemplarily, the preset enhancement rule may be a pre-set rule for image enhancement, and specifically may be a rule for image quantity enhancement, which can increase the training samples of the vascular lumen extraction model to be trained.
[0077] The target historical vascular image may be a historical vascular image obtained by performing a preset enhancement rule on the screened historical vascular image.
[0078] The target historical annotated image may be a historical annotated image obtained by performing the same enhancement method on the filtered historical annotated image as that on the corresponding filtered historical vascular image.
[0079] Optionally, the preset enhancement rule here may be to flip the filtered historical vascular images and the filtered historical annotated images up and / or down and / or left and / or right, respectively. This can further enrich the number of samples of the historical vascular images and the corresponding historical annotated images to be input into the vascular lumen extraction model to be trained, and reduce the degree of overfitting.
[0080] It should be noted that when the image enhancement is performed on the filtered historical vascular images and the filtered historical annotated images according to the preset enhancement rule, the image enhancement should be performed synchronously on the filtered historical vascular images and the filtered historical annotated images. For example, if a filtered historical vascular image is flipped up, the historical annotated image corresponding to the filtered historical vascular image is also flipped up.
[0081] S340 , inputting multiple groups of target historical blood vessel images and target historical annotated images into a blood vessel lumen extraction model, iteratively training the blood vessel lumen extraction model, and obtaining a trained blood vessel lumen extraction model.
[0082] Exemplarily, the obtained multiple groups of target historical vascular images and corresponding target historical annotated images are input into the vascular lumen extraction model to be trained, and the vascular lumen extraction model to be trained is iteratively trained to obtain a trained vascular lumen extraction model.
[0083] S350: Acquire an image with blood vessels to be extracted.
[0084] S360, inputting the image to be extracted into a trained vascular lumen extraction model to obtain the vascular lumen of the image to be extracted; wherein the trained vascular lumen extraction model is trained based on multiple groups of historical vascular images and historical annotated images corresponding to the historical vascular images.
[0085] For example, the accuracy of the extracted vascular lumen by the technical solution of the embodiment of the present invention is higher than that of the prior art. Specifically, as shown in the following table, the two test indicators of JACC (Jaccard Measure) and HD (Hausdorff Distance) are compared for the vascular lumen and the vascular adventitia to illustrate the accuracy of the technical solution of the embodiment of the present invention.
[0086]
[0087] It should be noted that, as known to those skilled in the art, as long as any one of the indicators JACC and HD is better than that in the prior art, it is confirmed that the accuracy of the model is higher than that of the prior art. And as known to those skilled in the art, although the value of the HD indicator in the present application in the above table is lower than that in the prior art, this does not mean that the HD indicator in the present application is not as effective as the prior art. As known to those skilled in the art, the HD indicator is determined according to the actual application scenario, that is, in some application scenarios, a high HD value has a good effect, but in some application scenarios, a low HD value has a good effect, and in the application scenario of the embodiment of the present invention, a low HD value has a good effect.
[0088] It should be noted that, for the technical solution of the embodiment of the present invention, the PAD (Percentage of Area Difference) index is also calculated. In the technical solution of the embodiment of the present invention, for the PAD index, the value of Lumen is 0.0670, and the value of Media is 0.0423.
[0089] The technical solution of the embodiment of the present invention is to screen the historical annotated images based on preset screening rules, remove unqualified historical annotated images, and remove the historical vascular images corresponding to the removed historical annotated images, so that the unqualified historical annotated images and the corresponding historical vascular images are deleted to avoid these unqualified images from affecting the subsequent training of the vascular lumen extraction model to be trained, and affecting the training accuracy of the vascular lumen extraction model to be trained. The screened historical vascular images and the screened historical annotated images are respectively enhanced according to preset enhancement rules to obtain the target historical vascular images and the target historical annotated images corresponding to the target historical vascular images, so that the number of samples of the historical vascular images and the corresponding historical annotated images to be input into the vascular lumen extraction model to be trained can be further enriched, and the degree of overfitting can be reduced.
[0090] Embodiment 4
[0091] Figure 4 This is a schematic diagram of the structure of a blood vessel lumen extraction device provided in Embodiment 4 of the present invention, as shown in FIG. Figure 4 As shown, the device includes: an image acquisition module 31 to be extracted and a blood vessel lumen extraction module 32.
[0092] The to-be-extracted image acquisition module 31 is used to acquire the to-be-extracted image with blood vessels;
[0093] The vascular lumen extraction module 32 is used to input the image to be extracted into a trained vascular lumen extraction model to obtain the vascular lumen of the image to be extracted; wherein the trained vascular lumen extraction model is trained based on multiple groups of historical vascular images and historical annotated images corresponding to the historical vascular images.
[0094] Based on the technical solution of the embodiment of the present invention, the device also includes:
[0095] A historical image acquisition module, used to acquire multiple groups of historical vascular images and historical annotated images corresponding to the historical vascular images, wherein the historical annotated images are images obtained by outlining the inner and outer walls of the blood vessels in the historical vascular images;
[0096] The model training module is used to input multiple groups of the historical vascular images and the historical annotated images into the vascular lumen extraction model, iteratively train the vascular lumen extraction model, and obtain the trained vascular lumen extraction model.
[0097] Based on the technical solution of the embodiment of the present invention, the model training module includes:
[0098] A classification unit, configured to classify the historical blood vessel image based on the blood vessel lumen extraction model to obtain a classified image of the inner and outer walls of the historical blood vessel image;
[0099] The model completion training unit is used to compare the vascular inner and outer wall classification images of the historical vascular image with the corresponding historical annotated image to determine the loss function of the vascular lumen extraction model. When the loss function is less than a preset threshold, it is determined that the vascular lumen extraction model training is completed.
[0100] Based on the technical solution of the embodiment of the present invention, the model completion training unit is specifically used for:
[0101] The classified images of the inner and outer walls of the historical vascular images are compared with the corresponding historical annotated images, and the loss function of the vascular lumen extraction model is determined based on the following formula. When the loss function is less than a preset threshold, it is determined that the training of the vascular lumen extraction model is completed.
[0102]
[0103] For any sample (x, y), y is the true classification result corresponding to the sample, and the predicted classification result is the set of all classification results. Assuming there are k label values, the probability that the i-th sample is predicted to be the K-th label is p i,k , N is the total number of samples, L log It is the loss function of the preset classification results and the actual classification results of the historical vascular images.
[0104] Based on the technical solution of the embodiment of the present invention, the device also includes:
[0105] A first screening module is used to screen the historically annotated images based on preset screening rules, and remove unqualified historically annotated images; wherein the unqualified historically annotated images include: historically annotated images with incorrect delineation of the inner and outer walls of blood vessels and / or historically annotated images with missed delineation of the inner and outer walls of blood vessels;
[0106] The second screening module is used to eliminate the historical blood vessel images corresponding to the eliminated historical annotated images.
[0107] Based on the technical solution of the embodiment of the present invention, the device also includes:
[0108] An image enhancement module, used to perform image enhancement according to preset enhancement rules on the filtered historical vascular image and the filtered historical annotated image, respectively, to obtain a target historical vascular image and a target historical annotated image corresponding to the target historical vascular image;
[0109] Correspondingly, the model training module is specifically used for:
[0110] A plurality of groups of the target historical blood vessel images and the target historical annotated images are input into the blood vessel lumen extraction model, and the blood vessel lumen extraction model is iteratively trained to obtain the trained blood vessel lumen extraction model.
[0111] The vascular lumen extraction device provided in the embodiment of the present invention can execute the vascular lumen extraction method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.
[0112] Embodiment 5
[0113] Figure 5 A schematic diagram of the structure of an electronic device provided in Embodiment 5 of the present invention is shown in FIG. Figure 5 As shown, the electronic device includes a processor 70, a memory 71, an input device 72 and an output device 73; the number of processors 70 in the electronic device can be one or more. Figure 5 A processor 70 is taken as an example; the processor 70, the memory 71, the input device 72 and the output device 73 in the electronic device can be connected by a bus or other means. Figure 5 The example of connecting through bus is taken in the following.
[0114] The memory 71 is a computer-readable storage medium that can be used to store software programs, computer executable programs and modules, such as program instructions / modules corresponding to the vascular lumen extraction method in the embodiment of the present invention (for example, the image acquisition module 31 to be extracted and the vascular lumen extraction module 32). The processor 70 executes various functional applications and data processing of the electronic device by running the software programs, instructions and modules stored in the memory 71, that is, realizing the above-mentioned vascular lumen extraction method.
[0115] The memory 71 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system and at least one application required for a function; the data storage area may store data created according to the use of the terminal, etc. In addition, the memory 71 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some instances, the memory 71 may further include a memory remotely arranged relative to the processor 70, and these remote memories may be connected to the electronic device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.
[0116] The input device 72 may be used to receive input digital or character information and generate key signal input related to user settings and function control of the electronic device. The output device 73 may include a display device such as a display screen.
[0117] Embodiment 6
[0118] Embodiment 6 of the present invention further provides a storage medium comprising computer executable instructions, wherein the computer executable instructions are used to execute a method for extracting a blood vessel lumen when executed by a computer processor.
[0119] Of course, the computer executable instructions of a storage medium containing computer executable instructions provided by an embodiment of the present invention are not limited to the operations of the method described above, and can also execute related operations in the vascular lumen extraction method provided by any embodiment of the present invention.
[0120] Through the above description of the implementation methods, the technicians in the relevant field can clearly understand that the present invention can be implemented by means of software and necessary general hardware, and of course it can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product, which can be stored in a computer-readable storage medium, such as a computer floppy disk, read-only memory (ROM), random access memory (RAM), flash memory (FLASH), hard disk or optical disk, etc., including a number of instructions for a computer electronic device (which can be a personal computer, server, or network device, etc.) to execute the methods described in each embodiment of the present invention.
[0121] It is worth noting that in the above-mentioned embodiment of the vascular lumen extraction device, the various units and modules included are only divided according to functional logic, but are not limited to the above-mentioned division, as long as the corresponding functions can be achieved; in addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of the present invention.
[0122] Note that the above are only preferred embodiments of the present invention and the technical principles used. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and that various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in more detail through the above embodiments, the present invention is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present invention, and the scope of the present invention is determined by the scope of the appended claims.
Claims
1. A method for extracting a vascular lumen, characterized in that: include: Acquire an image with blood vessels to be extracted; Inputting the image to be extracted into a trained vascular lumen extraction model to obtain the vascular lumen of the image to be extracted; wherein the trained vascular lumen extraction model is trained based on multiple groups of historical vascular images and historical annotated images corresponding to the historical vascular images; The training method of the vascular lumen extraction model includes: obtaining multiple groups of historical vascular images and historical annotated images corresponding to the historical vascular images, wherein the historical annotated images are images after outlining the inner and outer walls of the blood vessels in the historical vascular images; inputting the multiple groups of historical vascular images and the historical annotated images into the vascular lumen extraction model, iteratively training the vascular lumen extraction model, and obtaining the trained vascular lumen extraction model; The iterative training of the vascular lumen extraction model includes: classifying the historical vascular images based on the vascular lumen extraction model to obtain the vascular inner and outer wall classification images of the historical vascular images; comparing the vascular inner and outer wall classification images of the historical vascular images with the corresponding historical annotated images to determine the loss function of the vascular lumen extraction model, and determining that the vascular lumen extraction model training is completed when the loss function is less than a preset threshold; Wherein, the determining of the loss function of the vascular lumen extraction model includes: The loss function of the vascular lumen extraction model is determined based on the following formula: For any sample (x, y), y is the true classification result corresponding to the sample, and the predicted classification result is the set of all classification results. Assuming there are k label values, the probability that the i-th sample is predicted to be the K-th label is p i,k , N is the total number of samples, L log is the loss function between the preset classification results and the real classification results of the historical vascular images; Before inputting the plurality of groups of historical vascular images and the historical annotated images into the vascular lumen extraction model, the method further includes: Based on preset screening rules, the historically annotated images are screened to remove unqualified historically annotated images; Wherein, the unqualified historically annotated images include: historically annotated images with incorrect delineation of the inner and outer walls of blood vessels and / or historically annotated images with missed delineation of the inner and outer walls of blood vessels; Correspondingly, after screening the historical annotated images, the historical blood vessel images corresponding to the eliminated historical annotated images are eliminated; The preset screening rule is a preset rule for deleting historically annotated images in which the inner and outer walls of blood vessels are incorrectly delineated and / or historically annotated images in which the inner and outer walls of blood vessels are not delineated.
2. The method according to claim 1, characterized in that Before inputting the plurality of groups of historical vascular images and the historical annotated images into the vascular lumen extraction model, the method further includes: Performing image enhancement according to preset enhancement rules on the filtered historical vascular image and the filtered historical annotated image to obtain a target historical vascular image and a target historical annotated image corresponding to the target historical vascular image; Correspondingly, the step of inputting the plurality of groups of historical vascular images and the historical annotated images into the vascular lumen extraction model includes: A plurality of groups of the target historical blood vessel images and the target historical annotated images are input into the blood vessel lumen extraction model.
3. The method according to claim 2, characterized in that The preset enhancement rule includes: flipping the filtered historical blood vessel image and the filtered historical annotated image up and / or down and / or left and / or right respectively.
4. A blood vessel lumen extraction device, characterized in that: include: An image acquisition module to be extracted, used to acquire an image to be extracted having blood vessels; A vascular lumen extraction module, used for inputting the image to be extracted into a trained vascular lumen extraction model to obtain the vascular lumen of the image to be extracted; wherein the trained vascular lumen extraction model is trained based on multiple groups of historical vascular images and historical annotated images corresponding to the historical vascular images; The vascular lumen extraction device further includes: a historical image acquisition module, used to acquire multiple groups of historical vascular images and historical annotated images corresponding to the historical vascular images, wherein the historical annotated images are images after the inner and outer walls of the blood vessels in the historical vascular images are outlined; a model training module, used to input the multiple groups of historical vascular images and the historical annotated images into the vascular lumen extraction model, iteratively train the vascular lumen extraction model, and obtain the trained vascular lumen extraction model; The model training module includes: a classification unit, which is used to classify the historical blood vessel images based on the blood vessel lumen extraction model to obtain the classified images of the inner and outer walls of the blood vessels of the historical blood vessel images; a model completion training unit, which is used to compare the classified images of the inner and outer walls of the blood vessels of the historical blood vessel images with the corresponding historical annotated images to determine the loss function of the blood vessel lumen extraction model, and when the loss function is less than a preset threshold, it is determined that the training of the blood vessel lumen extraction model is completed; The model completion training unit is specifically used for: The loss function of the vascular lumen extraction model is determined based on the following formula: For any sample (x, y), y is the true classification result corresponding to the sample, and the predicted classification result is the set of all classification results. Assuming there are k label values, the probability that the i-th sample is predicted to be the K-th label is p i,k , N is the total number of samples, L log is the loss function between the preset classification results and the real classification results of the historical vascular images; The device further comprises: A first screening module is used to screen the historically annotated images based on preset screening rules, and remove unqualified historically annotated images; wherein the unqualified historically annotated images include: historically annotated images with incorrect delineation of the inner and outer walls of blood vessels and / or historically annotated images with missed delineation of the inner and outer walls of blood vessels; A second screening module is used to remove the historical blood vessel images corresponding to the removed historical annotated images; The preset screening rule is a preset rule for deleting historically annotated images in which the inner and outer walls of blood vessels are incorrectly delineated and / or historically annotated images in which the inner and outer walls of blood vessels are not delineated.
5. An electronic device, characterized in that: The electronic device comprises: one or more processors; A storage device for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the blood vessel lumen extraction method as described in any one of claims 1-3.
6. A storage medium containing computer executable instructions, characterized in that: The computer executable instructions are used to perform the blood vessel lumen extraction method as described in any one of claims 1 to 3 when executed by a computer processor.
Citation Information
Patent Citations
Blood vessel extraction method, system and device and storage medium
CN111062963A
