Image processing system and image processing method

Abstract

To accurately divide an anatomical structure covered with a shield pattern in medical image data.SOLUTION: An image processing system includes a removal part and a complementation dividing part. The removal part removes a shield pattern area in medical image data including a target anatomical structure and a shield pattern area, and generates a non-shield pattern data. The complementation dividing part uses medical image data that does not include the shield pattern area and / or deleted image data which deletes at least a part of the medical image data that does not include the shield pattern area, trains an image complementation model and an image division model or an acquisition model, divides the target anatomical structure from the complementation medical image data by using the image division model after complementing image information with respect to the non-shield pattern data by using the image complementation model and generating the complementation medical image data, and directly acquires the target anatomical structure from the non-shield pattern data by using the acquisition model.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The embodiments disclosed in this specification and the drawings relate to an image processing system and an image processing method.

[0002] In particular, embodiments of the present application relate to image processing systems and methods capable of segmenting anatomical structures covered by occluding patterns in medical image data. [Background technology]

[0003] In the prior art, methods for segmenting anatomical structures in medical image data include segmentation methods using deep learning networks, structure fitting, or geometric reconstruction.

[0004] However, with changes in human body functions, many anatomical structures may become calcified, and when anatomical structures in medical image data are covered by occlusion patterns such as calcified areas, it is difficult to present true anatomical structure information in the medical image, and the above-mentioned methods in the prior art have difficulty in performing effective segmentation.

[0005] An example of segmenting an aortic valve included in medical image data will be described below. In Fig. 22, the left diagram shows an image including a normal aortic valve (hereinafter, normal data) (in the diagram, the aortic valve is indicated by the reference numeral 221), and the right diagram shows an example of an image in which the region corresponding to the aortic valve in the same image as the left diagram is shown as a mask (mesh portion in the diagram) (in the diagram, the mask of the region corresponding to the aortic valve is indicated by the reference numeral 222). As shown in Fig. 22, the region corresponding to the normal aortic valve can be accurately segmented from the image using a conventional segmentation method.

[0006] On the other hand, in the case of an image including a calcified aortic valve (hereinafter, calcification data) as shown in the left diagram of Fig. 23 (in the diagram, the aortic valve is indicated by reference numeral 231, and the calcified portion is indicated by reference numeral 232), since the aortic valve is covered with calcium, it is difficult to observe its complete anatomical structure, and when the aortic valve is segmented using the conventional segmentation method, the structure of the aortic valve cannot be accurately segmented as shown in the right diagram of Fig. 23 (in the diagram, the segmented area is indicated by reference numeral 233). The reason is as follows.

[0007] While common segmentation techniques and learning models are targeted at normal structures, calcification data contains high-intensity calcium regions, which have image intensity distributions that are significantly different from those of normal data. Therefore, current mainstream segmentation techniques and models perform poorly when segmenting calcification data.

[0008] Therefore, although it is possible to segment the aortic valve in the calcification data using the conventional segmentation method, the structure of the segmented aortic valve is not accurate, making it difficult to perform accurate analysis and measurement thereon or to accurately analyze the degree of calcification.

[0009] To address the above-mentioned problems, a deep learning method using calcification data can be considered. However, deep learning technology is data-driven, and requires annotation to label a large amount of data. On the other hand, calcification data has few samples and is difficult to obtain in large quantities. With a small number of calcification data samples, it is difficult to improve the accuracy of the training model, and adding only a small amount of calcification data to the training dataset may even have a negative effect on the accuracy of normal data.

[0010] In addition, annotation of calcification data is difficult and labor-intensive, and the accuracy and consistency of the labels cannot be guaranteed.

[0011] Therefore, in the conventional technology, it is difficult to perform aortic valve segmentation using a model that uses data on a calcified aortic valve.

[0012] Another possible method is to segment the calcified aortic valve region and the structures related to the valve together. However, as shown in the left diagram of Fig. 24, the edge of the calcified aortic valve is irregular (in the diagram, the aortic valve is indicated by reference number 241, and the calcified portion is indicated by reference number 242), so the edge of the segmentation result is also irregular (in the diagram, the segmented region is indicated by reference number 243) as shown in the right diagram of Fig. 24. This makes it difficult for doctors to accurately understand the original structure of the aortic valve, and makes subsequent clinical analysis difficult because the segmentation result differs depending on the irregularity of calcification.

[0013] In view of the above, embodiments of the present application provide an image processing system and method capable of accurately segmenting anatomical structures covered by occlusion patterns in medical image data. [Prior art documents] [Patent documents]

[0014] [Patent Document 1] JP 2018-196742 A [Patent Document 2] JP 2020-062371 A [Patent Document 3] International Publication No. 2016 / 159379 Summary of the Invention [Problem to be solved by the invention]

[0015] One of the problems to be solved by the embodiments disclosed in this specification and the drawings is to accurately segment anatomical structures covered by occlusion patterns in medical image data. However, the problems to be solved by the embodiments disclosed in this specification and the drawings are not limited to the above problem. Problems corresponding to the effects of each configuration shown in the embodiments described later can also be positioned as other problems. [Means for solving the problem]

[0016] The image processing system according to the embodiment is an image processing system for processing medical image data including a target anatomical structure and an occluding pattern region, and includes a removing unit for the occluding pattern region, a complementary division unit for image data, and a display unit. The removing unit for the occluding pattern region removes the occluding pattern region in the medical image data to generate unoccluded pattern data. The complementary division unit for image data trains an image completion model and an image segmentation model, or an acquisition model of a target anatomical structure, using medical image data not including the occluding pattern region and / or erased image data in which at least a part of the medical image data not including the occluding pattern region is erased, and then uses the image completion model to complement image information on the unoccluded pattern data to generate complementary medical image data, and then uses the image segmentation model to segment the target anatomical structure from the complementary medical image data, or uses the acquisition model of the target anatomical structure to directly acquire the target anatomical structure from the unoccluded pattern data. The display unit displays the target anatomical structure. [Brief description of the drawings]

[0017] [Figure 1] FIG. 1 is a block diagram showing the configuration of an image processing system according to the first embodiment. [Diagram 2] FIG. 2 is a flow chart showing the training and inference of a calcification region extraction model. [Diagram 3] FIG. 3 is a diagram showing an example of segmenting a calcified region from valve image data having the calcified region. [Figure 4] FIG. 4 is a flow chart illustrating the training and inference of an image completion model. [Diagram 5] FIG. 5 shows an example in which a complete valve is formed by complementing valve image data from which calcified regions have been removed. [Figure 6] FIG. 6 is a flowchart illustrating training and inference of an image segmentation model. [Figure 7] FIG. 7 is a diagram showing an example of valve image data in which a calcified region has been complemented by an image complementation model, and an example of a valve mask output for the valve image data after the complementation. [Figure 8] FIG. 8 is a Dice distribution diagram of a valve divided using the conventional division method and the division method by the image processing system according to the first embodiment. [Figure 9] FIG. 9 is a Hausdorff distance distribution diagram of a valve divided using the conventional division method and the division method by the image processing system according to the first embodiment. [Figure 10] FIG. 10 is a diagram comparing the effect of division when the calcified valve data of a moderately calcified valve is directly divided and when the data is divided by the image processing system according to the first embodiment. [Figure 11] FIG. 11 is a diagram comparing the effect of division when calcified valve data of a severely calcified valve is directly divided and when the data is divided by the image processing system according to the first embodiment. [Figure 12] FIG. 12 is a flowchart of the image processing method according to the first embodiment. [Figure 13] FIG. 13 is a block diagram showing the configuration of an image processing system according to the second embodiment. [Figure 14] FIG. 14 is a diagram illustrating the training of an acquired model of a target anatomical structure. [Figure 15] FIG. 15 is a flowchart of an image processing method according to the second embodiment. [Figure 16] FIG. 16 is a block diagram showing the configuration of an image processing system according to the third embodiment. [Figure 17] FIG. 17 is a flowchart of an image processing method according to the third embodiment. [Figure 18] FIG. 18 is a block diagram showing the configuration of an image processing system according to the fourth embodiment. [Figure 19] FIG. 19 is a flowchart of an image processing method according to the fourth embodiment. [Figure 20] FIG. 20 is a block diagram showing the configuration of an image processing system according to the fifth embodiment. [Figure 21]FIG. 21 is a flowchart of an image processing method according to the fifth embodiment. [Figure 22] FIG. 22 is a diagram showing an example of segmenting an aortic valve included in medical image data. [Figure 23] FIG. 23 is a diagram showing an example of a calcified aortic valve divided by a conventional division method. [Figure 24] FIG. 24 is a diagram showing an example of segmentation of the calcified aortic valve region and the correlation structure as a whole. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0018] An image processing system according to an embodiment of the present application is an image processing system that processes medical image data including a target anatomical structure and an occluding pattern region, and includes: an occluding pattern region removal unit that removes the occluding pattern region in the medical image data and generates unoccluded pattern data; an image data complementation and division unit that trains an image completion model and an image segmentation model, or an acquisition model of a target anatomical structure, using medical image data that does not include the occluding pattern region and / or erased image data in which at least a portion of the medical image data that does not include the occluding pattern region is erased, complements image information on the unoccluded pattern data using the image completion model to generate complemented medical image data, and then segments the target anatomical structure from the complemented medical image data using the image segmentation model, or directly acquires the target anatomical structure from the unoccluded pattern data using the acquisition model of the target anatomical structure; and a display unit that displays the target anatomical structure.

[0019] The image processing system may further include a complement determination unit that determines whether or not to complement the image information using the image complement model, depending on a feature of the unoccluded pattern data.

[0020] The image processing system may further include a complementary image evaluation unit that evaluates whether or not to segment the target anatomical structure using the image segmentation model depending on characteristics of the complementary medical image data.

[0021] The image processing system may decide whether or not to segment the target anatomical structure by the image segmentation model depending on a user's assessment of features of the complementary medical image data.

[0022] The image processing system may further include an occlusion pattern region identification unit that identifies a region of interest in the occlusion pattern region based on anatomical information of an abnormal area in medical image data including the target anatomical structure and the occlusion pattern region, and the image data complementary division unit may select a method of acquiring the target anatomical structure according to the anatomical information of the region of interest.

[0023] In the image processing system, the image data complementation and division unit generates the complementary medical image data using a deep learning network, and in the deep learning network, the erased image data is used as an input, medical image data that does not include the occluding pattern area is used as a true value to train the image completion model, and the complementary medical image data may be generated by inputting the unoccluded pattern data to the image completion model.

[0024] In the image processing system, the deep learning network may be a generative adversarial network.

[0025] In the image processing system, when training an acquisition model of the target anatomical structure, the image data complementary division unit may use the erased image data as input and the medical image data not including the occlusion pattern region as true values, learn associated low-dimensional and high-dimensional complementary features by a self-supervised learning network, and share the learned associated low-dimensional and high-dimensional complementary features as encoder weights, thereby applying them to a deep learning network for acquiring the target anatomical structure.

[0026] In the image processing system, the medical image data may be two-dimensional image data or three-dimensional image data collected by a modality.

[0027] In the image processing system, the occlusion pattern region may be a calcified anatomical structure region.

[0028] In the image processing system, the occluded pattern region may be a region where image noise exists or a region where a treatment device is embedded, and the occluded pattern region removal unit may remove the occluded pattern region selected by a user and generate the unoccluded pattern data. It may be configured.

[0029] An image processing method according to an embodiment of the present application is an image processing method for processing medical image data including a target anatomical structure and an occluding pattern region, the method including: a occluding pattern region removal step for removing the occluding pattern region in the medical image data to generate unoccluded pattern data; a training step for an image completion model and an image segmentation model, or an acquisition model of a target anatomical structure, using medical image data not including the occluding pattern region and / or erased image data in which at least a part of the medical image data not including the occluding pattern region is erased, and a complementary segmentation step for image data for generating complementary medical image data by complementing image information on the unoccluded pattern data using the image completion model, and then segmenting the target anatomical structure from the complementary medical image data using the image segmentation model, or directly acquiring the target anatomical structure from the unoccluded pattern data using the acquisition model of the target anatomical structure; and a display step for displaying the target anatomical structure.

[0030] The image processing method may further include a completion determination step of determining whether or not to complement the image information using the image completion model, depending on a feature of the unoccluded pattern data.

[0031] The image processing method may further include a complementary image evaluation step of evaluating whether or not to segment the target anatomical structure using the image segmentation model depending on characteristics of the complementary medical image data.

[0032] The image processing method may determine whether or not to segment the target anatomical structure using the image segmentation model depending on a user's assessment of features of the complementary medical image data.

[0033] The image processing method may further include a step of identifying an occluding pattern region, based on anatomical information of an abnormal area in medical image data including the target anatomical structure and an occluding pattern region, identifying a region of interest in the occluding pattern region, and in a complementary division step of the image data, selecting a method for obtaining the target anatomical structure according to the anatomical information of the region of interest.

[0034] The image processing method may generate the complementary medical image data by a deep learning network in the complementary division step of the image data, and may train the image completion model using the erased image data as input and medical image data that does not include the occluding pattern area as a true value in the deep learning network, and generate the complementary medical image data by inputting the unoccluded pattern data to the image completion model.

[0035] In the image processing method, the deep learning network may be a generative adversarial network.

[0036] In the image processing method, in the complementary division step of the image data, when training an acquisition model of the target anatomical structure, the erased image data is used as an input, and medical image data not including the occlusion pattern region is used as a true value, and associated low-dimensional and high-dimensional complementary features are learned by a self-supervised learning network, and the learned associated low-dimensional and high-dimensional complementary features are shared as encoder weights, thereby being applied to a deep learning network for acquiring the target anatomical structure.

[0037] In the image processing method, the medical image data may be two-dimensional image data or three-dimensional image data collected by a modality.

[0038] In the image processing method, the occlusion pattern region may be a calcified anatomical structure region.

[0039] In the image processing method, the occluded pattern area may be an area where image noise exists or an area where a treatment device is embedded, and in the occluded pattern area removal step, the occluded pattern area selected by a user may be removed to generate the unoccluded pattern data.

[0040] Hereinafter, an image processing system and an image processing method according to an embodiment of the present application will be described in detail with reference to the drawings.

[0041] The image processing system according to the embodiment of the present application is composed of multiple functional modules, and can be installed as software in a device such as an independent computer having a CPU (Central Process Unit) and memory, or can be distributed and installed in multiple devices, and is realized by executing each functional module of the image processing system stored in the memory with a processor. The image processing system may also be realized in the form of hardware as a circuit capable of executing each function of the image processing system. The circuit that realizes the image processing system can send and receive data and collect data via a network such as the Internet.

[0042] The image processing system according to the embodiment of the present application can be installed at a medical image collection site to process images at the site. The image processing system can also be directly implemented in a medical image collection device (also called a modality) such as a CT (Computed Tomography) device or an MRI (Magnetic Resonance Imaging) device as a part of the medical image collection device.

[0043] In the first to fourth embodiments, an example will be described in which image processing is performed on medical image data including a calcified aortic valve (hereinafter, sometimes referred to as a "valve") to divide the valve structure. However, the present embodiment is not limited to this, and the present embodiment may be applied when image data of other calcified anatomical structures is processed, or when medical image data containing an occlusion pattern region is processed.

[0044] Each embodiment of the present application will be described in detail below.

[0045] (First embodiment) First, the first embodiment will be described with reference to FIGS.

[0046] FIG. 1 is a block diagram showing the configuration of an image processing system according to the first embodiment.

[0047] As shown in FIG. 1, the image processing system 1 includes a section 11 for removing calcified anatomical structure regions, a section 12 for complementary division of image data, and a display section 13.

[0048] The calcified anatomical structure region removal unit 11 removes calcified anatomical structure regions from input medical image data including calcified anatomical structure regions (occluded pattern regions) and generates non-calcified anatomical structure data (non-occluded pattern data).

[0049] Here, the medical image data may be data collected by various modalities such as CT data, MRI data, etc. Furthermore, the medical image data may be either two-dimensional image data or three-dimensional image data.

[0050] Moreover, the calcified anatomical structure region is a region in which the anatomical structure is calcified (hereinafter, referred to as the calcified region).

[0051] Specifically, the portion 11 for removing calcified anatomical structure regions includes an extraction model 111 of calcified regions and a removal unit 112 .

[0052] The calcification region extraction model 111 is a model for extracting a calcification region in an anatomical structure. The calcification region extraction model 111 may be realized using an AI (Artificial Intelligence) technique such as deep learning, or may be realized using a conventional algorithm such as threshold processing.

[0053] Here, an example will be described in which a calcification region extraction model 111 is realized by deep learning, for example, a Unet division network.

[0054] As shown in FIG. 2, in the training stage, in step S1, input medical image data including calcification regions are pre-segmented into calcification regions. This pre-segmentation is performed using a conventional segmentation method. Then, in step S2, fine adjustment of labels is performed for the calcification regions pre-segmented in step S1, and ground truth (GT) values ​​are prepared. Such a series of processes for preparing ground truth values ​​for medical image data is called annotation. In addition, in steps S1 and S2, a calcification mask pre-segmented using a conventional algorithm is obtained, thereby improving the annotation efficiency of a doctor when obtaining ground truth values. Then, in step S3, an extraction model 111 of calcification regions is trained. In addition, in step S4, the accuracy of the trained extraction model 111 of calcification regions is evaluated, and the extraction model 111 of calcification regions is prepared.

[0055] In the inference stage, new calcification data is prepared in step S5. That is, medical image data including calcification regions is input. Then, in step S6, calcification regions are segmented (extracted) from the input medical image data using the calcification region extraction model 111 trained in the training stage.

[0056] FIG. 3 shows an example of segmenting a calcified region (hatched region in the right diagram) from valve image data (left diagram) having a calcified region (in the diagram, the valve is indicated by reference numeral 31, and the calcified region is indicated by reference numeral 32).

[0057] The removal unit 112 is a unit that removes calcified regions extracted by the calcified region extraction model 111 to generate calcification-free anatomical structure data. For example, as shown in the left diagram of Fig. 5, the removal unit 112 removes calcified regions by setting the brightness value of the extracted calcified regions to 0 to form black regions (in the diagram, the valves are indicated by reference numeral 31, and the regions from which the calcified regions have been removed are indicated by reference numeral 33).

[0058] The image data complementation and division unit 12 trains an image completion model and an image segmentation model using medical image data that does not contain calcified regions (occluding pattern regions) and / or erased image data in which at least a portion of the medical image data that does not contain calcified regions has been erased, uses the image completion model to complement image information for non-calcified anatomical structure data (unoccluded pattern data) to generate complementary medical image data, and then uses the image segmentation model to segment the target anatomical structure from the complementary medical image data.

[0059] Specifically, the image data complementation / division unit 12 includes an image complementation model 121 and an image division model 122 .

[0060] The image completion model 121 is a model that generates complementary medical image data by complementing non-calcified anatomical structure (non-occluding pattern) data after removing calcified regions. Here, the image completion model 121 is a model trained using, for example, deleted image data in which at least a part of medical image data not including calcified regions is deleted by a random mask. Note that the means for creating the deleted data is not limited to this, and the position of deletion may be controlled based on morphological information of the anatomical structure or distribution of pixel values. In this embodiment, the image data complementation division unit 12 uses the image completion model 121 to complement image information for the non-calcified anatomical structure data generated by the calcified anatomical structure region removal unit 11, and generates complementary medical image data.

[0061] Here, normal medical image data is medical image data that does not have occluding pattern regions, such as calcified anatomical structure regions.

[0062] In addition, any image completion method can be used as the completion method performed by the image completion model 121. For example, a completion method using a deep learning network can be used.

[0063] When the image completion model 121 performs completion using a completion method using a deep learning network, in the training stage, for example, deleted image data in which at least a part of normal medical image data is deleted using a random mask is input, and the normal medical image data is used as a true value to train the image completion model 121. In addition, in the inference stage, non-calcified anatomical structure data after removing calcified regions is input to the image completion model 121 to generate completed medical image data in which the removed calcified regions are completed.

[0064] For example, since calcified regions in anatomical structures are irregular and vary in size, and there may be cases where severe calcification exists, i.e., the removed calcified region is large, it is possible to train the image completion model 121 using a condition-based Generative Adversarial Network (GAN) completion method. In the following, an example of training the image completion model 121 using a condition-based GAN completion method will be described with reference to FIG. 4.

[0065] As shown in Fig. 4, in the training stage, in step S11, medical image data including a normal valve structure is selected as a training data set for the model. In addition, in step S12, at least a part of the valve region in the medical image data is erased by a random mask. Then, in step S13, the medical image data from which at least a part of the valve region is erased in step S12 is used as an input of the training data set, and the medical image data including the normal valve structure selected in step S11 is used as a true value to train an image completion model 121 that completes the erased region in step S14. Finally, in step S15, the accuracy of the trained image completion model 121 is evaluated.

[0066] Here, in the training stage, a large amount of existing medical image data having normal valve structures is used to train the image completion model 121. At this time, a random mask is generated using a preset random seed, and at least a part of the valve region in the medical image is randomly erased by the generated random mask to generate a plurality of medical image data from which at least a part of the valve region is erased. The generated plurality of medical image data are input to the image completion model 121, thereby improving the accuracy of completion.

[0067] Also, in the inference stage, in step S16, medical image data (in this embodiment, medical image data including valves (also called valve image data)) from which calcified regions have been removed by the calcified anatomical structure region remover 11 is obtained. Then, in step S17, the removed calcified regions are complemented using the image complementation model 121 trained in the training stage, thereby restoring the complete valve structure in the image data.

[0068] FIG. 5 shows an example in which valve image data (left) from which calcified regions have been removed is complemented to form a complete valve (right). (In the figure, the valve is indicated by the reference symbol 31, and the portion from which calcified regions have been removed is indicated by the reference symbol 33.)

[0069] The image segmentation model 122 is a model for segmenting medical image data including complete anatomical structures. Here, the image segmentation model 122 is a model trained using normal medical image data. In this embodiment, the image data complementation segmentation unit 12 uses the image segmentation model 122 to segment a target anatomical structure corresponding to a calcified anatomical structure region from the complemented medical image data generated by the image complementation model 121.

[0070] In this embodiment, the image segmentation model 122 is trained using a Unet segmentation network, but the image segmentation model 122 may be trained using other deep learning networks.

[0071] As shown in Fig. 6, in the training stage, in step S21, valve annotation is performed on normal medical image data. Then, in step S22, an image segmentation model 122 that segments the valve structure is trained using AI technology. Finally, in step S23, the accuracy of the trained image segmentation model 122 is evaluated to prepare the image segmentation model 122. Note that annotation may also be performed on image data in which calcified regions generated in step S17 in the past are stored, and training may be performed using these.

[0072] Also, in the inference stage, in step S24, valve image data in which calcification regions have been complemented by the image complement model 121 is acquired. Then, in step S25, the image segmentation model 122 trained in the training stage is used to segment the complemented valve image data acquired in step S24 to segment the valve structure. Then, in step S26, masks of the segmented valves are output.

[0073] 7 shows an example of valve image data (left) in which a calcified region has been complemented by the image complement model 121, and an example of image data in which the complemented valve image data has been superimposed with a valve mask (right) output in step S26 of the inference stage (in the figure, the valve is indicated by reference numeral 31, and the valve mask is indicated by reference numeral 34).

[0074] The display unit 13 displays the target anatomical structure acquired by the image data complementary segmentation unit 12. Specifically, in this embodiment, the display unit 13 displays a mask of the valve segmented and output by the image segmentation model 122.

[0075] The effect of dividing the valves by the image processing system 1 will be described below with reference to FIGS.

[0076] Using an image segmentation model trained on normal medical image data, the calcified valve data was directly segmented, and the calcified valve data complemented by the image processing system 1 according to this embodiment was segmented. The segmentation results were compared with the true value of the aortic valve. The performance of each segmentation result is shown in Figures 8 and 9. Here, 11 sets of 2D cross-sectional image data of the center position of a typical calcified valve were used as the test data set.

[0077] FIG. 8 is a Dice distribution plot of the test data set, and FIG. 9 is a Hausdorff distance distribution plot of the test data set.

[0078] In Fig. 8, the dashed line indicates the division effect of the image processing system 1 according to this embodiment, and the solid line indicates the division effect when the calcified valve data is directly divided. The Dice distribution diagram proves that the closer the Dice value is to 1, the better the division effect. Therefore, it can be seen from Fig. 8 that the division effect of the image processing system 1 according to this embodiment is better.

[0079] In Fig. 9, the dashed line indicates the division effect of the image processing system 1 according to this embodiment, and the solid line indicates the division effect when the calcified valve data is directly divided. In the Hausdorff distance distribution diagram, it is proven that the closer the value is to 0, the better the division effect. Therefore, it can be seen from Fig. 9 that the division effect of the image processing system 1 according to this embodiment is better.

[0080] Fig. 10 is a diagram comparing the effect of division when the calcified valve data of a moderately calcified valve is directly divided and when the data is divided by the image processing system 1 according to this embodiment. Also, Fig. 11 is a diagram comparing the effect of division when the calcified valve data of a severely calcified valve is directly divided and when the data is divided by the image processing system 1 according to this embodiment.

[0081] In FIG. 10, the upper left diagram is image data of a moderately calcified valve, the lower left diagram is a diagram showing the true value of the valve mask when the image data of the moderately calcified valve is divided, the upper center diagram is the same image data as the upper left diagram, the lower center diagram is a diagram showing the valve mask obtained when the image data of the calcified valve is directly divided using a division model trained based on normal medical image data, the upper right diagram is complemented image data obtained by complementing the image data of the moderately calcified valve of the upper left diagram by the image processing system 1, and the lower right diagram is a diagram showing the valve mask output after the image processing system 1 divides the complemented image data. Here, the Dice value of the valve division for the lower center diagram is 0.19, and the Dice value of the valve division for the lower right diagram is 0.81. That is, the division effect of the image processing system 1 according to this embodiment is better. Of course, as is clear from FIG. 10, the division mask shown in the lower right diagram is closer to the true value of the division mask shown in the lower left diagram.

[0082] Therefore, as can be seen from FIG. 10, the image processing system 1 of this embodiment can achieve good segmentation optimization effects on data with moderate calcification.

[0083] In FIG. 11, the upper left diagram is image data of a severely calcified valve, the lower left diagram is a diagram showing the true value of the valve mask when the image data of the severely calcified valve is divided, the upper center diagram is the same image data as the upper left diagram, the lower center diagram is a diagram showing the valve mask obtained when the image data of the calcified valve is directly divided using a division model trained based on normal medical image data, the upper right diagram is complemented image data obtained by complementing the image data of the severely calcified valve of the upper left diagram by the image processing system 1, and the lower right diagram is a diagram showing the valve mask output after the image processing system 1 divides the complemented image data. Here, the Dice value of the valve division for the lower center diagram is 0.24, and the Dice value of the valve division for the lower right diagram is 0.79. That is, the division effect of the image processing system 1 according to this embodiment is better. Of course, as is clear from FIG. 11, the division mask shown in the lower right diagram is closer to the true value of the division mask shown in the lower left diagram.

[0084] Until now, it has been very difficult to segment data with severe calcification using conventional segmentation methods. However, as can be seen from FIG. 11, the image processing system 1 according to this embodiment can achieve good segmentation optimization effects even for data with severe calcification.

[0085] Hereinafter, with reference to FIG. 12, the image processing method according to the first embodiment will be described.

[0086] In step S101, the calcified anatomical structure region removal unit 11 removes calcified anatomical structure regions from the input valve image data, which is medical image data including calcified anatomical structure regions (occlusion pattern regions), and generates non-calcified anatomical structure data.

[0087] In step S102, the image data complementation and division unit 12 trains the image completion model 121 and the image segmentation model 122 using normal medical image data, complements image information for the non-calcified anatomical structure data using the image completion model 121 to generate complemented medical image data, and then uses the image segmentation model 122 to segment the valve structure, which is the target anatomical structure corresponding to the calcified anatomical structure region, from the complemented medical image data.

[0088] In step S103, the display unit 13 displays the valve structure divided in step S102.

[0089] In this embodiment, the problem of a small amount of calcification data samples in training a segmentation model is addressed by eliminating the need for training using calcification data during the processing of the image processing system 1, and in addition, a better segmentation effect of the calcification data can be obtained.

[0090] In addition, in this embodiment, a large amount of normal medical image data is used to train the image completion model, which can effectively learn the features of the target anatomical structure and can complete the target anatomical structure covered by the calcified region. Furthermore, the completed data can be segmented using a normal segmentation model to obtain excellent segmentation accuracy.

[0091] In addition, in this embodiment, the segmentation results of the image processing system 1 can be used as pre-segmentation results when a doctor annotates calcification data. This eliminates the need for a doctor to annotate the calcification data from the beginning, thereby improving annotation efficiency.

[0092] In addition, in this embodiment, the calcified region and the target anatomical structure can be separated, and the problem that the subsequent clinical analysis becomes difficult due to the irregular edges of the segmentation result when the calcified region and the target anatomical structure are segmented together can be solved. In addition, the interference by the calcified region can be avoided, and the structural features of the target anatomical structure can be obtained more accurately.

[0093] Second embodiment Next, a second embodiment will be described with reference to FIGS.

[0094] The image processing system 1a according to the second embodiment differs from the image processing system 1 according to the first embodiment in that the configuration of the image data complementation division unit is different, and all other configurations are the same. In the following, only the parts that differ from the first embodiment will be described, and the same parts will be given the same reference numerals, and duplicated explanations will be omitted.

[0095] As shown in FIG. 13, an image processing system 1a according to the second embodiment includes a removal unit 11 for calcified anatomical structure regions, a complementary division unit 12a for image data, and a display unit 13. The image processing system 1a includes a calcified anatomical structure region removal unit 11, a calcified anatomical structure region removal unit 12a, and a display unit 13.

[0096] The image data complementary division unit 12a trains an acquisition model of a target anatomical structure using erased image data in which at least a portion of medical image data that does not include calcified regions (occluding pattern regions) is erased, and directly acquires the target anatomical structure from non-calcified anatomical structure data (non-occluding pattern data) using the acquisition model of the target anatomical structure.

[0097] In particular, the image data complementary segmentation unit 12a comprises an acquired model 123 of the target anatomical structure.

[0098] The target anatomical structure acquisition model 123 is a model that directly acquires a target anatomical structure corresponding to a calcified anatomical structure region from non-calcified anatomical structure (non-occluding pattern) data. Here, the target anatomical structure acquisition model 123 is a model trained using, for example, erased image data in which at least a part of medical image data not including a calcified region is erased by a random mask. Note that the means for creating the erased data is not limited to this, and the position to be erased may be controlled based on the morphological information of the anatomical structure or the distribution of pixel values. In this embodiment, the image data complementation division unit 12a directly acquires a target anatomical structure from the non-calcified anatomical structure data generated by the removal unit 11 of the calcified anatomical structure region using the target anatomical structure acquisition model 123.

[0099] FIG. 14 shows training of the target anatomical structure acquisition model 123. In the training, for example, erased image data in which at least a part of normal medical image data is erased by a random mask is input to an encoder of a self-supervised learning net (SSLNet), and a complemented image in which the erased area is complemented is output from the decoder of the self-supervised learning net with the normal medical image data as the true value. In addition, in the training, erased image data in which at least a part of normal medical image data is erased by a random mask is input to the encoder of a division network, and a division mask of the target anatomical structure corresponding to the area erased by the random mask is output from the decoder of the division network with the normal medical image data as the true value. Here, the division network may be any deep learning division network.

[0100] Here, in the training phase, in the completion based on the self-supervised learning network, relevant low-dimensional and high-dimensional completion features are learned by the self-supervised learning network, and the learned relevant low-dimensional and high-dimensional completion features are applied to the segmentation network to acquire the target anatomical structure by sharing them as encoder weights.

[0101] In this way, when training the target anatomical structure acquisition model 123 using image data of a normal aortic valve, by inputting image data in which at least a portion of the aortic valve has been erased, it is possible to output an aortic valve structure (a segmented mask of the aortic valve) in which at least a portion of the erased aortic valve has been complemented.

[0102] In addition, in the inference stage, the valve structure can be output (directly acquired) by inputting valve data of a calcified anatomical structure after removing calcified regions into the acquisition model 123 of the target anatomical structure.

[0103] Hereinafter, with reference to FIG. 15, an image processing method according to the second embodiment will be described.

[0104] In step S201, the calcified anatomical structure region removal unit 11 removes calcified anatomical structure regions from the input valve image data, which is medical image data including calcified anatomical structure regions (occlusion pattern regions), and generates non-calcified anatomical structure data.

[0105] In step S202, the image data complementary division unit 12a trains an acquisition model of the target anatomical structure using normal medical image data from which at least a portion has been randomly erased, and directly acquires, using the acquisition model of the target anatomical structure, a valve structure, which is a target anatomical structure corresponding to the calcified anatomical structure region, from the non-calcified anatomical structure data generated in step S201.

[0106] In step S203, the display unit 13 displays the valve structure divided in step S202.

[0107] In this embodiment, in the training stage of the target anatomical structure acquisition model 123, the complementary features learned by the self-supervised learning network are shared as encoder weights in the segmentation network, so that the segmentation network can include both complementary and segmentation functions. As a result, in the inference stage, the image data complementary and segmentation unit 12a does not require an image complementary model, and the workflow of the inference stage can be simplified. In addition, the anatomical structure covered by the calcified region can be directly segmented from the data from which the calcified region has been removed.

[0108] (Third embodiment) Next, a third embodiment will be described with reference to FIGS.

[0109] 16, an image processing system 1b according to the third embodiment includes a calcified anatomical structure region removal unit 11, an image data complementation / division unit 12, a display unit 13, a complement determination unit 14, and a complement image evaluation unit 15. Among these, the calcified anatomical structure region removal unit 11, the image data complementation / division unit 12, and the display unit 13 have the same functions as the corresponding units in the first embodiment, and therefore descriptions thereof will be omitted here.

[0110] The complementation determination unit 14 determines whether or not to complement image information using the image complementation model 121 in the image data complementation division unit 12, depending on the characteristics of the non-calcified anatomical structure (non-occluded pattern) data.

[0111] Here, the features of the calcification-free anatomical data include the number, distribution, and other relevant features of the removed calcified regions, and when processing image data of a calcified aortic valve, the other relevant features include, for example, the size and shape of the heart chamber.

[0112] For example, the complement determination unit 14 sets a threshold value for the number of removed calcified regions, and determines that the complement of the subsequent image information is possible when the number of removed calcified regions is less than the threshold value, and determines that the complement of the subsequent image information is impossible when the number of removed calcified regions is equal to or greater than the threshold value. That is, the complement determination unit 14 determines that the complement of the subsequent image information is possible when the features of the non-calcified anatomical structure data (features of the removed calcified regions) satisfy a predetermined index condition, and determines that the complement of the subsequent image information is impossible when the features of the non-calcified anatomical structure data do not satisfy the predetermined index condition.

[0113] This can reduce the risk that the image information cannot be complemented by the image complement model 121.

[0114] The image processing system 1b may be configured to determine whether or not to complement subsequent image information depending on a user's judgment regarding the characteristics of the calcification-free anatomical structure data, without including the complement determination unit 14. For example, when the user judges that the number of removed calcification regions is small and the distribution regions are concentrated in the calcification-free anatomical structure data, the image processing system 1b determines that the image information can be complemented by the image complement model 121, and when the user judges that the number of removed calcification regions is large or the distribution regions are dispersed in the calcification-free anatomical structure data, the image processing system 1b determines that the image information cannot be complemented by the image complement model 121.

[0115] The complementary image evaluation unit 15 evaluates whether or not to segment the target anatomical structure by the image segmentation model 122 according to the characteristics (quality) of the complementary medical image data. For example, the complementary image evaluation unit 15 determines to perform the subsequent structure segmentation process when the quality of the complementary medical image data is acceptable, and determines not to perform the subsequent structure segmentation process when the quality of the complementary medical image data is unacceptable. For example, when processing image data of a calcified aortic valve, the quality of the complementary medical image data is the continuity of the valve. In this case, the complementary image evaluation unit 15 determines that the quality of the complementary medical image data is acceptable when the valve is continuous, and determines that the quality of the complementary medical image data is unacceptable when the valve is not continuous.

[0116] As a result, the risk that the image segmentation model 122 cannot segment the target anatomical structure can be reduced.

[0117] The image processing system 1b may determine whether or not to segment the target anatomical structure by the image segmentation model 122 depending on the user's evaluation of the features of the complementary medical image data, without including the complementary image evaluation unit 15. For example, the image processing system 1b determines to perform the step of segmenting the target anatomical structure by the image segmentation model 122 when the user judges that the quality of the complementary medical image data is acceptable, and determines not to perform the step of segmenting the target anatomical structure by the image segmentation model 122 when the user judges that the quality of the complementary medical image data is unacceptable.

[0118] Hereinafter, with reference to FIG. 17, an image processing method according to the third embodiment will be described.

[0119] In step S301, the calcified anatomical structure region removal unit 11 removes calcified anatomical structure regions from the input valve image data, which is medical image data including calcified anatomical structure regions (occlusion pattern regions), and generates non-calcified anatomical structure data.

[0120] In step S302, the complement determination unit 14 determines whether the characteristics of the calcification-free anatomical structure data satisfy a predetermined index condition. If the predetermined index condition is not satisfied ("NO" in S302), the processing of the image processing system 1b is terminated. If the predetermined index condition is satisfied ("YES" in S302), the image data complement division unit 12 complements the image information of the calcification-free anatomical structure data using the trained image complementation model to generate complemented medical image data (step S303).

[0121] In step S304, the complementary image evaluation unit 15 determines whether the quality of the complementary medical image data is acceptable or not. If it is determined to be unacceptable ("NO" in S304), the processing of the image processing system 1b is terminated. If it is determined to be acceptable ("YES" in S304), the trained image segmentation model is used to segment the valve structure from the complementary medical image data (step S305).

[0122] In step S306, the display unit 13 displays the valve structure divided in step S305.

[0123] In this embodiment, the risk of failure in interpolation and segmentation of image data can be reduced, and the target anatomical structure can be reliably segmented.

[0124] (Fourth embodiment) Next, a fourth embodiment will be described with reference to FIGS.

[0125] 18, an image processing system 1c according to the fourth embodiment includes a calcified anatomical structure region removal unit 11, an image data complementation / division unit 12, a display unit 13, a complement determination unit 14, a complement image evaluation unit 15, and a calcified anatomical structure region specification unit 16. Among these, the units other than the calcified anatomical structure region specification unit 16 and the image data complementation / division unit 12 have the same functions as the corresponding units in the third embodiment, and therefore will not be described here.

[0126] The calcified anatomical structure region identifying unit 16 identifies a region of interest in the calcified anatomical structure region based on anatomical information of an abnormal part in the input medical image data including the calcified anatomical structure region (occlusion pattern region). The image data complementation division unit 12 selects a method for acquiring a target anatomical structure according to the anatomical information of the region of interest.

[0127] When processing image data of a calcified aortic valve, most of the calcified region of the aortic valve occurs on the aortic arch side, not on the left ventricular outflow tract (LVOT) side. In this case, the calcified anatomical structure region identifying unit 16 identifies a region of interest in the calcified anatomical structure region, i.e., the aortic arch region, based on anatomical information of the calcified region of the valve. Then, the image data complementation and division unit 12 selects a method for acquiring the valve structure according to the anatomical information (size, form, etc.) of the aortic arch region. For example, the image data complementation and division unit 12 may select a deep learning method, or may select to perform complementation and division using a moderate calcification model or a severe calcification model.

[0128] Hereinafter, with reference to FIG. 19, an image processing method according to the fourth embodiment will be described.

[0129] Here, steps S403 to S408 in the image processing method according to the fourth embodiment correspond to steps S301 to S306 in the image processing method according to the third embodiment, respectively, and therefore a description thereof will be omitted here.

[0130] In step S401, the calcified anatomical structure region identifying unit 16 identifies a region of interest in the calcified anatomical structure region based on anatomical information of the calcified site in the input medical image data. In this embodiment, the calcified anatomical structure region identifying unit 16 identifies an aortic arch region in the calcified valve region.

[0131] In step S402, the image data complementary division unit 12 selects a method for acquiring a target anatomical structure, i.e., a valve structure, according to the anatomical information of the region of interest, i.e., the aortic arch region. For example, when the aortic arch region is determined to be moderately calcified, the image data complementary division unit 12 selects to perform the subsequent processes of steps S403 to S408 using a moderate calcification model.

[0132] In this embodiment, a method and a model for acquiring a target anatomical structure can be appropriately selected according to the anatomical information of the region of interest, and the target anatomical structure can be accurately segmented.

[0133] Fifth embodiment Next, a fifth embodiment will be described with reference to FIGS.

[0134] 20, an image processing system 1d according to the fifth embodiment includes an occlusion pattern region removal unit 11d, an image data complement division unit 12, a display unit 13, a complement determination unit 14, and a complement image evaluation unit 15. Among these, the units other than the occlusion pattern region removal unit 11d have the same functions as the corresponding units in the third embodiment, and therefore descriptions thereof will be omitted here.

[0135] The occluded pattern region removal unit 11d removes the occluded pattern region selected by the user to generate non-occluded pattern data. Here, the occluded pattern region is, for example, a region where image noise exists or a region where a medical device is embedded.

[0136] Specifically, the occluding pattern region removal section 11d includes an occluding pattern region extraction model 111d and a removal unit 112d.

[0137] The occlusion pattern region extraction model 111d is a model for extracting an occlusion pattern region selected by a user. Here, the occlusion pattern region extraction model 111d may be realized using an AI technology such as deep learning, or may be realized using a conventional algorithm such as threshold division.

[0138] The removal unit 112d is a unit that removes the occluding pattern region extracted by the extraction model 111d of the occluding pattern region to generate non-occluding pattern data. For example, the removal unit 112d removes the occluding pattern region by forming a black region by setting the brightness value of the extracted occluding pattern region to 0.

[0139] Hereinafter, with reference to FIG. 21, an image processing method according to the fifth embodiment will be described.

[0140] In step S501, the occluding pattern region removal unit 11d removes occluding pattern regions, such as a region in which a treatment device is embedded, selected by the user and present in the medical image data, to generate medical image data without occluding patterns.

[0141] In step S502, the complement determination unit 14 determines whether the characteristics of the data without occlusion patterns satisfy a predetermined index condition. If the predetermined index condition is not satisfied ("NO" in S502), the processing of the image processing system 1d is terminated. If the predetermined index condition is satisfied ("YES" in S502), the image data complement division unit 12 uses the trained image complement model to complement image information for the data without occlusion patterns and generate complemented medical image data (step S503).

[0142] In step S504, the complementary image evaluation unit 15 determines whether the quality of the complementary medical image data is acceptable or not, and if it determines that it is unacceptable ("NO" in S504), it terminates the processing of the image processing system 1d, and if it determines that it is acceptable ("YES" in S504), it uses the trained image segmentation model to segment the target anatomical structure that is partially or completely occluded by the occlusion pattern from the complementary medical image data (step S505).

[0143] In step S506, the display unit 13 displays the target anatomical structure segmented in step S505.

[0144] In this embodiment, anatomical structures occluded by occluding pattern regions, such as regions where image noise is present or where a therapeutic device is implanted, can be reliably segmented.

[0145] Note that each component of each image processing system according to each embodiment described above is functionally conceptual, and does not necessarily have to be physically configured as shown in the drawings. In other words, the specific form of distribution or integration of each system is not limited to that shown in the drawings, and all or a part of it can be functionally or physically distributed or integrated in any unit according to various loads, usage conditions, etc. Furthermore, all or any part of each processing function performed in each system may be realized by a CPU (Central Processing Unit) and a program analyzed and executed by the CPU, or may be realized as hardware using wired logic.

[0146] The image processing system described in each of the above-mentioned embodiments can be realized by a computer such as a personal computer or a workstation executing a program prepared in advance. The program may be distributed via a network such as the Internet. The program may be recorded on a computer-readable non-volatile recording medium such as a hard disk, a flexible disk (FD), a compact disk (CD)-read only memory (ROM), a magneto-optical disk (MO), or a digital versatile disk (DVD), and may be read out from the recording medium and executed by a computer.

[0147] The various data handled in this specification are typically digital data.

[0148] According to at least one of the embodiments described above, anatomical structures covered by occlusion patterns in medical image data can be accurately segmented.

[0149] Although some embodiments have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These embodiments can be implemented in various other forms, and various omissions, substitutions, modifications, and combinations of embodiments can be made without departing from the spirit of the invention. These embodiments and their modifications are included in the scope of the invention and its equivalents as described in the claims, as well as in the scope and spirit of the invention. [Explanation of symbols]

[0150] 1,1a,1b,1c Image processing system 11 Removal of areas of calcified anatomical structures 111 Extraction model of calcified region 112 Removal Unit 12, 12a Image data complementary division section 121 Image Completion Model 122 Image Segmentation Model 123 Acquisition model of target anatomical structure 13 Display section 14 Complementary decision unit 15 Complementary image evaluation unit 16 Specific areas of calcified anatomical structures

Claims

1. 1. An image processing system for processing medical image data including target anatomical structures and occlusion pattern regions, comprising: an occluding pattern region removal unit for removing the occluding pattern region in the medical image data to generate non-occluding pattern data; an image data complementation and division unit that uses the medical image data that does not include the occluding pattern region and / or erased image data in which at least a part of the medical image data that does not include the occluding pattern region is erased to train an image completion model and an image segmentation model, or an acquisition model of a target anatomical structure, complements image information of the unoccluded pattern data using the image completion model to generate complementary medical image data, and then segments the target anatomical structure from the complementary medical image data using the image segmentation model, or directly acquires the target anatomical structure from the unoccluded pattern data using the acquisition model of the target anatomical structure; A display unit for displaying the target anatomical structure; An image processing system comprising:

2. a complementation determination unit that determines whether or not to complement the image information by the image complement model according to a feature of the unoccluded pattern data; The image processing system according to claim 1 .

3. Further comprising a complementary image evaluation unit that evaluates whether or not to segment the target anatomical structure by the image segmentation model according to the characteristics of the complementary medical image data. The image processing system according to claim 1 .

4. determining whether to segment the target anatomical structure using the image segmentation model in response to a user's assessment of features of the complementary medical image data; The image processing system according to claim 1 .

5. The medical image data includes an anatomical structure and an occlusion pattern region, and the occlusion pattern region is determined based on anatomical information of an abnormal part in the medical image data. The image data complementary division unit selects a method for acquiring the target anatomical structure according to anatomical information of the region of interest. The image processing system according to claim 1 .

6. The image data complement division unit The complementary medical image data is generated by a deep learning network; In the deep learning network, the erased image data is used as an input, and the medical image data that does not include the occlusion pattern region is used as a true value to train the image completion model; inputting the unoccluded pattern data into the image completion model to generate the completion medical image data; The image processing system according to claim 1 .

7. The deep learning network is a generative adversarial network. The image processing system according to claim 6.

8. When training an acquisition model of the target anatomical structure, the image data complementation division unit uses the erased image data as an input and the medical image data not including the occlusion pattern region as a true value, learns associated low-dimensional and high-dimensional complement features by a self-supervised learning network, and applies the learned associated low-dimensional and high-dimensional complement features to a deep learning network for acquiring the target anatomical structure by sharing them as encoder weights. The image processing system according to claim 1 .

9. The medical image data is two-dimensional image data or three-dimensional image data collected by a modality. The image processing system according to claim 1 .

10. The occlusion pattern region is a calcified anatomical structure region.

10. The image processing system according to claim 1.

11. The shielding pattern region is a region where image noise exists or a region where a treatment device is embedded, The occluding pattern region removal unit removes the occluding pattern region selected by a user and generates the non-occluded pattern data. The image processing system according to claim 1 .

12. 1. An image processing method for processing medical image data including a target anatomical structure and an occlusion pattern region, comprising: a step of removing the occluding pattern region in the medical image data to generate non-occluding pattern data; an image data complementation and segmentation step of training an image completion model and an image segmentation model, or an acquisition model of a target anatomical structure, using medical image data that does not include the occluding pattern region and / or erased image data in which at least a part of the medical image data that does not include the occluding pattern region is erased, and generating complementary medical image data by complementing image information of the unoccluded pattern data using the image completion model, and then segmenting the target anatomical structure from the complementary medical image data using the image segmentation model, or directly acquiring the target anatomical structure from the unoccluded pattern data using the acquisition model of the target anatomical structure; A display step of displaying the target anatomical structure. An image processing method comprising:

13. a completion determination step of determining whether or not to complete the image information by the image completion model in accordance with a feature of the unobstructed pattern data, The image processing method according to claim 12.

14. Further comprising a complementary image evaluation step of evaluating whether or not to segment the target anatomical structure by the image segmentation model according to characteristics of the complementary medical image data. The image processing method according to claim 12.

15. determining whether to segment the target anatomical structure using the image segmentation model in response to a user's assessment of features of the complementary medical image data; The image processing method according to claim 12.

16. The method further includes a step of identifying an area of ​​interest in the occlusion pattern area based on anatomical information of an abnormal part in medical image data including the target anatomical structure and the occlusion pattern area; In the complementary segmentation step of the image data, a method for acquiring the target anatomical structure is selected according to anatomical information of the region of interest. The image processing method according to claim 12.

17. In the step of complementing and dividing the image data, The complementary medical image data is generated by a deep learning network; In the deep learning network, the erased image data is used as an input, and the medical image data that does not include the occlusion pattern region is used as a true value to train the image completion model; inputting the unoccluded pattern data into the image completion model to generate the completion medical image data; The image processing method according to claim 12.

18. The deep learning network is a generative adversarial network. The image processing method according to claim 17.

19. In the image data complementation and division step, when training an acquisition model of the target anatomical structure, the erased image data is used as an input, and the medical image data not including the occlusion pattern region is used as a true value, and related low-dimensional and high-dimensional complement features are learned by a self-supervised learning network, and the learned related low-dimensional and high-dimensional complement features are applied to a deep learning network for acquiring the target anatomical structure by sharing them as encoder weights. The image processing method according to claim 12.

20. The medical image data is two-dimensional image data or three-dimensional image data collected by a modality. The image processing method according to claim 12.

21. The occlusion pattern region is a calcified anatomical structure region. The image processing method according to any one of claims 12 to 20.

22. The shielding pattern region is a region where image noise exists or a region where a treatment device is embedded, In the removing step of the occluding pattern area, the occluding pattern area selected by a user is removed, and the non-occluded pattern data is generated. The image processing method according to claim 12.

Images