Medical image processing apparatus and medical image processing method

Abstract

The present application provides a kind of medical image processing device and medical image processing method capable of improving the analysis accuracy of medical image and the explainability of analysis result.The medical image processing device has: segmentation unit, obtains medical image, determines the region of interest in medical image;Block extraction unit, extract the block consisting of pixels with specified characteristics in the region of interest as target block;Block extraction correction unit, according to the influencing factor when block extraction unit carries out block extraction, corrects extraction result;Basic feature extraction unit, extract the basic feature of the target block after correction;Clinically interpretable feature generation unit, using the conversion model that establishes the association between basic feature and clinically interpretable feature, according to basic feature, generate the clinically interpretable feature of the target block after correction;And classification unit, according to clinically interpretable feature, classify the medical image containing the target block after correction, obtain classification result.

Description

Technical Field

[0001] The embodiments relate to medical image processing apparatus and medical image processing method. Background Technology

[0002] Currently, medical image acquisition devices such as X-ray imaging equipment, CT scanners, and ultrasound examination devices are widely used. Analyzing medical images obtained by these devices helps in identifying the location of lesions. For example, in the diagnosis of pneumonia, especially infectious pneumonia, the results of medical image analysis are of great clinical significance.

[0003] Currently available identification methods include manual image reading and machine learning-assisted diagnosis.

[0004] Manual image interpretation involves doctors and other qualified personnel reviewing medical images to identify the presence of lesions and the type of disease. However, this method becomes inefficient with the dramatic increase in daily examination volume, and inconsistencies in interpretation results may occur due to differences in doctors' experience levels.

[0005] Machine learning-assisted diagnosis utilizes machine learning to aid in image interpretation. For example, statistical information or basic image features of lesion areas can be extracted from medical images, and then machine learning methods such as support vector machines and random forests can be used to generate models for classifying lesions such as pneumonia. Alternatively, deep learning networks can be generated using training data comprised of existing lesion area cases, and the lesion areas or images can be directly input into the deep learning network for lesion classification, such as pneumonia.

[0006] However, in current applications of machine learning-assisted diagnosis, such as in differentiating pneumonia types, changes in lung grayscale values ​​due to factors like poor breath-holding, heartbeat, and gravity during image acquisition can lead to inaccurate feature extraction. Furthermore, sometimes doctors interpreting images cannot understand how the machine learning-generated model performs classification or what image features play a role in classification, making it difficult to judge the reasonableness of the model's results. This raises the issue of doctors' lack of trust in the auxiliary model.

[0007] Furthermore, existing machine learning-based diagnostic models that take lesion areas or images as input and output lesion classification results do not provide image findings that are of great reference value to doctors' diagnoses, such as distribution characteristics and signs (such as reverse halo sign, cobblestone sign, etc.), thus hindering the automation of image reporting. Summary of the Invention

[0008] The present invention was made in view of the above problems, and its purpose is to provide a medical image processing device and a medical image processing method that can improve the accuracy of medical image analysis and the interpretability of analysis results.

[0009] The medical image processing apparatus of the embodiment includes: a segmentation unit that acquires a medical image and performs prescribed segmentation processing on the medical image to determine a region of interest (ROI) in the medical image; a block extraction unit that extracts blocks composed of pixels having prescribed features from the ROI as target blocks; a block extraction correction unit that corrects the extraction result of the block extraction unit based on influencing factors during block extraction and outputs the corrected block as the corrected target block; a basic feature extraction unit that extracts basic features of the corrected target block; a clinically interpretable feature generation unit that generates clinically interpretable features of the corrected target block based on the basic features extracted by the basic feature extraction unit using a conversion model that establishes a correlation between the basic features and clinically interpretable features; and a classification unit that classifies the medical image containing the corrected target block based on the clinically interpretable features generated by the clinically interpretable feature generation unit and obtains a classification result.

[0010] Furthermore, the medical image processing method in the medical image processing apparatus of the embodiment includes: a segmentation step, acquiring a medical image and performing a prescribed segmentation process on the medical image to determine a region of interest (ROI) in the medical image; a block extraction step, extracting a block composed of pixels with prescribed features from the ROI as a target block; a block extraction correction step, correcting the extraction result of the block extraction step according to the influencing factors during block extraction, and outputting the corrected block as the corrected target block; a basic feature extraction step, extracting basic features of the corrected target block; a clinically interpretable feature generation step, using a conversion model that establishes a correlation between the basic features and clinically interpretable features, generating the clinically interpretable features of the corrected target block based on the basic features extracted in the basic feature extraction step; and a classification step, classifying the medical image containing the corrected target block based on the clinically interpretable features generated in the clinically interpretable feature generation step, and obtaining a classification result.

[0011] According to the above technical solution of this application, by removing the influence of factors that can cause false findings, such as heart rate and gravity effects, the accuracy of classification results can be improved. Furthermore, since pneumonia is classified based on clinically significant and interpretable characteristics, the interpretability is stronger, making it easier to gain the trust of doctors. Simultaneously, it not only improves the accuracy of lesion classification results but also generates clinically significant imaging findings, which is beneficial for the automation of imaging reports. Attached Figure Description

[0012] Figure 1 This is a block diagram illustrating an example of the functional configuration of the medical image processing apparatus according to the first embodiment.

[0013] Figure 2 This is a schematic diagram illustrating the image view tag database in the medical image processing apparatus of the first embodiment.

[0014] Figure 3 (a) is a schematic diagram illustrating the dynamic influencing factor correction model in the first embodiment. Figure 3 (b) is a schematic diagram illustrating the static influencing factor correction model in the first embodiment.

[0015] Figure 4 (a) to (c) are schematic diagrams used to illustrate the block extraction and correction process in the first embodiment.

[0016] Figure 5 This is a schematic diagram illustrating the generation process of clinically interpretable features in the first embodiment.

[0017] Figure 6 This is a flowchart illustrating the lesion image classification processing in the medical image processing apparatus of the first embodiment.

[0018] Figure 7 This is a block diagram illustrating an example of the functional configuration of the medical image processing apparatus according to the second embodiment.

[0019] Figure 8 This is a schematic diagram illustrating an example of an analysis report displayed by the display unit in the second embodiment.

[0020] Figure 9 This is a schematic diagram illustrating another example of the analysis report displayed by the display unit in the second embodiment.

[0021] Figure 10 This is a flowchart illustrating the lesion image classification processing in the medical image processing apparatus of the second embodiment.

[0022] Explanation of reference numerals in the attached figures:

[0023] 100, 200 Medical image processing device; 10 Segmentation unit; 20, 20a Block extraction unit; 30, 30a Block extraction correction unit; 31 Static influencing factor correction unit; 32 Dynamic influencing factor correction unit; 33 Target block judgment unit; 40 Basic feature extraction unit; 50 Clinically interpretable feature generation unit; 51 Quantitative information extraction module; 52 Location information extraction module; 53 Distribution information extraction module; 54 Morphological information extraction module; 55 Lesion degree extraction module; 56 Image sign extraction module; 60, 60a Classification unit; 70 Image appearance label database; 80 Display unit; 90 Nucleic acid detection result prediction unit; 110 Optimization unit. Detailed Implementation

[0024] Hereinafter, with reference to the accompanying drawings, preferred embodiments of the medical image processing apparatus and medical image processing method of the present invention will be described in detail.

[0025] The medical image processing device of this invention comprises multiple functional modules. It can be installed as software in a standalone computer or other device with a CPU (central processing unit) and memory, or it can be distributed across multiple devices, with a processor executing the various functional modules stored in memory. Alternatively, it can be implemented in hardware as a circuit capable of performing the various functions of the medical image processing device. The circuitry implementing the medical image processing device can transmit and receive data or acquire data via a network such as the Internet. Furthermore, the medical image processing device of this invention can also be directly installed within a medical image acquisition device, such as a CT scanner or an MRI scanner.

[0026] Furthermore, in the following description, using a medical image taken of the lungs as an example, the actions of the medical image processing device in the embodiment when classifying the presence and type of lesions in such medical images (determining the type of lesion image) are explained. However, the present invention is not limited to this, and can of course be applied to the classification processing of specific parts in other medical images.

[0027] (First Implementation)

[0028] First, refer to Figures 1 to 6 The first embodiment of the present invention will be described.

[0029] Figure 1 This is a block diagram illustrating an example of the functional configuration of the medical image processing apparatus according to the first embodiment. For example... Figure 1As shown, the medical image processing apparatus 100 of the first embodiment includes a segmentation unit 10, a block extraction unit 20, a block extraction correction unit 30, a basic feature extraction unit 40, a clinically interpretable feature generation unit 50, and a classification unit 60.

[0030] The segmentation unit 10 acquires medical images of the subject and performs prescribed segmentation processing on the medical images to determine the region of interest within them. The acquired medical images can be 3D chest scans such as CT or MRI images, containing the region to be analyzed, such as the lungs. Medical images with other dimensions can also be used.

[0031] After acquiring the medical image, segmentation unit 10 segments the image to identify regions of interest (ROIs). ROIs can be defined as the left or right lung, lung lobes, lung periphery, or lung segments, depending on the specific scanning purpose or predicted lesion. Traditional ROI segmentation methods can be used.

[0032] The block extraction unit 20 extracts blocks composed of pixels with defined characteristics from the region of interest segmented by the segmentation unit 10, and uses these blocks as target blocks. The block extraction unit 20 is capable of extracting blocks that meet preset conditions.

[0033] Specifically, the block extraction unit 20 includes an image pattern segmentation unit 21 and a connected region calculation unit 22. The image pattern segmentation unit 21 further segments typical patterns from the region of interest segmented by the segmentation unit 10. A typical pattern refers to an image pattern composed of pixels with the same characteristics or similar pixel values. In this embodiment, the image pattern segmentation unit 21 uses methods such as texture analysis to analyze the lung region, which is the region of interest, and extracts typical patterns considered to be lesion patterns. Furthermore, the type of typical pattern to extract can be determined based on the characteristics of a certain lesion image. For example, image patterns such as ground-glass opacities, or real variations, mesh-like opacities, etc., can be segmented. Multiple different image patterns can also be segmented. For example... Figure 4 (a) shows an example of an image pattern. Figure 4 In (a), the shaded area forms a ground-glass image pattern. To show the position of this image pattern relative to the lung image, the image pattern is superimposed on the region of interest of the lung image.

[0034] The connected region calculation unit 22 calculates connected regions as target blocks within the image patterns segmented by the image pattern segmentation unit 21. A connected region, a concept in image processing, generally refers to a region in an image composed of foreground pixels with the same or similar pixel values ​​and adjacent positions. Connected region extraction refers to identifying and labeling the various connected regions distributed within the image pattern. If multiple image patterns are segmented from the region of interest, the connected region calculation unit 22 can calculate connected regions using a pre-saved image label database.

[0035] like Figure 1 As shown, in this embodiment, the medical image processing device 100 pre-stores an image finding label database 70. The image finding label database 70 stores the type of the lesion image, the image finding as a type label, and the attributes corresponding to the image finding. Image finding, as a term used in image reading, refers to the features that can be observed on the image. The attributes of image finding are used to describe the characteristics of the observed area; these attributes are generally clinically interpretable features.

[0036] Figure 2 This is a schematic diagram illustrating the image view tag database in the medical image processing apparatus of the first embodiment. Figure 2 The text provides examples of imaging features of common pneumonias. As shown in the figure, in... Figure 2 The system stores disease names, image labels, label attributes, and attribute values ​​in a corresponding manner. The disease name section records examples of lesion types such as COVID-19 and community-acquired pneumonia. As a lesion type of COVID-19, one of the characteristics of images seen in the medical images of COVID-19 patients is "GGO (ground-glass opacity)." Regarding "GGO," in the medical images of COVID-19 patients, the GGO region has attributes such as: presence ("present"), location ("both lungs"), distribution ("predominantly peripulmonary distribution"), and shape ("circular"). Similarly, as a lesion type of community-acquired pneumonia, one of the characteristics of images seen in the medical images of community-acquired pneumonia patients is "consolidation." Regarding "consolidation," in the medical images of community-acquired pneumonia patients, the consolidated region has attributes such as: presence ("present"), location ("one or both lungs"), and distribution ("lobar distribution").

[0037] The connected region calculation unit 22 retrieves the image view corresponding to a certain lesion type from the image view label database 70 as the image view to be identified, and searches for an image pattern that matches the image view to be identified in the image pattern. If no image pattern matches the image view to be identified, it is determined that the existence of the image view to be identified in the corresponding medical image is "non-existent". After outputting the determination result to the classification unit 60, the next image view is used as the image view to be identified for comparison.

[0038] When an image pattern exists that matches the image to be identified, the connected region calculation unit 22 calculates the connected regions in the image pattern using methods such as region growing, obtaining one or more connected regions. For example, in Figure 4 In the image pattern shown in (a), three connected regions, namely connected region 1, connected region 2, and connected region 3, can be obtained.

[0039] Back Figure 1 As explained, the block extraction correction unit 30 corrects the blocks extracted by the block extraction unit 20 based on the influencing factors when the block extraction unit 20 extracts blocks, and outputs the corrected blocks.

[0040] Specifically, the block extraction correction unit 30 includes a dynamic influencing factor correction unit 31 for correcting artifacts caused by dynamic influencing factors and a static influencing factor correction unit 32 for correcting artifacts caused by static influencing factors. For example, in the lung region, static influencing factors mainly include gravity effects or poor respiratory control, while dynamic influencing factors mainly include heartbeats. This will be used as an example for explanation.

[0041] The dynamic influencing factor correction unit 31 has a pre-generated dynamic influencing factor correction model. The dynamic influencing factor correction model takes medical images, blocks and regions of interest as inputs and outputs blocks after removing artifacts at the junction of the lungs, mediastinum and diaphragm caused by heartbeats, etc. Figure 3 Figure (a) is a schematic diagram illustrating the dynamic influencing factor correction model in the first embodiment. As shown in the figure, the dynamic influencing factor correction unit 31 uses the blocks extracted by the block extraction unit 20 (connected regions are considered as blocks in this embodiment), the region of interest, and medical images, such as... Figure 4 The part corresponding to (a) is input into the dynamic influencing factor correction model, thereby obtaining Figure 4 The image shown in (b) contains, as Figure 4 As shown in (b), connected regions 1 and 3 remain unchanged, while the area of ​​connected region 2 is reduced due to the removal of the dynamic artifact portion.

[0042] The dynamic influencing factor correction model is as follows: Figure 3 The result shown in (a) is generated by training with the ground truth of motion artifact regions on medical images as training data, and can be loaded from the outside into the dynamic influencing factor correction unit 31 for use.

[0043] The static influencing factor correction unit 32 has a pre-generated static influencing factor correction model, which takes medical images and blocks as input and outputs blocks after removing artifacts caused by gravity effects or poor respiratory control. Figure 3 (b) is a schematic diagram illustrating the static influencing factor correction model in the first embodiment. As shown in the figure, the static influencing factor correction unit 32 corrects the blocks and medical images, such as those processed by the dynamic influencing factor correction unit 31, by... Figure 4 The part corresponding to (b) is input into the static influencing factor correction model, thereby obtaining Figure 4 The image shown in (c) contains, as Figure 4 As shown in (c), connected region 3 is deleted, leaving connected regions 1 and 2.

[0044] The static influencing factor correction model is as follows: Figure 3 The model shown in (b) is generated by training with ground truth values ​​of static artifacts in medical images as training data and can be externally loaded into the static influencing factor correction unit 32 for use. Alternatively, the extraction results of artificial blocks can be used as ground truth values ​​to train the static influencing factor correction model.

[0045] Furthermore, the present invention is not limited to the above configuration. It can also integrate the dynamic influencing factor correction unit 31 and the static influencing factor correction unit 32, and use a correction model that can simultaneously correct dynamic artifacts and static artifacts to perform a one-time correction on the block.

[0046] If any blocks still exist after the above corrections, it is considered that there are true images in the medical images that correspond to the lesion type, and the lesion images are classified based on these true images.

[0047] also, Figure 1The basic feature extraction unit 40 extracts the basic features of the corrected connected regions (target blocks). The basic features of an image refer to image features that can be accurately obtained from the image through measurement, including traditional morphological features, texture features, and dynamic features. Basic morphological features represent the shape and structure of a graphic, such as the outer boundary dimensions of a contour and the size of a region. Basic texture features represent the surface features of an image or region of interest, such as the coarseness and density of image textures. Basic dynamic features are dynamic features on the image; for example, when extracting basic dynamic features from multi-stage contrast images, there are dynamic features such as the enhancement ratio between the contrast image and the pre-contrast image.

[0048] The clinically interpretable feature generation unit 50 uses a conversion model that establishes a correlation between the basic features of the image and the clinically interpretable features, and generates the corrected clinically interpretable features of the block based on the basic features extracted by the basic feature extraction unit 40.

[0049] Clinically interpretable features refer to quantifiable characteristics that describe specific locations in clinical practice. These quantifiable features are easy to interpret and are therefore commonly used in image interpretation processes such as grading and scoring. As descriptive features, the specific content of clinically interpretable features can be determined and interpreted through industry guidelines or standards. For example, clinically interpretable features include clinically interpretable quantity information, clinically interpretable location information, clinically interpretable distribution information, clinically interpretable morphological information, clinically interpretable lesion severity information such as large, medium, and small, or clinically interpretable imaging signs such as mosaicism and perfusion.

[0050] The clinically interpretable feature generation unit 50 generates corresponding clinically interpretable features by using image basic features as input and outputting corresponding clinically interpretable features for each clinically interpretable feature.

[0051] Figure 5 This is a schematic diagram illustrating the generation process of clinically interpretable features in the first embodiment. For example... Figure 5As shown, the clinically interpretable feature generation unit 50 includes: a quantitative information extraction module 51, which uses a quantitative information conversion model that establishes an association between basic features and clinically interpretable quantitative information to generate the corrected clinically interpretable quantitative information of the block based on the basic features; a location information extraction module 52, which uses a location information conversion model that establishes an association between basic features and clinically interpretable location information to generate the corrected clinically interpretable location information of the block based on the basic features; and a distribution information extraction module 53, which uses a distribution information conversion model that establishes an association between basic features and clinically interpretable distribution information to generate the corrected clinically interpretable location information of the block based on the basic features. The system includes: a bed-interpretable distribution information module 54, a morphological information extraction module 55, a morphological information conversion model that establishes a correlation between basic features and clinically interpretable morphological information, and a modified clinically interpretable morphological information of the block based on the basic features; a lesion severity extraction module 55, a lesion severity conversion model that establishes a correlation between basic features and clinically interpretable lesion severity, and a modified clinically interpretable lesion severity of the block based on the basic features; and an image sign extraction module 56, an image sign conversion model that establishes a correlation between basic features and clinically interpretable image signs, and a modified clinically interpretable image sign of the block based on the basic features.

[0052] Each conversion model is a pre-generated model using previously reviewed image examples as training data and employing machine learning or deep learning. The algorithm used for the generated model can be any existing algorithm. Figure 5 The morphological information extraction model in the morphological information extraction module 55 is explained.

[0053] like Figure 5 As shown, the morphological information extraction module 55 contains a morphological information generation model y = f(c1, c2, ..., c) trained using the morphological ground truth values ​​of medical images as training data. n ), where y represents morphological information, c1, c2, ..., c n This represents the basic features obtained from the basic feature extraction unit 40, such as the volume, volume ratio, longest axis, and shortest axis of the blocks formed by connected regions. y = f(c1, c2, ..., c n For example, it can be expressed as equation (1):

[0054] y = Lung volume ratio ≤ T1 + volume ≤ T2 + majority voting algorithm (longest axis of connected region 1 / shortest axis of connected region 1 >= T3; ...; longest axis of connected region n / shortest axis of connected region n >= T3) Equation (1)

[0055] Where T1, T2, and T3 are preset thresholds. When the corrected connected region in a medical image satisfies equation (1), it indicates that there is a morphological feature in the medical image that is restricted by T1, T2, and T3.

[0056] The generation and operation of transformation models in other modules are similar to those in the morphological information extraction module 55, so examples are omitted.

[0057] Clinically interpretable feature generation unit 50, such as Figure 5 As shown, the image view attributes can be read from the image view label database 70 as clinically interpretable features to be generated. For example, in... Figure 2 In the case of the image-based label database 70 shown, clinically interpretable location information, clinically interpretable distribution information, and clinically interpretable shape information can be generated to determine whether it is COVID-19.

[0058] Furthermore, the present invention is not limited to the above configuration. The clinically interpretable feature generation unit 50 may also have some or one of the extraction modules 51 to 56, or it may have an extraction module for extracting other clinically interpretable features.

[0059] The clinically interpretable feature generation unit 50 can generate various clinically interpretable features and output them to the classification unit 60.

[0060] The classification unit 60 classifies medical images containing corrected connected blocks based on clinically interpretable features from the clinically interpretable feature generation unit 50, and obtains the classification result.

[0061] For example, in is Figure 2 In the case of the image view label database 70 shown, the classification unit 60 compares the clinically interpretable location information, clinically interpretable distribution information, clinically interpretable shape information, etc. generated by the clinically interpretable feature generation unit 50 with the attribute values ​​in the image view label database 70. For example, if each piece of information is consistent with the attribute values ​​of each attribute in the image view of COVID-19, the type of medical image in which the block composed of connected regions is located is determined to be an image of COVID-19.

[0062] Furthermore, some lesion types are characterized by the absence of a certain image. Therefore, if the block extraction unit 20 does not extract the image-defined (prescribed feature) block in a medical image, or if there are no remaining blocks after the correction process of the block extraction correction unit 30, the classification unit 60 will treat such image-defined images as image-defined images with an attribute value of "absence" of "absence" and use this attribute value in the classification process of the corresponding medical image.

[0063] The medical image processing method in the first embodiment will be described below from the perspective of the overall process of classification processing performed by the medical image processing device 100.

[0064] Figure 6 This is a flowchart illustrating the lesion image classification process in the medical image processing apparatus of the first embodiment. Furthermore, for the sake of simplicity, the classification process in this flowchart does not involve images where "existence" is absent, and the processing ends when it is determined that no more blocks exist.

[0065] After the classification process begins, the segmentation unit 10 first obtains the medical image to be classified (step S601) and performs segmentation processing on the medical image to segment out the lungs as the region of interest (step S602).

[0066] Next, in step S603, the image pattern segmentation unit 21 further segments typical image patterns related to lung lesions within the segmented region of interest. Then, in step S604, the connected region calculation unit 22 determines whether any lesion image is stored in the image sighting label database 70 among the segmented image patterns. If no lesion image is stored in the image sighting label database 70 among all image patterns (step S604: No), the process ends.

[0067] On the other hand, if a lesion image stored in the image image label database 70 is present in a certain image pattern (step S604: Yes), proceed to step S605, where the connected region calculation unit 22 calculates the connected region in the obtained image pattern as the target block.

[0068] Next, for the obtained target block, the dynamic influencing factor correction unit 31 uses the motion artifact correction model to remove the connected regions identified as motion artifact regions (step S606), and the static influencing factor correction unit 32 uses the static artifact correction model to remove the connected regions identified as static artifact regions (step S607).

[0069] After the correction process, it is determined whether there is still a connected region (step S608). If there is no connected region (step S608: no), it means that there is no target block in the medical image and the processing ends.

[0070] In the case of a connected region (step S608: Yes), the basic feature extraction unit 40 extracts the basic features of the image from the connected region for the corrected connected region (step S609).

[0071] Next, in step S610, the clinically interpretable feature generation unit 50 obtains the basic features of the image from the basic feature extraction unit 40, and reads the various attributes corresponding to the image views used in segmenting typical patterns from the image view label database 70. Using a transformation model that establishes a correlation between the basic features and the clinically interpretable features, the unit generates the attribute values ​​of the various attributes corresponding to the image views mentioned above based on the basic features of the image, which serve as the clinically interpretable features of the connected regions.

[0072] Finally, in step S611, the classification unit 60 classifies and outputs the disease types of the lesion images in the connected regions based on clinically interpretable features.

[0073] The above process can be modified appropriately as an example. For instance, the processing order of step S606 in dynamic artifact correction and step S607 in static artifact correction can also be changed, that is, static artifact correction can be performed first and then dynamic artifact correction.

[0074] Regarding the performance of the medical image processing device according to the first embodiment described above, testing using a test set revealed that, compared to the case without block correction, both sensitivity and specificity were significantly improved with block correction. Furthermore, by comparing specific embodiments of this embodiment with comparative embodiments of existing deep learning methods and clinical expert image interpretation methods, it was found that the sensitivity of embodiments of this embodiment is significantly superior to the two existing technologies described above.

[0075] Furthermore, in the first embodiment, the segmentation unit 10 corresponds to the "segmentation section", the block extraction unit 20 corresponds to the "block extraction section", the block extraction correction unit 30 corresponds to the "block extraction correction section", the basic feature extraction unit 40 corresponds to the "basic feature extraction section", the clinically interpretable feature generation unit 50 corresponds to the "clinically interpretable feature generation section", and the classification unit 60 corresponds to the "classification section".

[0076] According to the first embodiment, the influence of factors that can cause false findings, such as heart rate and gravity effects, is eliminated, thereby improving the accuracy of classification results. Furthermore, since pneumonia is classified based on clinically significant and interpretable characteristics, the interpretability is stronger, making it easier to gain the trust of doctors. Simultaneously, not only is the accuracy of lesion classification improved, but clinically significant imaging findings are also generated. Moreover, by storing an imaging findings label database, the same standards can be used before and after classification processing, thereby improving the accuracy and efficiency of classification results.

[0077] (A variation of the first embodiment)

[0078] In the first embodiment, the image view label database 70 is used to define lesion types and corresponding image views and attributes, thereby enabling block extraction and classification. Alternatively, the medical image processing apparatus 100 may also include an image view label database generation module, thereby generating and updating the image view label database. Specifically, for example, this image view label database generation module can collect existing image report sets, perform report extraction processing on each report in the report set using a medical dictionary and semantic rules, extracting image view labels, label types, pneumonia types, etc. from the reports. Then, it uses methods such as correlation and redundancy analysis to calculate the importance of each image view label, storing the image view labels with high importance and their related attributes corresponding to the lesion type of the report, thereby generating the image view label database.

[0079] Furthermore, in the first embodiment, the extracted blocks are corrected from the perspectives of static influencing factors and dynamic influencing factors by using the dynamic influencing factor correction unit 31 and the static influencing factor correction unit 32, respectively. However, the present invention is not limited to this, and only one type of influencing factor may be corrected.

[0080] Furthermore, the dynamic influencing factor correction unit 31 may also include a model generator for generating the dynamic influencing factor correction model to be used or to perform further learning and optimization on the dynamic influencing factor correction model. Similarly, the static influencing factor correction unit 32 may include a model generator for generating the static influencing factor correction model to be used or to perform further learning and optimization on the static influencing factor correction model.

[0081] Furthermore, in the first embodiment, typical lesion images of the region of interest are extracted using image views and attributes from the image view tag database 70, and connected regions are calculated from the lesion images as target blocks. This eliminates the problem of some lesion boundaries being blurred or overlapping. However, typical images can also be directly used as target blocks for block correction, which can also improve the accuracy of medical image analysis and the interpretability of the analysis results.

[0082] (Second Implementation)

[0083] The following reference Figures 7 to 10The second embodiment of the present invention will be described. The medical image processing apparatus of the second embodiment is based on the first embodiment. The main difference compared to the first embodiment is that, in the second embodiment, the medical image processing apparatus 200 does not have the image-visible label database 70 but instead has a display unit 80, a nucleic acid detection result prediction unit 90, and an optimization unit 110. Furthermore, the block extraction unit 20, the block extraction correction unit 30, and the classification unit 60 are replaced by a block extraction unit 20a, a block extraction correction unit 30a, and a classification unit 60a, respectively. The following description mainly focuses on the differences, and identical or similar configurations are labeled with the same reference numerals while appropriately omitting repetitive descriptions.

[0084] Figure 7 This is a block diagram illustrating an example of the functional configuration of the medical image processing apparatus according to the second embodiment. For example... Figure 7 As shown, the medical image processing apparatus 200 of the second embodiment includes a segmentation unit 10, a block extraction unit 20a, a block extraction correction unit 30a, a basic feature extraction unit 40, a clinically interpretable feature generation unit 50, a display unit 80, a classification unit 60a, a nucleic acid detection result prediction unit 90, and an optimization unit 110.

[0085] The segmentation unit 10 acquires a medical image of the subject and performs prescribed segmentation processing on the medical image to determine the region of interest in the medical image. Here, a three-dimensional chest scan image is used as an example for illustration.

[0086] The block extraction unit 20a extracts connected regions composed of pixels with defined features from the region of interest segmented by the segmentation unit 10, and uses these as target blocks. The block extraction unit 20a can extract connected regions composed of pixels satisfying the defined features from the region of interest using pixel extraction methods such as region growing, according to pre-set pixel features. The pixel features used to determine the connected regions can be set as one or more based on commonly used lesion image views. Alternatively, each image view can be extracted separately according to guidelines in this field. Furthermore, the image views corresponding to the suspected lesion type of the examinee can be set as the defined features.

[0087] The block extraction correction unit 30a corrects the connected regions extracted by the block extraction unit 20a based on the influencing factors when the block extraction unit 20a extracts blocks, and outputs the corrected connected regions as the corrected target blocks.

[0088] Specifically, the block extraction and correction unit 30a includes a dynamic influencing factor correction unit 31 for correcting artifacts caused by dynamic influencing factors, a static influencing factor correction unit 32 for correcting artifacts caused by static influencing factors, and a target block judgment unit 33 for judging whether the corrected connected regions meet the specified target block conditions and outputting only the connected regions that meet the specified target block conditions as the corrected target blocks. For example, in the lung region, static influencing factors mainly include gravity effects or poor respiratory control, while dynamic influencing factors mainly include heartbeat, etc. This will be used as an example for explanation.

[0089] The dynamic influencing factor correction unit 31 uses a dynamic influencing factor correction model generated by using the true value of the motion artifact region as training data to correct the extraction results of the block extraction unit 20a, thereby eliminating the dynamic artifacts in the extraction results.

[0090] The static influencing factor correction unit 32 uses the static influencing factor correction model generated by using the manual block extraction results as training data to correct the extraction results of the block extraction unit 20a, thereby eliminating static artifacts in the extraction results.

[0091] Furthermore, the target block determination unit 33 determines whether the corrected connected region meets the specified target block conditions, and only outputs the connected regions that meet the specified target block conditions as the corrected target blocks. The specified target block conditions can be preset according to the lesion being targeted or the location being photographed. Moreover, the determination by the target block determination unit 33 can be performed before or after the correction by the dynamic influencing factor correction unit 31 and the static influencing factor correction unit 32. The determination by the target block determination unit 33 can also be performed multiple times during the block correction process.

[0092] For example, the target block judgment unit 33 can judge the size of the connected region after the correction by the dynamic influencing factor correction unit 31 and before the correction by the static influencing factor correction unit 32, and remove connected regions with a size smaller than a predetermined threshold. This is because connected regions with excessively small sizes are often considered noise.

[0093] In addition, the basic feature extraction unit 40 extracts the basic features of the corrected connected regions.

[0094] The clinically interpretable feature generation unit 50 uses a conversion model that establishes a correlation between the basic features of the image and the clinically interpretable features, and generates the corrected clinically interpretable features of the block based on the basic features extracted by the basic feature extraction unit 40.

[0095] Clinically interpretable feature generation unit 50 can, as Figure 5As shown, it includes multiple extraction modules such as quantity information extraction module 51, location information extraction module 52, distribution information extraction module 53, morphological information extraction module 54, lesion severity extraction module 55, and imaging sign extraction module 56, which extract the corresponding clinically interpretable features from the corrected block. Alternatively, it can extract only the clinically interpretable features corresponding to the attributes of the specified features used during block extraction.

[0096] The clinically interpretable feature generation unit 50 can generate various clinically interpretable features of the block and output them to the classification unit 60a.

[0097] The classification unit 60a classifies medical images containing modified connected blocks based on clinically interpretable features from the clinically interpretable feature generation unit 50, and obtains the classification result.

[0098] Furthermore, the nucleic acid test result prediction unit 90 uses a nucleic acid test result prediction model generated by using existing nucleic acid test results as training data to generate a nucleic acid test prediction result for the lung image. Alternatively, it can predict nucleic acid test results based on medical images and lesion area masks. The optimization unit 110 then optimizes the classification results based on the nucleic acid test prediction results predicted by the nucleic acid test result prediction unit 90.

[0099] For example, in the detection of COVID-19, if the classification result of the medical image by the classification unit 60a is not COVID-19, but the result predicted by the nucleic acid test result prediction unit 90 is positive, the optimization unit 110 takes the nucleic acid test prediction result as the standard, modifies the classification result to COVID-19, and outputs it.

[0100] Furthermore, the medical image processing apparatus 200 also includes a display unit 80 that displays an analysis report generated by, for example, the classification unit 60a, which records the classification results, in order to provide the user with classification information for the medical image. This analysis report may include the final classification result and its corresponding image appearance and attributes.

[0101] For example, Figure 8 An example of an analysis report is shown. For example... Figure 8 As shown, the report records the classification result as "non-pneumonia," and lists the various attributes of the ground-glass opacities seen in the images in sequence: presence, location, morphology, signs, and number. Furthermore, other attributes of images such as nodules can be further described below.

[0102] Furthermore, users can modify the analysis report displayed on the display unit 80. For example, Figure 8As shown, the display unit 80 indicates to the user that the editable entries, such as the presence of ground-glass opacity or nodules, can be marked or displayed using a different font to indicate that the section can be modified. The display unit 80 accepts the user's modifications to these editable sections and updates the analysis report accordingly. Furthermore, the display unit 80 can also distinguish between the user-modified content and the unchanged content, for example, by using a different font.

[0103] Furthermore, the display unit 80 is not limited to this; it can also overlay the target block extracted by the block extraction unit 20a with the medical image and display a classification report showing the classification results alongside the overlaid image.

[0104] Figure 9 Another example of an analysis report is shown. In Figure 9 The right side shows the analysis report, which records the classification result as "typical COVID-19". The report lists the various attributes of the ground-glass opacities seen in the images: presence, location, morphology, signs, and quantity. Because the presence is set to "present", the attribute values ​​for other attributes can be obtained, such as the location attribute value being "peripheral to the lungs". In this analysis report, the modifiable parts are shown enclosed in black boxes.

[0105] On the other hand, Figure 9 The left side simultaneously displays the medical images categorized by the analysis report on the right, as well as the blocks generated during the classification process. The target blocks are overlaid with the medical images. Figure 9 In the example, four views of the same medical image are shown. When the user clicks on a specific image view in the analysis report on the right, the display unit 80 overlays all target blocks extracted by the block extraction unit 20a for that image view onto the medical image on the left. Furthermore, the corrected target blocks are displayed separately from the deleted target blocks. Figure 9 When the image of ground-glass opacity is selected, the shaded areas can be clearly seen on the medical image on the left. For example, area A, as the corrected area, is displayed darker, while area B, as the area deleted in the correction process, is displayed lighter.

[0106] Through such Figure 9 This display can more comprehensively reflect the suspicious blocks and block types that appear during the classification process, thereby helping with subsequent manual re-examination and treatment, and improving the convenience and accuracy of image reading.

[0107] The medical image processing method in the second embodiment will be described below from the perspective of the overall process of classification processing performed by the medical image processing device 200.

[0108] Figure 10 This is a flowchart illustrating the lesion image classification process in the medical image processing apparatus of the second embodiment. Furthermore, for the sake of simplicity, the classification process in this flowchart does not involve images where "existence" is non-existent, and the process ends when it is determined that no connected region exists.

[0109] After the classification process begins, the segmentation unit 10 first obtains the medical image to be classified (step S1001) and performs segmentation processing on the medical image to segment out the lungs as the region of interest (step S1002).

[0110] Next, in step S1003, the block extraction unit 20a determines whether there is a connected region composed of pixels with specified characteristics in the region of interest segmented by the segmentation unit 10. If there is no connected region composed of pixels with specified characteristics (step S1003: No), the process ends.

[0111] On the other hand, if there is a connected region composed of pixels with specified characteristics in the region of interest (step S1003: Yes), proceed to step S1004, where the block extraction unit 20a extracts the connected region from the region of interest as the target block.

[0112] Next, for the obtained target block, the dynamic influencing factor correction unit 31 inputs all connected regions of the medical image into the motion artifact correction model to remove connected regions identified as motion artifact regions (step S1005). The static influencing factor correction unit 32 inputs the remaining connected regions processed by the dynamic influencing factor correction unit 31 into the static artifact correction model to remove connected regions identified as static artifact regions (step S1006). Afterwards, the target block determination unit 33 removes connected regions with a size smaller than a predetermined threshold from the remaining connected regions (step S1007).

[0113] After the above correction process, it is determined whether there is still a connected region (step S1008). If there is no connected region (step S1008: no), it means that there is no target block in the medical image and the processing ends.

[0114] In the case of a connected region (step S1008: Yes), the basic feature extraction unit 40 extracts the basic features of the image from the connected region for the corrected connected region (step S1009).

[0115] Next, in step S1010, the clinically interpretable feature generation unit 50 obtains the basic features of the image from the basic feature extraction unit 40, and uses a transformation model that establishes a correlation between the basic features and the clinically interpretable features to generate clinically interpretable features of the connected regions based on the basic features of the image.

[0116] Next, in step S1011, the classification unit 60a classifies the disease types of the lesion images in the connected regions based on clinically interpretable characteristics.

[0117] Next, in step S1012, the nucleic acid test result prediction unit 90 generates a nucleic acid test prediction result for the lung image using the nucleic acid test result prediction model. Then, the optimization unit 110 optimizes the classification result based on the nucleic acid test prediction result predicted by the nucleic acid test result prediction unit 90 (step S1013). The classification unit 60a generates an analysis report based on the optimized classification result and displays it on an output terminal such as a monitor via the display unit 80.

[0118] The above process can be modified appropriately as an example. For instance, the processing order of steps S1005, S1006, and S1007 can be changed arbitrarily.

[0119] Furthermore, in the second embodiment, the segmentation unit 10 corresponds to the "segmentation section", the block extraction unit 20a corresponds to the "block extraction section", the block extraction correction unit 30a corresponds to the "block extraction correction section", the basic feature extraction unit 40 corresponds to the "basic feature extraction section", the clinically interpretable feature generation unit 50 corresponds to the "clinically interpretable feature generation section", the classification unit 60a corresponds to the "classification section", the nucleic acid detection result prediction unit 90 corresponds to the "nucleic acid detection result prediction section", the optimization unit 110 corresponds to the "optimization section", and the display unit 80 corresponds to the "display section".

[0120] According to the second embodiment, the influence of factors such as heart rate and gravity effects that can cause false findings is eliminated, thereby improving the accuracy of classification results. Furthermore, since pneumonia is classified based on clinically significant and interpretable characteristics, the interpretability is stronger, making it easier to gain the trust of doctors. Simultaneously, it not only improves the accuracy of lesion classification results but also generates clinically significant imaging findings.

[0121] Furthermore, since some patients with pneumonia such as COVID-19 may be misdiagnosed due to nonspecific imaging findings, introducing a predictive mechanism based on nucleic acid test results can optimize the classification of pneumonia based on imaging findings. This can correct misjudgments caused by imaging findings that are atypical for COVID-19, thereby improving classification performance.

[0122] (A variation of the second embodiment)

[0123] In the second embodiment, a connected region consisting of pixels with defined characteristics is extracted from the region of interest as the target block. However, a typical pattern consisting of pixels with defined characteristics in the region of interest can also be extracted as the target block using methods such as texture analysis. Modifying typical images into blocks can also improve the accuracy of medical image analysis and the interpretability of the analysis results.

[0124] Furthermore, in the second embodiment, a display unit for displaying reports is provided, but this display unit may also be omitted and the results or analysis reports may be output directly.

[0125] The constituent elements of each device in the above-described embodiments are functional concepts and do not necessarily need to be physically configured as shown in the illustrations. That is, the specific form of the distributed / integrated arrangement of each device is not limited to the content shown in the illustrations, and all or part of them can be configured in any unit, functionally or physically, according to various loads and usage conditions. Furthermore, all or any part of the processing functions performed in each device can be implemented by a CPU and the program parsed and executed by the CPU, or it can be implemented as hardware based on wiring logic.

[0126] Furthermore, the medical image processing apparatus and method described in the above embodiments can be implemented by executing a pre-prepared program using a computer such as a personal computer or workstation. This program can be distributed via a network such as the Internet. Additionally, the program can be recorded on a computer-readable non-transitory recording medium such as a hard disk, floppy disk (FD), CD-ROM, MO, or DVD, and then read from the recording medium by a computer for execution.

[0127] Several embodiments of the present invention have been described, but these embodiments are provided as examples and are not intended to limit the scope of the invention. These embodiments can be implemented in a wide variety of other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their variations are included in the scope and spirit of the invention, as well as in the invention as described in the claims and its equivalents.

Claims

1. A medical image processing device, wherein, have: The segmentation unit acquires a medical image, performs prescribed segmentation processing on the medical image, and determines the region of interest in the medical image. The block extraction unit extracts blocks from the region of interest that are composed of pixels with defined characteristics, and uses them as target blocks. The block extraction correction unit corrects the extraction results of the block extraction unit based on the static and dynamic influencing factors when the block extraction unit extracts blocks, and outputs the corrected block as the corrected target block. The basic feature extraction unit extracts the basic features of the corrected target block; The clinically interpretable feature generation unit utilizes a transformation model that establishes a correlation between the basic features and the clinically interpretable features, and generates the clinically interpretable features of the corrected target block based on the basic features extracted by the basic feature extraction unit. as well as The classification unit, based on the clinically interpretable features generated by the clinically interpretable feature generation unit, classifies the medical image containing the corrected target block to obtain a classification result. The block extraction correction unit has: The static influencing factor correction unit corrects the extraction results of the block extraction unit using a first correction model generated by using the results of manual block extraction as training data; and The dynamic influencing factor correction unit uses a second correction model generated by using the ground truth of the motion artifact region as training data to correct the extraction results of the block extraction unit.

2. The medical image processing device as described in claim 1, wherein, The block extraction unit has: A typical pattern segmentation unit segments a typical pattern composed of pixels with defined characteristics within the region of interest; and The connected region calculation unit calculates connected regions as target blocks in each typical pattern cut out from the typical pattern segmentation part.

3. The medical image processing device as described in claim 1, wherein, The block extraction unit extracts a connected region composed of pixels with defined characteristics from the region of interest, which is then used as the target block.

4. The medical image processing apparatus as described in claim 2 or 3, wherein, It also has an image view tag database, which stores the type of the target block, the image view as a type tag, and the corresponding attributes of the image view. The block extraction unit uses the image views in the image view label database as the defined features to extract the connected regions. When a target block exists whose clinically interpretable features match the attribute in the image view label database, the classification department uses the type corresponding to the attribute in the image view label database as the type of the medical image containing the target block.

5. The medical image processing apparatus as described in claim 1, wherein, The block extraction and correction unit has a target block judgment unit, which judges whether the corrected connected region meets the specified target block conditions, and outputs only the connected regions that meet the specified target block conditions as the corrected target blocks.

6. The medical image processing apparatus as described in claim 1, wherein, The medical image is a lung image. The static influencing factors mentioned are static artifacts caused by gravity effects or poor respiratory control. The dynamic influencing factor is the lung periphery volume effect artifact caused by breathing or cardiac movement.

7. The medical image processing apparatus as claimed in claim 1, wherein, The clinically explainable features include clinically explainable quantity information, clinically explainable location information, clinically explainable distribution information, clinically explainable morphological information, clinically explainable lesion degree, or clinically explainable imaging signs.

8. The medical image processing apparatus as claimed in claim 1, wherein, The medical image is a lung image.

9. The medical image processing apparatus as described in claim 8, wherein, It also includes: a nucleic acid test result prediction unit, which uses a nucleic acid test result prediction model generated by using nucleic acid test results as training data to generate nucleic acid test prediction results for the lung image; as well as The optimization department optimizes the classification results of the classification department based on the nucleic acid detection prediction results.

10. The medical image processing apparatus as claimed in claim 1, wherein, It also has a display unit that displays the classification results of the classification unit and is capable of accepting modifications to the displayed classification results.

11. The medical image processing apparatus as claimed in claim 1, wherein, It also has a display unit that overlays the target block extracted by the block extraction unit with the medical image, and displays a classification report showing the classification results side by side with the overlaid image.

12. The medical image processing apparatus as claimed in claim 11, wherein, In the display unit, on the overlaid image, the corrected target block extracted by the block extraction unit is displayed in a way that distinguishes it from other target blocks.

13. A medical image processing method, wherein, include: The segmentation step involves acquiring a medical image, performing a prescribed segmentation process on the medical image, and determining the region of interest in the medical image. The block extraction step involves extracting blocks composed of pixels with defined characteristics from the region of interest, which are then used as target blocks. The block extraction correction step corrects the extraction results of the block extraction step based on the static and dynamic influencing factors during the block extraction process, and outputs the corrected block as the corrected target block. The basic feature extraction step extracts the basic features of the corrected target block; The clinically interpretable feature generation step utilizes a transformation model that establishes a correlation between the basic features and the clinically interpretable features, and generates the corrected clinically interpretable features of the target block based on the basic features extracted in the basic feature extraction step. as well as The classification step involves classifying the medical image containing the corrected target block based on the clinically interpretable features generated in the clinically interpretable feature generation step, thereby obtaining the classification result. The block extraction and correction steps include: A step of correcting the extraction results of the block extraction step using a first correction model generated by using the results of manual block extraction as training data; as well as The step of correcting the extraction results of the block extraction step by using a second correction model generated by using the ground truth of the motion artifact region as training data.

Images