Image diagnosis support system, image diagnosis support method, and recording medium

Abstract

The image diagnosis support system (100) comprises: an image input unit (110) which receives input of an image; a first processing unit (112) which detects a region of interest including an object of interest from the image, classifies the detected region of interest and outputs a first classification result; and a second processing unit (114) which performs semantic segmentation on the region of interest, classifies the region of interest based on the result of the semantic segmentation and outputs a second classification result. For the region of interest, when a predetermined condition is satisfied, the first classification result is adopted as the classification of the region of interest, and when the predetermined condition is not satisfied, the second classification result is adopted as the classification of the region of interest.

Description

Technical Field

[0001] The present invention relates to an image diagnosis support system and an image diagnosis support device. Background Art

[0002] In recent years, attempts have been made to apply supervised learning, represented by deep learning, to the diagnosis of endoscopic images. Conventionally, a technique has been proposed for detecting lesions of interest from endoscopic images by combining existing methods with supervised learning (Patent Document 1).

[0003] Prior art literature

[0004] Patent Literature

[0005] Patent Document 1: WO2017 / 042812 Summary of the invention

[0006] Problem that the invention aims to solve

[0007] Patent Document 1 describes classifying (identifying) a region of interest including a detected object of interest by supervised learning, but does not disclose problems and research for classifying the region of interest with high accuracy. The inventors searched for a technique that can classify the region of interest with higher accuracy and came up with the present invention.

[0008] The present invention has been made in view of such circumstances, and an object of the present invention is to provide a technique capable of classifying a region of interest with higher accuracy.

[0009] Means used to solve problems

[0010] In order to solve the above-mentioned problem, an image diagnosis support system of a certain scheme of the present invention comprises: an input unit, which accepts input of an image; a first processing unit, which detects a region of interest including an object of interest from the image, classifies the detected region of interest and outputs a first classification result; and a second processing unit, which performs semantic segmentation on the region of interest, classifies the region of interest based on the result of performing the semantic segmentation and outputs a second classification result, and for the region of interest, when a specified condition is met, the first classification result is adopted as the classification of the region of interest, and when the specified condition is not met, the second classification result is adopted as the classification of the region of interest.

[0011] Another embodiment of the present invention is an image diagnosis support device. The device comprises: an input unit that receives input of an image; a first processing unit that detects a region of interest including an object of interest from the image, classifies the detected region of interest and outputs a first classification result; and a second processing unit that performs semantic segmentation on the region of interest, classifies the region of interest based on the result of performing the semantic segmentation and outputs a second classification result, wherein for the region of interest, if a predetermined condition is satisfied, the first classification result is adopted as the classification of the region of interest, and if the predetermined condition is not satisfied, the second classification result is adopted as the classification of the region of interest.

[0012] Another embodiment of the present invention is an image diagnosis support system. The image diagnosis support system comprises: an input unit that receives input of an image; a first processing unit that detects a region of interest including an object of interest from an image obtained by reducing the resolution of the image; and a second processing unit that performs semantic segmentation on a region corresponding to the region of interest in an image having a higher resolution than the image after the resolution is reduced, and classifies the region of interest based on the result of performing the semantic segmentation.

[0013] Optional combinations of the above-described constituting elements and embodiments in which expressions of the present invention are converted between methods, apparatuses, systems, recording media, computer programs, and the like may also be practiced as additional modes of the present invention.

[0014] Effects of the Invention

[0015] According to the present invention, it is possible to provide a technique capable of classifying a region of interest with higher accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 This is a block diagram showing the function and configuration of the image diagnosis support system according to the first embodiment.

[0017] Figure 2 Is used to illustrate the Figure 1 A diagram of the processing performed by a decision unit.

[0018] Figure 3 This is a flowchart showing an example of a series of processes in the image diagnosis support system according to the first embodiment.

[0019] Figure 4 This is a flowchart showing an example of a series of processes in the image diagnosis support system according to the second embodiment.

[0020] Figure 5 This is a block diagram showing the function and structure of the image diagnosis support system according to the fourth embodiment. DETAILED DESCRIPTION

[0021] Hereinafter, the present invention will be described based on preferred embodiments with reference to the drawings.

[0022] (First embodiment)

[0023] Figure 1 This is a block diagram showing the functions and structure of the image diagnosis support system 100 according to the first embodiment. The modules shown here can be implemented in hardware by components and mechanical devices represented by the CPU (central processing unit) of a computer, and in software by computer programs, etc., but here, the functional blocks implemented by their cooperation are described. Therefore, those skilled in the art should understand that these functional blocks can be implemented in various forms by combining hardware and software. Figure 4 The same is true.

[0024] The image diagnosis support system 100 supports diagnosis of lesions using endoscopic images. In addition, endoscopic images are captured by, for example, inserting a conventional endoscope, a capsule endoscope, or an NBI (Narrow Band Imaging) endoscope into the body of the body.

[0025] The image diagnosis support system 100 includes an image input unit 110 , a first processing unit 112 , a second processing unit 114 , a determination unit 116 , and an output unit 118 .

[0026] The image input unit 110 receives an input of an endoscopic image from a user terminal or other device. The first processing unit 112 performs a detection process of detecting a lesion candidate region (region of interest) for the endoscopic image received by the image input unit 110. The lesion candidate region is a rectangular region including a lesion (object of interest). Then, the first processing unit 112 performs a first classification process of classifying (identifying) the malignancy of the detected lesion candidate region (i.e., lesion), and outputs the classification result, i.e., the first classification result.

[0027] Specifically, the first processing unit 112 inputs an endoscopic image to a learning model for detection as a detection process, and the learning model is a learning model that has been learned in a manner of detecting a lesion candidate region including a lesion from an endoscopic image. In addition, the learning model for detection is preferably constructed as a neural network, and more preferably as a convolutional neural network such as a Region Proposal Network (RPN). That is, as a detection process, the first processing unit 112 preferably performs a process based on a neural network, and more preferably performs a process based on a convolutional neural network. Depending on the endoscopic image, there are cases where no lesion candidate region is detected in the detection process, and there are also cases where one or more lesion candidate regions are detected.

[0028] In addition, as the first classification processing, the first processing unit 112 inputs information related to the detected lesion candidate area to the learning model for the first classification that has been learned in a manner to classify the lesion candidate area. Here, the learning model for the first classification is a model obtained by learning in a manner to classify the lesion candidate area as any one of benign, mildly malignant, and malignant. In addition, the learning model for the first classification is preferably constructed as a neural network, and more preferably constructed as a convolutional neural network such as Fast R-CNN (FRCNN). That is, as the first classification processing, the first processing unit 112 preferably performs processing based on a neural network, and more preferably performs processing based on a convolutional neural network.

[0029] The second processing unit 114 performs semantic segmentation for classifying the malignancy of each pixel of the lesion candidate area detected by the detection processing, that is, classifying the lesion candidate area in units of pixels, and then based on the execution result of the semantic segmentation, that is, the classification result of each pixel, performs the second classification processing for classifying the malignancy of the lesion candidate area, and outputs the classification result, that is, the second classification result.

[0030] Specifically, the second processing unit 114 inputs information related to the detected lesion candidate area into the learned semantic segmentation learning model. Here, the semantic segmentation learning model is a model obtained by learning in a manner that classifies each pixel in the lesion candidate area as benign, mildly malignant, malignant, or normal. In addition, the semantic segmentation learning model is preferably constructed as a neural network, and more preferably as a convolutional neural network such as DeepLab. That is, as the second classification processing, the second processing unit 114 preferably performs processing based on a neural network, and more preferably performs processing based on a convolutional neural network.

[0031] In addition, in the second classification processing, the second processing unit 114 sets the classification with the largest proportion of pixels classified as lesions (i.e., pixels classified as benign, mildly malignant, or malignant) as the classification result of the lesion candidate area (i.e., lesion), i.e., the second classification result.

[0032] The decision unit 116 decides which classification result to adopt as the classification of the lesion candidate region, the first classification result or the second classification result. Here, in the first classification process, the lesion candidate region is classified as one category as a whole. Therefore, when the exclusive area of ​​the lesion in the lesion candidate region is small, specifically, when the lesion is elongated along the inclined direction, the classification accuracy tends to be low. In response to this, the decision unit 116 decides to adopt the first classification result of the first classification process as the classification of the lesion candidate region when the exclusive area of ​​the lesion in the lesion candidate region is large, and decides to adopt the second classification result of the second classification process as the classification of the lesion candidate region when the exclusive area of ​​the lesion in the lesion candidate region is small.

[0033] Figure 2 (a) and (b) are diagrams for explaining the processing performed by the determination unit 116. Figure 2 (a) shows the execution result of semantic segmentation. Figure 2 (b) shows the known method for Figure 2 (a) is an ellipse obtained by elliptical approximation of the lesion area. Figure 2 (a) and (b) will more specifically describe the processing performed by the decision unit 116. First, the decision unit 116 performs an elliptical approximation on the lesion region A1 in the result of the semantic segmentation, that is, the lesion region A1 which is a set of pixels classified as lesions (i.e., benign, mildly malignant, or malignant) by the semantic segmentation. Next, the decision unit 116 determines the major diameter D of the lesion region A2 obtained by the elliptical approximation. L 、Short diameter D M Then, when the following conditions are met, the decision unit 116 estimates that the lesion is not so long and thin or, even if it is long and thin, it is not long and thin in the oblique direction, and decides to adopt the first classification result as the classification of the lesion candidate area. On the other hand, when the following conditions are not met, the decision unit 116 estimates that the lesion is long and thin to a certain extent in the oblique direction, and decides to adopt the second classification result as the classification result of the lesion candidate area.

[0034] (condition)

[0035] Short diameter D M / Long diameter D L ≥Threshold Th1 (e.g. Th1=0.3)

[0036] Or the acute angle α between the longitudinal direction of the image and the long axis Ax is within a specified angle range (e.g. 0° to 30°)

[0037] The output unit 118 outputs the classification determined by the determination unit 116 to be adopted as the classification of the lesion candidate region, for example, to a display.

[0038] The above is the configuration of the image diagnosis support system 100. Next, the operation of the image diagnosis support system 100 will be described.

[0039] Figure 31 is a flowchart showing an example of a series of processes in the image diagnosis support system 100. The image input unit 110 receives the input of an endoscopic image (S110). The first processing unit 112 performs a detection process for detecting a candidate lesion region on the endoscopic image (S112). When a candidate lesion region is detected, that is, when a candidate lesion region exists in the endoscopic image (Yes in S114), the first processing unit 112 performs a first classification process on the detected candidate lesion region and outputs a first classification result (S116). The second processing unit 114 performs semantic segmentation on the detected candidate lesion region (S118), and performs a second classification process based on the result to output a second classification result (S120). The decision unit 116 decides which of the first classification result and the second classification result to adopt as the classification of the candidate lesion region based on the shape of the lesion region shown by the execution result of the semantic segmentation (S122). The output unit 118 outputs the classification decided to be adopted (S124). When the lesion candidate region is not detected (No in S114 ), S116 to S124 are skipped and the processing ends.

[0040] In addition, if a lesion candidate area is detected in the detection process of S112, the first classification process of S116 may be skipped and the execution of semantic segmentation of S118 may be entered. Then, after executing semantic segmentation, the second classification process of S120 may be skipped and the process of deciding the classification result to be adopted may be entered in S122. Then, if it is decided to adopt the first classification result, the first classification process of S116 may be executed, and if it is decided to adopt the second classification result, the second classification process of S120 may be executed.

[0041] The above operation assumes the case where the endoscopic image is a static image, but in the case of a dynamic image, the classification result may be continuously outputted as the endoscopic image is read, and the processing may be terminated at the last point in time of the dynamic image, for example.

[0042] According to the image diagnosis support system 100 of the first embodiment described above, when the lesion in the lesion candidate region is obliquely elongated, the second classification result is used instead of the first classification result which tends to have low classification accuracy. This can improve classification accuracy.

[0043] (Second embodiment)

[0044] In the first embodiment, a case is described in which the first classification result and the second classification result are determined as the classification of the lesion candidate region based on the shape of the lesion indicated by the execution result of the semantic segmentation of the second processing unit. In the second embodiment, a case is described in which the first classification result and the second classification result are determined as the classification of the lesion candidate region based on the shape of the lesion candidate region. The description will focus on the differences from the first embodiment.

[0045] Here, when the lesion candidate region detected by the detection processing of the first processing unit 112 is an elongated shape, it is estimated that the shape of the lesion included in the lesion candidate region is an elongated shape. On the other hand, in most cases, the lesion appears to be elongated when viewed from the side, and in the classification performed by the first processing unit 112, the classification accuracy of the lesion photographed from the side tends to be low. In response to this, the decision unit 116 decides to use the first classification result of the first classification processing as the classification of the lesion candidate region when the lesion candidate region is a square or a not so elongated rectangle, and decides to use the second classification result of the second classification processing as the classification of the lesion candidate region when the lesion candidate region is a rectangle that is elongated to a certain extent.

[0046] Specifically, the decision unit 116 decides to adopt the first classification result as the classification result of the lesion candidate region when the following conditions are met, and decides to adopt the second classification result as the classification result of the lesion candidate region when the following conditions are not met.

[0047] (condition)

[0048] The length of the short side of the lesion candidate region / the length of the long side of the lesion candidate region ≥ threshold Th2 (e.g. Th2 = 0.2)

[0049] Next, the operation of the image diagnosis support system will be described.

[0050] Figure 4 FIG. 1 is a flowchart showing an example of a series of processes in the image diagnosis support system according to the second embodiment. Figure 3The description will focus on the differences between the first and second classification results. When the first classification result is output, the decision unit 116 decides which of the first and second classification results to use as the classification of the lesion candidate area based on the shape of the lesion candidate area (S118). In the case where it is decided to use the first classification result (Yes in S118), the execution of the semantic segmentation of S120 and the second classification processing of S122 are skipped, and the first classification result decided to be used is output (S124). In the case where it is decided to use the second classification result (No in S118), the semantic segmentation and the second classification processing (S120, S122) are performed, and the second classification result is output as the classification result (S124).

[0051] The process of S118 by the decision unit 116 may be executed immediately after the process of S114. In this case, the first classification process may be executed only when it is determined to adopt the first classification result, and the second classification process may be executed only when it is determined to adopt the second classification result.

[0052] In addition, when the lesion candidate region is a square or a rectangle close to a square, there may be a case where the lesion is elongated in the oblique direction. Therefore, the technology of the first embodiment can be further combined to further determine whether to use the first classification result or the second classification result when the lesion candidate region is a square or a rectangle close to a square.

[0053] According to the image diagnosis support system of the second embodiment described above, when the lesion candidate region is a rectangular shape that is elongated to a certain extent, the second classification result is used instead of the first classification result which tends to have low classification accuracy in this case. This can improve classification accuracy.

[0054] (Third embodiment)

[0055] The inventors have confirmed that when the output of the SoftMax function is implemented as the likelihood during learning and the output of the SoftMax function is not implemented as the likelihood during recognition, there is a correlation between the likelihood of classification and the accuracy of classification. In this regard, in the third embodiment, the case of determining which classification result of the first classification result and the second classification result to use as the classification of the lesion candidate area based on the likelihood of the first classification result and the likelihood of the second classification result is described. The description is centered on the differences from the first embodiment.

[0056] The first processing unit 112 performs a detection process and a first classification process. The decision unit 116 determines whether the likelihood of the first classification result of the first classification process is greater than a predetermined threshold value Th3 (e.g., Th3=0.6). When the likelihood of the first classification result is greater than the threshold value Th3, the decision unit 116 decides to adopt the first classification result as the classification of the lesion candidate region. When the likelihood of the first classification result is less than the threshold value Th3, the decision unit 116 decides not to adopt the first classification result as the classification of the lesion candidate region. In this case, the second processing unit 114 performs semantic segmentation and a second classification process. The decision unit 116 determines whether the likelihood of the second classification result of the second classification process is greater than a predetermined threshold value Th4 (e.g., Th4=0.6). When the likelihood of the second classification result is greater than the threshold value Th4, the decision unit 116 decides to adopt the second classification result as the classification of the lesion candidate region. When the likelihood of the second classification result is less than the threshold value Th4, the decision unit 116 decides not to adopt the second classification result as the classification of the lesion candidate region. In this case, the decision unit 116 determines that the lesion candidate region cannot be classified. The output unit 118 outputs the first classification result, the second classification result, or cannot be classified.

[0057] According to the image diagnosis support system of the third embodiment described above, since a classification result having a low likelihood of classification is not adopted, the classification accuracy can be improved.

[0058] (Fourth embodiment)

[0059] Figure 5 This is a block diagram showing the functions and configuration of an image diagnosis support system 200 according to the fourth embodiment. The following description will focus on the differences from the first embodiment.

[0060] The image diagnosis support system 200 does not include a determination unit. The first processing unit 112 performs detection processing on the endoscopic image received by the image input unit 110 or an image obtained by reducing the resolution of the endoscopic image. In addition, the first processing unit 112 of the present embodiment does not perform the first classification processing. The second processing unit 114 performs the second classification processing on the area corresponding to the lesion candidate area detected by the detection processing and the area corresponding to the lesion candidate area in the image having a higher resolution than the image used for the detection processing. The output unit 118 outputs the second classification result of the second classification processing as the classification of the lesion candidate area.

[0061] According to the image diagnosis support system 200 of the fourth embodiment described above,

[0062] The present invention has been described above based on the embodiments. These embodiments are illustrative, and those skilled in the art will appreciate that the combination of these components and processing steps may have various modifications, and such modifications are also within the scope of the present invention.

[0063] In the embodiment, the case where the image diagnosis support system 100 supports the diagnosis of a lesion using an endoscopic image captured by a medical endoscope is described, but the present invention is not limited thereto. The image diagnosis support system 100 can also be applied to a case where, for example, the inspection of a metal surface is supported using an endoscopic image captured by an industrial endoscope. For example, in the case of confirming the degree of damage of a damage, a region of interest as a candidate region of a damage may be detected based on the endoscopic image, a mirror reflection region and a non-mirror reflection region of the region of interest may be determined, an edge may be extracted from the non-mirror reflection region, the blur amount of the edge may be calculated, and it may be determined whether the region of interest is a diagnosis failure region accompanied by blur based on the blur amount. If it is not a diagnosis failure region, a classification process for classifying the degree of damage of the damage may be performed, and the classification result may be output. If it is a diagnosis failure region, the classification process may not be performed, and the region of interest may be output as a diagnosis failure region.

[0064] In the embodiment, the case where the learning model for detection and the learning model for the first classification are constructed as different learning models is described, but they can also be constructed as one learning model, preferably as one neural network, and more preferably as one convolutional neural network such as Faster R-CNN.

[0065] The methods of the first embodiment to the fourth embodiment may be arbitrarily combined within a range not contradictory to each other.

[0066] In the embodiments and variations, the image diagnosis support system may also include a processor and a storage device such as a memory. Regarding the processor here, for example, the functions of each part can be implemented by independent hardware, or the functions of each part can also be implemented by integrated hardware. For example, the processor includes hardware, and the hardware can include at least one of a circuit for processing digital signals and a circuit for processing analog signals. For example, the processor can be composed of one or more circuit devices (such as ICs, etc.) mounted on a circuit substrate, and one or more circuit elements (such as resistors, capacitors, etc.). The processor may also be a CPU (Central Processing Unit: Central Processing Unit). However, the processor is not limited to the CPU, and various processors such as GPU (Graphics Processing Unit: Graphics Processing Unit) or DSP (Digital Signal Processor: Digital Signal Processor) can be used. In addition, the processor may also be a hardware circuit based on ASIC (Application Specific Integrated Circuit: Application Specific Integrated Circuit) or FPGA (Field-programmable Gate Array: Field Programmable Gate Array). In addition, the processor may also include an amplifier circuit or a filter circuit for processing analog signals. The memory may be a semiconductor memory such as SRAM, DRAM, or a register, or a magnetic storage device such as a hard disk device, or an optical storage device such as an optical disk device. For example, the memory stores commands that can be read by a computer, and the processor executes the commands, thereby realizing the functions of various parts of the image diagnosis support system. The commands here may be commands that constitute a command set of a program, or commands that instruct the hardware circuit of the processor to act.

[0067] In the embodiment and the modified example, the processing units of the image diagnosis support system may be connected via any form or medium of digital data communication such as a communication network. Examples of the communication network include LAN, WAN, and computers and networks forming the Internet.

[0068] Description of symbols

[0069] 100 is an image diagnosis support system, 110 is an image input unit, 112 is a first processing unit, 114 is a second processing unit, and 116 is a determination unit.

[0070] Industrial Applicability

[0071] The present invention relates to an image diagnosis support system and an image diagnosis support device.

Claims

1. An image diagnosis support system, characterized in that: The image diagnosis support system comprises: An input unit that receives image input; a first processing unit configured to detect a region of interest including an object of interest from the image, classify the detected region of interest, and output a first classification result; as well as a second processing unit that performs semantic segmentation on the region of interest, classifies the region of interest based on a result of the semantic segmentation, and outputs a second classification result; For the shape of the object of interest determined based on the pixels classified as the object of interest by semantic segmentation, if a predetermined condition is satisfied, the first classification result is adopted as the classification of the region of interest, and if the predetermined condition is not satisfied, the second classification result is adopted as the classification of the region of interest, The condition is that the ratio of the short axis to the long axis of the following ellipse is above a specified threshold and the acute angle formed by the long axis of the ellipse and the longitudinal direction of the image is within a specified angle range, wherein the ellipse is obtained by elliptical approximation of a set of pixels classified as the object of interest through semantic segmentation.

2. An image diagnosis support system, characterized in that: The image diagnosis support system comprises: An input unit that receives image input; a first processing unit configured to detect a region of interest including an object of interest from the image, classify the detected region of interest, and output a first classification result; as well as a second processing unit that performs semantic segmentation on the region of interest, classifies the region of interest based on a result of the semantic segmentation, and outputs a second classification result; For the shape of the region of interest, when a predetermined condition is satisfied, the first classification result is used as the classification of the region of interest, and when the predetermined condition is not satisfied, the second classification result is used as the classification of the region of interest, The condition is that the ratio of the short axis to the long axis of the following ellipse is above a specified threshold and the acute angle formed by the long axis of the ellipse and the longitudinal direction of the image is within a specified angle range, wherein the ellipse is obtained by elliptical approximation of a set of pixels classified as the object of interest through semantic segmentation.

3. The image diagnosis support system according to any one of claims 1 to 2, characterized in that: The first processing unit and the second processing unit respectively perform processing based on convolutional neural networks.

4. A recording medium having a program recorded thereon, the program causing a computer to execute the following processing: Accept image input; detecting a region of interest including an object of interest from the image, classifying the detected region of interest and outputting a first classification result; and performing semantic segmentation on the region of interest, classifying the region of interest based on a result of performing the semantic segmentation and outputting a second classification result, For the shape of the object of interest determined based on the pixels classified as the object of interest by semantic segmentation, if a predetermined condition is satisfied, the first classification result is adopted as the classification of the region of interest, and if the predetermined condition is not satisfied, the second classification result is adopted as the classification of the region of interest, The condition is that the ratio of the short axis to the long axis of the following ellipse is greater than a predetermined threshold value, and the acute angle between the long axis of the ellipse and the longitudinal direction of the image is within a predetermined angle range, wherein: The ellipse is obtained by performing ellipse approximation on a set of pixels classified as the object of interest through semantic segmentation.

5. A recording medium having a program recorded thereon, the program causing a computer to execute the following processing: Accept image input; detecting a region of interest including an object of interest from the image, classifying the detected region of interest and outputting a first classification result; and performing semantic segmentation on the region of interest, classifying the region of interest based on a result of performing the semantic segmentation and outputting a second classification result, For the shape of the region of interest, when a predetermined condition is satisfied, the first classification result is used as the classification of the region of interest, and when the predetermined condition is not satisfied, the second classification result is used as the classification of the region of interest, The condition is that the ratio of the short axis to the long axis of the following ellipse is greater than a predetermined threshold value, and the acute angle between the long axis of the ellipse and the longitudinal direction of the image is within a predetermined angle range, wherein: The ellipse is obtained by performing ellipse approximation on a set of pixels classified as the object of interest through semantic segmentation.

6. An image diagnosis support method, characterized in that: The image diagnosis support method comprises the following steps: Accept image input; detecting a region of interest including an object of interest from the image, classifying the detected region of interest and outputting a first classification result; as well as performing semantic segmentation on the region of interest, classifying the region of interest based on a result of performing the semantic segmentation and outputting a second classification result, For the shape of the object of interest determined based on the pixels classified as the object of interest by semantic segmentation, if a predetermined condition is satisfied, the first classification result is adopted as the classification of the region of interest, and if the predetermined condition is not satisfied, the second classification result is adopted as the classification of the region of interest, The condition is that the ratio of the short axis to the long axis of the following ellipse is above a specified threshold and the acute angle formed by the long axis of the ellipse and the longitudinal direction of the image is within a specified angle range, wherein the ellipse is obtained by elliptical approximation of a set of pixels classified as the object of interest through semantic segmentation.

7. An image diagnosis support method, characterized in that: The image diagnosis support method comprises the following steps: Accept image input; detecting a region of interest including an object of interest from the image, classifying the detected region of interest and outputting a first classification result; as well as performing semantic segmentation on the region of interest, classifying the region of interest based on a result of performing the semantic segmentation and outputting a second classification result, For the shape of the region of interest, when a predetermined condition is satisfied, the first classification result is used as the classification of the region of interest, and when the predetermined condition is not satisfied, the second classification result is used as the classification of the region of interest, The condition is that the ratio of the short axis to the long axis of the following ellipse is above a specified threshold and the acute angle formed by the long axis of the ellipse and the longitudinal direction of the image is within a specified angle range, wherein the ellipse is obtained by elliptical approximation of a set of pixels classified as the object of interest through semantic segmentation.

Images