Identifier generation device and image diagnostic support device

Abstract

To update a discriminator, perform discriminant basis analysis and mis-discriminant cause analysis, update the discriminator using the analysis results, and create the discriminator capable of discriminating objects in images with high accuracy.SOLUTION: A discriminator generation apparatus is configured to: obtain a target image; extract the feature amount of the target image from the target image using a discriminator; discriminate the target image from the feature amount using the discriminator to determine a value of the discriminant; determine a discriminant basis using the feature amount and the value of the discriminant; determine an area to be improved using the discriminant basis and a teacher image; and update the discriminator based on the area to be improved.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to classifier generation and image diagnostic support, and for example, to an image processing technique using machine learning to detect specific objects (e.g., cancer cells, etc.) contained in a captured image. [Background technology]

[0002] In recent years, image recognition technology has been exploring methods that utilize machine learning. By using methods such as Deep Learning, the accuracy of object detection within images has been improved. For example, one technique proposed in Patent Document 1 is used to develop a classifier for detecting objects within images. In Patent Document 1, multiple training image data sets are set up, machine learning is performed, and the parameters of a neural network are calculated. [Prior art documents] [Patent Documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2016-143351 [Overview of the project] [Problems that the invention aims to solve]

[0004] However, as in Patent Document 1, even if the training images are divided into multiple image groups and retrained to obtain parameters, there is a problem that the image groups may include images that do not contribute to improving the classifier's recognition accuracy, and the classifier's recognition accuracy does not necessarily improve. Furthermore, in Patent Document 1, the cause of the impediment to the classifier's recognition accuracy is unknown, and there is a problem that the classifier's accuracy does not necessarily improve.

[0005] This invention has been made in view of these circumstances and provides a technology for creating a classifier that can identify objects in an image (for example, abnormal tissue, etc.) with high accuracy. [Means for solving the problem]

[0006] In order to solve the above problems, an identifier generation device according to one aspect of the present invention acquires a target image, extracts a feature amount of the target image from the target image using an identifier, uses the identifier to identify the target image from the feature amount and determines an identification value, determines an identification basis using the feature amount and the identification value, determines an improvement point using the identification basis and a teacher image, and updates the identifier based on the improvement point.

[0007] Further features related to the present invention will become apparent from the description of this specification and the accompanying drawings. Also, aspects of the present invention are achieved and realized by elements and combinations of various elements and the aspects of the following detailed description and the appended claims.

[0008] It should be understood that the description in this specification is merely a typical example and does not limit the scope of the claims of the present invention or its application examples in any sense.

Effect of the Invention

[0009] According to one aspect of the present invention, it becomes possible to create an identifier that can accurately identify an object (for example, an abnormal tissue, etc.) in an image. In addition, it becomes possible to present the identification basis of the identifier and the quality (numerical value) of the identification basis.

Brief Description of the Drawings

[0010] [Figure 1] It is a block diagram showing the functions of an identifier generation device according to the first embodiment. [Figure 2] It is a diagram showing an example of the hardware configuration of an identifier generation device according to the first embodiment or an image diagnosis support device according to the second embodiment. [Figure 3] It is a diagram for explaining an example of the operation of a feature extraction unit. [Figure 4] It is a diagram for explaining an example of the operation of a feature extraction unit. [Figure 5] It is a diagram for explaining an example of the operation of an identification unit. [Figure 6] This is a diagram illustrating an example of the operation of the identification unit. [Figure 7] This is a diagram illustrating an example of the operation of the identification basis analysis unit. [Figure 8] This is a diagram illustrating an example of the operation of the automated root cause analysis unit. [Figure 9A] This diagram illustrates an example of how the output unit displays its identification value (object resemblance), identification / judgment results, etc. [Figure 9B] This diagram illustrates an example of how the output section displays the identification value (identification score), identification basis, etc. [Figure 10A] This is a diagram illustrating an example of the operation of the output section. [Figure 10B] This is an example of displaying the results of the classifier's performance evaluation. [Figure 11] This is a flowchart illustrating the overall operation of the classifier generation device according to the first embodiment. [Figure 12] This block diagram shows the functions of the image diagnostic support device according to the second embodiment. [Figure 13] This is a flowchart illustrating the overall operation of the image diagnostic support device according to the second embodiment. [Figure 14] This diagram shows a schematic configuration of a remote diagnostic support system equipped with a classifier generation device and an image diagnostic support device. [Figure 15] This diagram shows the schematic configuration of an online outsourcing service provision system equipped with a classifier generation device and an image diagnostic support device. [Modes for carrying out the invention]

[0011] Embodiments of this specification provide a classifier generation apparatus and method that enables the creation of a classifier capable of accurately identifying objects in an image (e.g., abnormal tissue, etc.) by updating the classifier parameters during machine learning, analyzing the classifier's identification basis, automatically analyzing the causes of misclassification by the classifier using training images and the identification basis, and further updating the classifier by performing identification basis learning that reflects the analysis results in the creation of the classifier.

[0012] Even if the training image database contains images that do not contribute to improving the classifier's recognition accuracy, by adding a configuration that automatically analyzes the causes of misclassification through the classifier's recognition process and analysis of the recognition basis, and further updating the classifier by performing recognition basis learning that reflects the analysis results in the creation of the classifier, it becomes possible to create a classifier that can identify objects in images with high accuracy. In addition, the classifier's recognition basis and the quality (numerical value) of the recognition basis are presented.

[0013] Embodiments of this specification will be described below with reference to the attached drawings. In the attached drawings, functionally identical elements may be indicated by the same number. The attached drawings show specific embodiments and implementation examples in accordance with the principles of the present invention, but these are for the purpose of understanding the present invention and are not to be used in any way to limit the interpretation of the present invention.

[0014] In this embodiment, the description is sufficient for those skilled in the art to carry out the invention. However, it is important to understand that other implementations and forms are possible, and that the configuration and structure can be changed and various elements replaced without departing from the scope and spirit of the technical idea of ​​the invention. Therefore, the following description should not be interpreted as limiting it to this embodiment.

[0015] Furthermore, the embodiments described herein may be implemented as software running on a general-purpose computer, or as dedicated hardware or a combination of software and hardware, as will be described later.

[0016] In the following, each process in the embodiments of this specification will be described with "each processing unit as a program (e.g., an identification unit)" as the subject (operating entity). However, since a program is executed by a processor (CPU, etc.) and performs the defined processes using memory and communication ports (communication control devices), the description may also be given with the processor as the subject.

[0017] (1) First Embodiment <Functional Configuration of the Identifier Generator> Figure 1 is a block diagram showing the functional configuration of a classifier generation device according to an embodiment of this specification. The classifier generation device 1 includes an input unit 10, a feature extraction unit 11, a classification unit 12, a classification basis analysis unit 13, an automatic cause analysis unit 14, a classification basis learning unit 15, an output unit 16, a recording unit 17, a region division unit 18, a control unit 19, a training image (correct pattern, etc.) 50, and a memory 90.

[0018] However, in the classifier generation device 1, if a training image already exists, the region division unit 18 is not a necessary component. The classifier generation device 1 may be implemented within the image diagnostic support device of the image acquisition device, or, as described later (in the third and fourth embodiments), it may be implemented within a server connected to the image acquisition device via a network.

[0019] In the classifier generation device 1, the identification unit 12, the identification basis analysis unit 13, the automatic cause analysis unit 14, the identification basis learning unit 15, the output unit 16, and the recording unit 17 may be implemented by a program or by hardware modules.

[0020] Image data is input to the input unit 10. For example, the input unit 10 may acquire encoded still image data such as JPG, Jpeg2000, PNG, or BMP format, or WSI, which are captured at predetermined time intervals by an imaging means such as a camera built into an image acquisition device (not shown in Figure 1), and use that image as the input image.

[0021] The input unit 10 may extract still image data of frames at predetermined intervals from moving image data such as MotionJPEG, MPEG, H.264, or HD / SDI format, and acquire those images as input images. The input unit 10 may also use images acquired by the imaging means via a bus or network as input images. The input unit 10 may also acquire images already stored on a removable recording medium as input images. The images and training labels input from the input unit 10 are output to the feature extraction unit 11.

[0022] The feature extraction unit 11 calculates feature quantities (values ​​of each parameter) and output values ​​for each layer of the network (e.g., a Convolutional Neural Network) that constitutes the feature extraction unit 11 from the input image.

[0023] The identification unit 12 receives the output value from the final layer of the network of the feature extraction unit 11 and calculates the identification result of the classifier.

[0024] The identification basis analysis unit 13 uses the parameters of the feature extraction unit 11 and the identification unit 12, and the output values ​​of each layer for the input image, to calculate the identification basis (for example, the contribution rate (real number) of each feature to the identification result).

[0025] When creating a classifier using machine learning, the automated cause analysis unit 14 uses the identification basis and training image 50 calculated by the identification basis analysis unit 13 to analyze the causes of misclassification by the classifier, and calculates the areas of inconsistency (areas for improvement) between the identification basis and the training image, areas for improvement of the features in the classifier network, and the quality of the identification basis (degree of agreement between the identification basis and the training image).

[0026] The identification basis learning unit 15 uses the quality of the identification basis calculated by the automatic cause analysis unit 14, the parameters of the feature extraction unit 11 and the identification unit 12, the output values ​​of each layer of the classifier network for the input image, and the training labels (e.g., 0: normal tissue, 1: abnormal tissue, etc.) to perform machine learning to identify objects in the input image as objects, for example, to identify normal tissues and cells as normal tissues and cells, and to identify abnormal tissues and cells in the input image as abnormal tissues and cells, thereby creating a classifier (each parameter of the feature extraction unit 11 and the identification unit 12 necessary for identification (filter coefficient, offset value, etc.)) from the training image.

[0027] The output unit 16 displays information such as the identification result calculated by the identification unit 12, the identification basis calculated by the identification basis analysis unit 13, and the quality of the identification basis calculated by the automatic cause analysis unit 14 on an output device (display, etc.).

[0028] The recording unit 17 stores information such as values ​​calculated by each unit from the feature extraction unit 11 to the identification basis learning unit 15 in the memory 90.

[0029] The region division unit 18 is not used if a training image already exists. If a training image does not exist, it divides a region within the target image to create a training image (correct pattern).

[0030] The control unit 19 is implemented by a processor and is connected to each element in the classifier generation device 1. The operation of each element in the classifier generation device 1 is performed by the autonomous operation of the respective components described above, or by instructions from the control unit 19.

[0031] In this embodiment, the classifier generation device 1 updates the classifier parameters using training images during machine learning, while the feature extraction unit 11 and the classification unit 12 update the classifier parameters using training images, the classification basis analysis unit 13 analyzes the classification basis of the classifier using the parameters of the feature extraction unit 11 and the classification unit 12 and the output values ​​of each layer for the input image. The automatic cause analysis unit 14 automatically analyzes the cause of misclassification by the classifier using training images and classification basis, and further, the classification basis learning unit 15 updates the classifier by performing classification basis learning that reflects the analysis results in the creation of the classifier. Thus, the device is characterized by creating a classifier that can identify objects in an image (for example, abnormal tissue, etc.) with high accuracy.

[0032] <Hardware configuration of the discriminator generation device> Figure 2 shows an example of the hardware configuration of the classifier generation device 1 according to an embodiment of this specification. The classifier generation device 1 includes a CPU (processor) 201 for executing various programs, a memory (main memory) 202 for storing various programs, and an auxiliary storage device 203 (corresponding to memory 90) for storing various data. The classifier generation device 1 further includes an output device 204 for outputting identification results, identification basis, quality of identification basis, etc., an input device 205 for inputting user instructions, images, etc., and a communication device 206 for communicating with other devices. These are interconnected by a bus 207.

[0033] The CPU 201 reads and executes various programs from memory 202 as needed. Memory 202 stores the following as programs: input unit 10, feature extraction unit 11, identification unit 12, identification basis analysis unit 13, automatic cause analysis unit 14, identification basis learning unit 15, output unit 16, recording unit 17, and region division unit 18.

[0034] The auxiliary storage device 203 stores training images, evaluation images, teacher images, parameters of the classifier generated by the feature extraction unit 11 and the identification unit 12, identification results and identification values, identification basis (contribution rate of each feature to the identification result (real number)) generated by the identification basis analysis unit 13, inconsistencies (areas for improvement) between the identification basis and the teacher image generated by the automatic cause analysis unit 14, areas for improvement of features in the classifier network and quality of the identification basis (degree of agreement between the identification basis and the teacher image), the classifier updated by the identification basis learning unit 15 (parameters of the feature extraction unit 11 and the identification unit 12 necessary for identification (filter coefficients, offset values, etc.)), teacher images generated by the region segmentation unit 18, and display information generated by the output unit 16.

[0035] The output device 204 consists of devices such as a display, printer, and speaker. For example, the output device 204 displays the data generated by the output unit 16 on the display screen.

[0036] The input device 205 consists of devices such as a keyboard, mouse, and microphone. User instructions (including the decision to input each image, such as a training image) are input to the classifier generation device 1 via the input device 205.

[0037] The communication device 206 is not an essential component of the classifier generation device 1. If a communication device is included in a personal computer or the like connected to the image acquisition device, the classifier generation device 1 does not need to have the communication device 206. The communication device 206, for example, receives data (including images, classifiers, etc.) transmitted from other devices connected via a network (e.g., a server) and stores it in the auxiliary storage device 203.

[0038] The classifier generation device described herein creates a classifier capable of accurately identifying objects in an image (e.g., abnormal tissue) by updating the classifier's parameters, analyzing the classifier's identification basis, automatically analyzing the causes of misclassification using training images and the identification basis, and further updating the classifier through identification basis learning that reflects the analysis results in the creation of the classifier.

[0039] <Configuration and operation of each part> The following provides a detailed explanation of the structure and operation of each element.

[0040] (i) Feature extraction unit 11 The feature extraction unit 11 obtains feature quantities from the input image. As an example, an example of how to obtain feature quantities is shown in Figure 3. The CNN in Figure 3 represents a Convolutional Neural Network. For example, the feature extraction unit 11 uses a feature extractor FEA that performs the operation of equation (1) to obtain feature quantities FAi of objects (e.g., tissue, cells, etc.) in the input image A1 from the input image A1.

[0041] The filter coefficient wj shown in equation (1) is a coefficient obtained by machine learning or the like to distinguish objects other than those to be detected (for example, normal tissue and normal cells) from other objects, and objects to be detected from objects to be detected (for example, abnormal tissue and abnormal cells) from other objects.

[0042] In equation (1), pj is the pixel value, bi is the offset value, m is the number of filter coefficients, and h is the nonlinear function. As shown in Figure 4, the feature extraction unit 11 uses equation (1) to calculate the results of each filter 42 from the top left to the bottom right of the target image 41, thereby obtaining the feature quantity fi of an arbitrary filter i. For example, the matrix of feature quantities fi obtained by the feature extractor FEA is used as the feature quantity FAi of the input image A1. The method for creating the feature extractor FEA will be explained later in the section on the identification unit 12.

[0043]

number

[0044] (ii) Identification unit 12 As shown in Figure 5, the identification unit 12 uses the feature quantity FAi (matrix f) of the feature extractor FEA obtained by the feature extraction unit 11 to perform logistic regression processing 51 and calculate a value for the likelihood of detecting an object (likelihood of abnormal tissue, likelihood of lesion, etc.) according to equation (2). Based on the calculated value, the identification unit 12 determines whether an object (e.g., tissue or cell) in the input image A1 is an object to be detected (likely abnormal tissue) or something other than an object to be detected (likely normal tissue).

[0045] In equation (2), w represents the weight matrix, b represents the offset value, g represents the nonlinear function, and y represents the classification result. As described later, the classification unit 12 uses training images to obtain the weights w and offset values ​​b through machine learning. It then uses the weights w and offset values ​​b obtained during training to classify the target image and calculate the classification result y.

[0046]

number

[0047] The identification unit 12 learns the features of objects (e.g., tissues, cells, etc.) using the feature extraction unit 11, for example, known machine learning techniques, so that if an object in the input image is not an object to be detected (e.g., a normal tissue or normal cell), it determines that it is not an object to be detected (e.g., a normal tissue or normal cell) using, for example, logistic regression processing 51.

[0048] Furthermore, the identification unit 12 learns the features of objects (e.g., tissues, cells, etc.) using the feature extraction unit 11 so that if an object in the input image is an object to be detected (e.g., abnormal tissue or abnormal cells), it will determine that it is an object to be detected (e.g., abnormal tissue or abnormal cells, etc.) using logistic regression processing 51. For example, a Convolutional Neural Network may be used as a machine learning technique.

[0049] As shown in Figure 6, during training, the identification unit 12 uses machine learning to create a feature extractor FEA using the feature extraction unit 11 to calculate the feature quantity fi (let's call it FAi) of the input image A1 (e.g., HE-stained image) using equations (1) and (2) so that objects to be detected are identified as objects to be detected (e.g., abnormal tissue and abnormal cells are abnormal tissue and abnormal cells, etc.), and objects other than those to be detected are identified as objects other than those to be detected (e.g., normal tissue and normal cells are normal tissue and normal cells, etc.).

[0050] During training, the identification unit 12 repeatedly performs recognition basis learning using multiple training images, with the feature extraction unit 11, the identification unit 12, and the recognition basis learning unit 15 (described later), to obtain the weights w, filter coefficients wj, and offset values ​​b and bi shown in equations (1) and (2). Using the feature extraction unit 11, it creates a feature extractor FEA that calculates the feature quantity FAi of input image A1 from input image A1. The identification unit 12 then uses the feature extractor FEA to calculate the recognition result of the target image.

[0051] The identification unit 12 stores the calculated weights w, filter coefficients wj, and offset values ​​b and bi in the memory 90.

[0052] (iii) Identification basis analysis unit 13 Using the result z before applying the nonlinear function h in equation (1), the discrimination basis analysis unit 13 calculates the discrimination basis (the contribution rate of each feature to the discrimination result (real number)) as shown in Figure 7, using equations (3) and (4).

[0053] In the example shown in Figure 7, the input image 71 includes a region of abnormal tissue 73 and a region of normal tissue 72. Regions other than regions 73 and 72 are also regions of normal tissue. The identification basis 75 indicates the regions of identification basis for abnormal tissue and regions of identification basis for normal tissue. In the example in Figure 7, the regions of identification basis for abnormal tissue are shown in black, and the regions of identification basis for normal tissue are shown in white or with a dot pattern. Region 77 is a region of identification basis for abnormal tissue that is included in region 73 of abnormal tissue. Region 78 is a region of identification basis for normal tissue that is included in region 73 of abnormal tissue. Region 79 is a region of identification basis for abnormal tissue that is included in region 72 of normal tissue.

[0054] In equation (3), P is the input data from the previous layer consisting of pj, L is the index of the layer for which the identification basis is calculated, and C L This is the identification basis for layer L, C L+1 This represents the discrimination basis for layer L+1. Also, W -1 This is a matrix obtained by swapping any dimension of the matrix of filter coefficients wj in equation (1) and reversing the direction of any dimension. For example, W -1 This matrix is ​​obtained by swapping the first dimension (number of features) and the second dimension (number of channels) of the filter coefficient matrix wj in equation (1), and reversing the orientation of the third dimension (vertical size) and the fourth dimension (horizontal size).

[0055]

number

[0056] Furthermore, in the classifier layers, the layer that acquires the maximum value (for example, the Max Pooling layer) sets the classification basis to 1 only for pixels where the value of each region ri is the maximum, and 0 for all others. Therefore, for example, the classification basis is propagated only to pixels where the classification basis input from the adjacent layer on the output layer is 1.

[0057] Furthermore, the input data of the final layer of the classifier (e.g., Logistic Regression layer) is P LRAssuming the label is a one-hot vector format training signal, the discrimination basis is calculated using equation (4) and backpropagated to the layer before the classifier.

[0058]

number

[0059] (iv) Automatic cause analysis section 14 The automated cause analysis unit 14 uses the identification basis calculated by the identification basis analysis unit 13 and the manually created training image or the training image created by the region segmentation unit 18 to automatically analyze the region causing the misidentification, as shown in Figure 8. The training image (correct pattern) 81 is a monochrome image of the same size as the input image 71 and takes the value of, for example, 0 or 1. For example, a region with a training image of 0 represents a region other than an object region (for example, a normal tissue region, etc.), and a region with a training image of 1 represents an object region (for example, an abnormal tissue region, etc.).

[0060] It is desirable that regions other than object regions be identified from features within those regions, and it is undesirable to use object regions as the basis for identification. Furthermore, it is desirable that object regions (e.g., abnormal tissue regions) be identified from features within object regions (e.g., abnormal tissue regions), and it is undesirable to use regions other than object regions (e.g., normal tissue regions) as the basis for identification.

[0061] Therefore, the automated cause analysis unit 14 compares the identification basis 75 calculated by the identification basis analysis unit 13 with the areas other than the object area and the object area of ​​the training image (correct pattern) 81, and identifies the identification basis appearing in the inappropriate area as the cause of misidentification. The automated cause analysis unit 14 also analyzes the cause of misidentification by dividing it into the object area and the area other than the object area. In the example in Figure 8, location 82 indicates a location in the abnormal tissue area where the identification basis for the abnormal tissue area is insufficient, and location 83 indicates a location in the normal tissue area where the identification basis for the abnormal tissue area is shown.

[0062] The automated cause analysis unit 14 generates a misidentification cause score e for areas other than the object region. backand the misidentification cause score e for the object region fore is calculated using Equations (5) and (6). Here, C sum is the basis for identification with respect to the object region in the input image calculated by the identification basis analysis unit 13, and mask represents the teacher image. However, the basis for identification C sum for regions other than the object region may be used. In that case, the region where the teacher image is 0 represents the object region (e.g., abnormal tissue region, etc.), and the region where the teacher image is 1 represents regions other than the object region (e.g., normal tissue region, etc.).

[0063] [Number]

[0064] [Number]

[0065] If there is a basis for identification C sum in a region other than the object region, there is a possibility that features unsuitable for identification are being used as the basis for identification, which may cause a decrease in the accuracy of the identifier. Therefore, the automatic cause analysis unit 14 extracts the basis for identification C sum included in the region other than the object region and calculates the misidentification cause score e back for the region other than the object region. The larger the number of positive basis for identification values included in the region other than the object region, the larger the value of e back .

[0066] The object region is a region that contains features for identifying the object region, and it is desirable that a large number of positive basis for identification values C sum are included in the object region. If the basis for identification C sum in the object region is insufficient, it may cause the object to be missed. Therefore, the automatic cause analysis unit 14 calculates the misidentification cause score e sum for the object region using the negative values of the basis for identification C fore included in the object region.

[0067] The larger the ratio of negative basis for identification values and small basis for identification values, the larger the value of e foreThe value becomes large, and the greater the proportion of reasons for identifying a positive value, the higher e fore This will be a small value. In addition, the automatic cause analysis unit 14 uses, for example, equations (7) and (8) to determine the quality of the identification basis outside the object region (the degree of agreement between the identification basis and the training image). back The quality of the basis for identifying object regions fore You may also calculate the following, where X represents the horizontal number of pixels in the target image and Y represents the vertical number of pixels in the target image.

[0068]

number

[0069]

number

[0070] Using a certain number of evaluation images, the coin shown in equation (9) is used for each classifier. all Calculate coin all The performance of the classifier is determined by comparing these. For example, coin all The larger the value, the better the performance of the classifier is judged to be.

[0071]

number

[0072] (v) Identification basis learning unit 15 The identification basis learning unit 15 uses the misidentification cause score calculated by the automatic cause analysis unit 14 and the identification result y calculated by the identification unit 12 to perform identification basis learning to correct the identification basis of the region estimated to be the cause of misidentification. Multiple costs are used in the identification basis learning. For example, three types of costs can be used. The identification basis learning unit 15 uses the Negative Log Likelihood (nll) of equation (10), which is used in normal machine learning, as the first, and the cost cgl, which relates to the cause of misidentification in regions other than the object region, as shown in equation (11). back The third is the cost cgl related to the cause of misidentification within the object region shown in equation (12). foreUse the following. However, the label in equation (10) represents the teacher label indicating areas outside the object region and areas within the object region (for example, 0 for areas outside the object region and 1 for areas within the object region).

[0073]

number

[0074]

number

[0075]

number

[0076] The score e of the misidentification cause calculated by the automated cause analysis unit 14 back This represents the identification basis that is included in an area other than the object area that should not be used as the identification basis. Therefore, the identification basis learning unit 15 uses the e calculated by the automatic cause analysis unit 14. back The average value of cost cgl back By doing so, e back Machine learning progresses in a way that reduces the value of, suppressing the basis for identification in areas other than the object region. Also, the score e for the cause of misidentification fore This represents the region within the object region where the value of the identification basis is low. Therefore, the identification basis learning unit 15 uses the e calculated by the automatic cause analysis unit 14. fore The average value of cost cgl fore This increases the basis for identification within the object's domain.

[0077] As shown in equation (13), the identification basis learning unit 15 takes the sum of the three costs as the total cost, performs machine learning to reduce the cost, and updates the parameters of the feature extraction unit 11 and the identification unit 12 while correcting the identification basis of the misidentification cause region obtained by the automatic cause analysis unit 14, thereby creating a classifier capable of identifying objects with high accuracy.

[0078]

number

[0079] (vi) Output section 16 The output unit 16 uses the identification result y obtained by the created classifier to display, for example, the result of object-likeness determination (e.g., lesion-likeness determination), the identification basis, and the quality of the identification basis in a GUI (graphical user interface) as shown in Figures 9A and 9B.

[0080] The screen shown in Figure 9A is displayed, for example, when using the classifier. Figure 9A shows an example for the breast, illustrating the classification results for non-tumor and tumor (abnormal tissue, abnormal cells, etc.). The example screen in Figure 9A displays the classification result 91, object-likeness (tumor-likeness) 92, image display button 93, and identification reason display button 94. When the image display button 93 is pressed by the user, for example, the input image is displayed. When the identification reason display button 94 is pressed, the identification reason is displayed.

[0081] In the example shown in Figure 9A, the identification unit 12 classifies the breast as containing a tumor, which is abnormal tissue / cells, in the input target image, and calculates a tumor object-likeness value of 0.89.

[0082] Figure 9B is displayed, for example, during the training or use of the classifier. Figure 9B is an example of displaying an input image, the basis for identification of the input image, and the quality of the basis for identification. Figure 9B shows an example in which the quality of the basis for identification of non-tumor is calculated to be 0.96 and the quality of the basis for identification of tumor is calculated to be 0.93. The screen of Figure 9B may be displayed, for example, in response to the operation of the image display button 93 or the identification reason display button 94 shown in Figure 9A, and may be displayed independently of the screen of Figure 9A.

[0083] Furthermore, when an unknown image is input from the input unit 10 and the identification results of each classifier are displayed, if an object in the image is determined to be an object (for example, abnormal tissue or abnormal cells), the output unit 16 may draw a detection frame 102 within the input target image 101 to indicate the location of the object to be detected (for example, an area suspected to be abnormal tissue or abnormal cells), as shown in Figure 10A. On the other hand, if it is determined to be normal tissue or normal cells, the detection frame 102 may not be drawn on the input target image 101, and the input target image 101 may be displayed as is. Also, as shown in Figure 10A, the result of the determination of object-likeness (for example, a tumor) is displayed.

[0084] Furthermore, the output unit 16 outputs the calculated performance value of each discriminator, coin all Using this, as an example, as shown in Figure 10B, the performance of each classifier is determined and the results of the classifier performance determination are displayed. Figure 10B is displayed, for example, during classifier training or use. Figure 10B is an example where the performance value of classifier A is 1.97, the performance value of classifier B is 1.65, and the classifier performance determination result is displayed as classifier A.

[0085] The output unit 16 is not an essential component of the classifier generation device 1, and if the image diagnostic support device includes an output unit, the classifier generation device 1 does not need to have an output unit 16.

[0086] (vii) Record Section 17 The recording unit 17 stores in memory 90 the input image (training image, evaluation image, etc.), the training image, the parameters of the classifier generated by the feature extraction unit 11 and the classification unit 12, the classification result and classification value, and the classification basis (the contribution rate (real number) of each feature to the classification result) generated by the classification basis analysis unit 13. The recording unit 17 also stores in memory 90 the discrepancies (areas for improvement) between the classification basis and the training image generated by the automatic cause analysis unit 14, the areas for improvement of features in the classifier network, and the quality of the classification basis (degree of agreement between the classification basis and the training image). In addition, the recording unit 17 stores in memory 90 the classifier updated by the classification basis learning unit 15 (the parameters of the feature extraction unit 11 and the classification unit 12 necessary for classification (filter coefficients, weights, offset values, etc.)), the training image generated by the region segmentation unit 18, and display information generated by the output unit 16.

[0087] The region division unit 18 divides each pixel in the input image into regions (for example, object region, non-object region, etc.) using a region division method such as U-Net, and creates a training image.

[0088] <Processing procedure for the identifier generation device> Figure 11 is a flowchart illustrating the operation of the classifier generation device 1 according to the embodiment of this specification. In the following description, each processing unit (input unit 10, feature extraction unit 11, etc.) will be described as the main operating unit, but it may also be interpreted that the CPU 201 is the main operating unit and that the CPU 201 executes each processing unit as a program.

[0089] (i) Step 1101 The input unit 10 receives a training image and outputs the input image to the feature extraction unit 11.

[0090] (ii) Step 1102 The feature extraction unit 11 uses machine learning to obtain feature quantities FAi of objects (e.g., tissues, cells, etc.) in the input image A1 using a filter according to equation 1 described above, and creates a feature extractor FEA. The feature extraction unit 11 then obtains filter coefficients wj and offset values ​​bi for the feature quantities FAi.

[0091] (iii) Step 1103 The identification unit 12 uses machine learning to obtain identification results from the feature quantity FAi using Equation 2, calculates an identification value for object-likeness (e.g., lesion-likeness), and determines each parameter of Equation 2 (weight w, offset value b, etc.) for obtaining the identification value, in order to determine whether the object in the input image A1 is an object that should be detected (e.g., normal cell or abnormal cell).

[0092] (iv) Step 1104 The identification basis analysis unit 13 determines the identification basis of the target image using equations 3 and 4.

[0093] (v) Step 1105 The automated cause analysis unit 14 uses the identification basis and training image calculated by the identification basis analysis unit 13 according to equations 5 and 6 to calculate the misidentification cause score e back and e fore The quality of the identification basis is calculated using equations 7 and 8. back and coin fore We seek.

[0094] (vi) Step 1106 The discrimination basis learning unit 15 calculates the total cost in equation 12 using equations 9, 10, and 11, and uses machine learning to update the filter coefficients wj, weights w, and offsets bi and b so that the cost is minimized, thereby creating a classifier.

[0095] (vii) Step 1107 The recording unit 17 stores each parameter of Equation 1 and Equation 2 (filter coefficient wj, weight w, offset values ​​bi, b, etc.), the basis for identification, and the quality of the basis for identification in the memory 90.

[0096] According to the first embodiment, during machine learning, the classifier's parameters are updated while the classifier's identification basis is analyzed, the causes of misclassification by the classifier are automatically analyzed using the training image and the identification basis, and furthermore, the classifier is updated by performing identification basis learning that reflects the analysis results in the creation of the classifier, thereby making it possible to create a classifier that can identify objects in an image (for example, abnormal tissue, etc.) with high accuracy.

[0097] Furthermore, it becomes possible to present the identification criteria that explain the reason for identifying the target image, as well as the quality of the identification criteria that demonstrates the accuracy of the classifier.

[0098] (2) Second embodiment As shown in Figure 12, the image diagnostic support device 2 according to the second embodiment includes many of the same components as the classifier generation device 1 in Figure 1 of the first embodiment. However, unlike the first embodiment, it does not include the identification basis learning unit 15, but instead includes a region division unit 18, a feature extraction unit 21, an identification unit 22, an identification basis analysis unit 23, and an automatic cause analysis unit 24. Therefore, the configuration that differs from that in Figure 1 will be described here.

[0099] In one embodiment of this specification, the image diagnostic support device 2 reads the parameters of the classifier from memory, and the region division unit 18 creates a training image of the evaluation image by region division. Furthermore, the identification basis analysis unit 23 analyzes the identification basis of the evaluation image. The automatic cause analysis unit 24 automatically analyzes the cause of misidentification by the classifier using the training image and the identification basis, and analyzes the quality of the identification basis of the evaluation image, thereby presenting the identification result of objects (e.g., tissues, cells, etc.) in the evaluation image, the identification basis, and the quality of the identification basis.

[0100] <Configuration and operation of each part> The following provides a detailed explanation of the configuration and operation of each element, which differs from that shown in Figure 1.

[0101] (i) Area dividing section 18 The region segmentation unit 18 divides the evaluation image into regions using a segmentation model (e.g., U-Net) that separates the object region from the non-object region within the evaluation image, and creates a training image.

[0102] (ii) Feature extraction unit 21 The feature extraction unit 21 reads the parameters of the trained feature extractor from the memory 90 and uses the feature extractor to calculate the feature quantity FAi of the evaluation image according to Equation 1.

[0103] (iii) Identification part 22 The identification unit 22 reads each parameter of the trained classifier from the memory 90, and uses the classifier to identify the evaluation image from the feature quantity FAi according to equation 2, and calculates the identification result y and the identification value.

[0104] (iv) Identification basis analysis unit 23 The identification basis analysis unit 23 determines the identification basis of the evaluation image using equations 3 and 4.

[0105] (v) Automatic cause analysis section 24 The automated cause analysis unit 24 uses Equations 5 and 6 to calculate discrepancies between the identification basis and the training image (areas for improvement), the misidentification cause score, and the quality of the identification basis (degree of agreement between the identification basis and the training image), using the identification basis calculated by the identification basis analysis unit 23 and the training image (correct pattern) of the evaluation image obtained by the region segmentation unit 18.

[0106] <Hardware configuration of the medical imaging support system> The hardware configuration example of the image diagnostic support device 2 according to the embodiment of this specification has the same configuration as in Figure 2, but unlike the classifier generation device 1 according to the first embodiment, the memory 202 stores the region division unit 18, feature extraction unit 21, identification unit 22, identification basis analysis unit 23, and automatic cause analysis unit 24.

[0107] The auxiliary storage device 203 of the image diagnostic support device 2 stores the identification result y obtained by the identification unit 22, the identification basis obtained by the identification basis analysis unit 23, the quality of the identification basis obtained by the automatic cause analysis unit 24, and so on.

[0108] <Processing Procedure for Image Diagnostic Support Devices> Figure 13 is a flowchart illustrating the operation of the image diagnostic support device 2 according to the embodiment of this specification. In the following description, each processing unit (input unit 10, feature extraction unit 21, etc.) will be described as the main operating unit, but it may also be interpreted that the CPU 201 is the main operating unit and that the CPU 201 executes each processing unit as a program.

[0109] (i) Step 1301 The input unit 10 receives an evaluation image and outputs the input image to the feature extraction unit 21.

[0110] (ii) Step 1302 The feature extraction unit 21 reads the filter coefficients wj and offset value bi of the trained feature extractor FEA from the memory 90, and uses the filter according to equation 1 above to obtain the feature quantity FAi of the object (e.g., tissue or cell) in the input image A1.

[0111] (iii) Step 1303 The identification unit 22 reads the parameters of the trained classifier (weight w, offset value b, etc.) from the memory 90, obtains the identification result from the feature quantity FAi using equation 2, calculates an identification value for object-likeness (e.g., lesion-likeness), and determines whether the object in the input image A1 is an object that should be detected (e.g., a normal cell or an abnormal cell).

[0112] (iv) Step 1304 The identification basis analysis unit 23 determines the identification basis of the target image using equations 3 and 4.

[0113] (v) Step 1305 The automated cause analysis unit 24 uses the identification basis calculated by the identification basis analysis unit 23 and the training image of the target image obtained by the region segmentation unit 18, according to equations 5 and 6, to calculate the misidentification cause score e back and e fore The quality of the identification basis is calculated using equations 7 and 8. back and coin fore We seek.

[0114] (vi) Step 1306 The recording unit 17 stores the identification result, identification value, identification basis, and quality of the identification basis in the memory 90.

[0115] According to the second embodiment, by reading each parameter of the classifier from memory, creating a training image of the evaluation image by region segmentation, and further analyzing the quality of the identification basis of the evaluation image based on the classifier using the training image and the identification basis while analyzing the identification basis of the evaluation image, it becomes possible to present the identification result of objects (e.g., tissues, cells, etc.) in the evaluation image, the identification basis, and the quality of the identification basis.

[0116] Furthermore, by changing the classifier to a different classifier from another facility, etc., and presenting the basis for identification in the evaluation images and the quality of that basis, it becomes possible to judge the performance of classifiers created elsewhere. Also, by switching between multiple classifiers and presenting the basis for identification in the evaluation images and the quality of that basis, it becomes possible to determine which classifier can be used to identify objects in the evaluation images with high accuracy.

[0117] (3) Third Embodiment Figure 14 is a functional block diagram showing the configuration of a remote diagnostic support system 1400 according to a third embodiment of this specification. The remote diagnostic support system 1400 includes a server 1403 and an image acquisition device 1405. The server 1403 can be configured, for example, with one or more computers.

[0118] The image acquisition device 1405 is a device such as a virtual slide device or a personal computer equipped with a camera, and has an imaging unit 1401 for capturing image data and a display unit 1404 for displaying identification / determination results transmitted from the server 1403. Although not shown in the figures, the image acquisition device 1405 also has a communication device for transmitting image data to the server 1403 and receiving data transmitted from the server 1403.

[0119] Server 1403 includes an image diagnostic support device 2 that creates a classifier using the classifier generation device 1 according to the first embodiment of this specification on image data (input image, training image) transmitted from the image acquisition device 1405, and performs image processing using the created classifier, and a storage unit 1402 that stores the identification / judgment results, identification basis / quality of identification basis, etc. output from the image diagnostic support device 2. Although not shown in the figures, Server 1403 also has a communication device that receives image data transmitted from the image acquisition device 1405 and transmits data such as identification / judgment results, identification basis / quality of identification basis, etc. to the image acquisition device 1405.

[0120] The image diagnostic support device 2 uses the classifier generated by the classifier generation device 1 to classify objects (e.g., tissues and cells, etc.) in the image data captured by the imaging unit 1401, determining whether or not there are objects to be detected (e.g., abnormal tissues and abnormal cells (e.g., cancer), etc.). The display unit 1404 displays the identification / judgment results, identification basis / quality of identification basis, and other data transmitted from the server 1403 on the display screen of the image acquisition device 1405.

[0121] As the image acquisition device 1405, a regenerative medicine device or iPS cell culture device having an imaging unit, or an MRI or ultrasound imaging device may be used.

[0122] According to the third embodiment, for objects (e.g., tissues and cells) in images transmitted from facilities at different locations, the system accurately classifies whether or not they are objects to be detected (abnormal tissues and abnormal cells) using the parameters of the classifier obtained by the classifier generation device 1. The identification / judgment results, identification basis / quality of identification basis, and other data are transmitted to facilities at different locations, and the identification / judgment results, identification basis / quality of identification basis, and other data are displayed on the display unit of the image acquisition device at those facilities, thereby making it possible to provide a remote diagnostic support system.

[0123] (4) Fourth Embodiment Figure 15 is a functional block diagram showing the configuration of a network outsourcing service provision system 1500 according to the fourth embodiment of this specification. The network outsourcing service provision system 1500 includes a server 1503 and an image acquisition device 1505. The server 1503 can be configured with, for example, one or more computers.

[0124] The image acquisition device 1505 is, for example, a device such as a virtual slide device or a personal computer equipped with a camera. The image acquisition device 1505 includes an imaging unit 1501 that captures image data, a storage unit 1504 that stores classifiers transmitted from the server 1503, and an image diagnostic support device 2. The image diagnostic support device 2 reads the classifiers transmitted from the server 1503 and classifies whether objects (e.g., tissues, cells, etc.) in the newly captured image by the imaging unit 1501 of the image acquisition device 1505 are objects to be detected (e.g., abnormal tissues, abnormal cells, etc.) using classifiers obtained from the classifier generation device 1 according to the first embodiment of this specification. Although not shown in the figures, the image acquisition device 1505 also has a communication device that transmits image data to the server 1503 and receives data transmitted from the server 1503.

[0125] Server 1503 has a storage unit 1502 that creates a classifier from the classifier generation device 1 according to the first embodiment of this specification using image data transmitted from the image acquisition device 1505 and stores the classifier output from the classifier generation device 1. Server 1503 also accepts input of training images from the operator or creates training images using the region division unit 18. Server 1503 also presents the operator with each output of the classifier generation device 1 (identification / judgment result, identification basis, and quality of identification basis, etc.). Although not shown in the figures, Server 1503 has a communication device that receives image data transmitted from the image acquisition device 1505 and transmits classifiers to the image acquisition device 1505.

[0126] Furthermore, the classifier generation device 1 performs machine learning on objects (e.g., tissues and cells, etc.) in the image data captured by the imaging unit 1501 to determine which objects should be detected (e.g., normal tissues and cells are treated as normal tissues and cells, abnormal tissues and cells as abnormal tissues and cells, etc.), and creates a classifier that calculates feature quantities for objects (e.g., tissues and cells, etc.) in images of facilities at different locations. The storage unit 1504 stores the classifiers and the like transmitted from the server 1503.

[0127] The image diagnostic support device 2 within the image acquisition device 1505 reads an identifier from the storage unit 1504 and uses the identifier to classify whether an object (e.g., tissue or cells) in a newly captured image by the imaging unit 1501 of the image acquisition device 1505 is an object to be detected (e.g., abnormal tissue or abnormal cells), and displays the identification / judgment result, identification basis / quality of identification basis, and other data on the display screen of the output device 204 of the image diagnostic support device 2.

[0128] As the image acquisition device 1505, a regenerative medicine device or iPS cell culture device having an imaging unit, or an MRI or ultrasound imaging device may be used.

[0129] According to the fourth embodiment, a system for providing online outsourced services is created by performing machine learning to classify objects (e.g., tissues and cells) in images transmitted from facilities at different locations into objects to be detected (e.g., normal tissues and cells are classified as normal tissues and cells, abnormal tissues and cells as abnormal tissues and cells, etc.), transmitting the classifier to facilities at different locations, reading the classifier with an image acquisition device at those facilities, and classifying whether objects (e.g., tissues and cells) in newly captured images are objects to be detected (e.g., abnormal tissues and abnormal cells, etc.).

[0130] The embodiments described above can be modified as follows: In the feature extraction unit 11, multiple features were obtained using a filter via machine learning, but other features such as HOG may be used and will have a similar effect. Also, in the discrimination unit 12, negative log likelihood was used as the loss function (cost), but squared error or Hinge loss may be used and will have a similar effect.

[0131] The present invention can also be realized by software program code that implements the functions of the embodiments. In this case, a storage medium on which the program code is recorded is provided to a system or device, and the computer (or CPU or MPU) of that system or device reads the program code stored on the storage medium. In this case, the program code read from the storage medium itself implements the functions of the embodiments described above, and the program code itself and the storage medium on which it is stored constitute the present invention. Examples of storage media used to supply such program code include flexible disks, CD-ROMs, DVD-ROMs, hard disks, optical disks, magneto-optical disks, CD-Rs, magnetic tapes, non-volatile memory cards, ROMs, and the like.

[0132] Furthermore, based on the instructions in the program code, the operating system (OS) running on the computer may perform some or all of the actual processing, thereby realizing the functions of the embodiment described above. In addition, after the program code read from the storage medium is written to the computer's memory, the computer's CPU may perform some or all of the actual processing based on the instructions in the program code, thereby realizing the functions of the embodiment described above.

[0133] Furthermore, the program code for the software that realizes the functions of the embodiment may be distributed via a network and stored in a storage means such as a hard disk or memory of the system or device, or in a storage medium such as a CD-RW or CD-R, so that when in use the system or device's computer (or CPU or MPU) reads and executes the program code stored in the storage means or storage medium.

[0134] Finally, the processes and techniques described herein are not inherently related to any specific device and can be implemented by any suitable combination of components. Furthermore, a wide variety of general-purpose devices can be used according to the methods described herein. In some cases, it may be beneficial to construct a dedicated device to perform the steps of the methods described herein. Also, various inventions can be formed by appropriate combinations of the multiple components disclosed in the embodiments. For example, some components may be removed from all the components shown in the embodiments. Furthermore, components from different embodiments may be combined as appropriate. The present invention has been described in relation to specific examples, but these are for illustrative purposes only and not for limitation in all respects. Those skilled in the art will find that there are many combinations of hardware, software, and firmware suitable for carrying out the present invention. For example, the described software can be implemented in a wide range of programming or scripting languages ​​such as assembler, C / C++, Perl, Shell, PHP, Java®, etc.

[0135] Furthermore, in the embodiments described above, the control lines and information lines shown are those deemed necessary for illustrative purposes, and not all control lines and information lines are necessarily shown in the actual product. All components may be interconnected.

[0136] In addition, other implementations of the present invention will become apparent to those with ordinary skill in the art from the description and embodiments disclosed herein. Various aspects and / or components of the described embodiments can be used individually or in any combination. [Explanation of Symbols]

[0137] 1... Discriminator generation device 2. Medical imaging support device 10.. Input section 11. Feature extraction section 12. Identification section 13. Identification Basis Analysis Department 14...Automatic cause analysis department 15. Learning Department for Identification Basis Output section of 16... 17...Records Department 18...area division part 19.. Control Unit 21. Feature extraction section 22...Identification section 23. Identification Basis Analysis Department 24...Automatic cause analysis department 1400... Remote diagnostic support system 1500...Net outsourcing service provision system

Claims

1. A processor that executes a program for image processing on the target image, Includes the program and memory for storing the results of the image processing, The memory stores the target image and the classifier, The aforementioned processor, Using the aforementioned classifier, feature quantities of the target image are extracted from the target image. Using the aforementioned classifier, the target image is identified from the feature quantities and the identification value is determined. Using the aforementioned feature quantities and the aforementioned identification values, the basis for identification in the target image is determined. Using the aforementioned identification criteria and the training image of the aforementioned identification criteria, the areas to be improved are determined. A classifier generating device that updates the classifier using the cost based on the classifier's identification value and the cost based on the improvements made.

2. A classifier generating device according to claim 1, The processor is a classifier generating device that, in determining the areas for improvement, analyzes the causes of misidentification by the classifier and creates the analysis results and the quality of the identification basis for the target image.

3. A method for generating a classifier, wherein the system creates a classifier that classifies a desired object in a target image, The aforementioned system, A processor that executes a program for image processing on the target image, Includes the program and memory for storing the results of the image processing, The memory stores the target image and the classifier, The aforementioned discriminator generation method involves the processor, Using the aforementioned classifier, feature quantities of the target image are extracted from the target image. Using the aforementioned classifier, the target image is identified from the feature quantities and the identification value is determined. Using the aforementioned feature quantities and the aforementioned identification values, the basis for identification in the target image is determined. Using the aforementioned identification criteria and the training images of the aforementioned identification criteria, the areas for improvement of the classifier are determined. A method for generating a classifier, which involves updating the classifier using the cost based on the classifier's identification value and the cost based on the improvements made.

4. A method for generating a classifier according to claim 3, The method for determining the areas for improvement involves analyzing the causes of misidentification by the classifier and creating a quality assessment of the analysis results and the basis for identifying the target image.

5. An image acquisition device having an imaging device for capturing image data, The classifier generating device according to claim 1, Includes an image diagnostic support device, The image acquisition device transmits the image data to the classifier generation device. The classifier generation device generates a classifier using the received image data, The image diagnostic support device acquires the image of the object from the classifier and the image acquisition device. Using the aforementioned classifier, feature quantities of the object's image are extracted from the image of the object. Using the classifier, the object is identified from the feature quantities to determine the identification value and the identification result. Using the aforementioned feature quantities and the aforementioned identification values, the basis for identification in the target image is determined. Using the aforementioned identification criteria and the training images of the aforementioned identification criteria, the areas for improvement of the classifier are determined. The identification result, the basis for the identification, and the information based on the improvement location are transmitted to the image acquisition device. The image acquisition device is a remote diagnostic support system that displays an image of the object, the identification result, and information based on the identification basis and the improvement location on a display device.

6. An image acquisition device including an imaging device for capturing image data and an image diagnostic support device, The identification device according to claim 1, The image acquisition device transmits the image data to the classifier generation device. The classifier generation device generates a classifier using the received image data, The classifier generation device transmits the generated classifier to the image acquisition device. The image acquisition device stores the received identifier in memory. The aforementioned image diagnostic support device is The classifier and the image of the object from the imaging device are acquired. Using the aforementioned classifier, feature quantities of the object's image are extracted from the image of the object. Using the classifier, the object is identified from the feature quantities to determine the identification value and the identification result. Using the aforementioned feature quantities and the aforementioned identification values, the basis for identification in the target image is determined. Using the aforementioned identification criteria and the training images of the aforementioned identification criteria, the areas for improvement of the classifier are determined. A contracted service provision system that displays an image of the object, the identification result, and information based on the identification basis and the improvement location on a display device.

7. A contract service provision system according to claim 6, The aforementioned image diagnostic support device is In determining the areas for improvement, the cause of misidentification by the classifier is analyzed, and the analysis results and the quality of the identification basis for the target image are created. A contracted service delivery system that includes the quality of the identification basis in the information to be displayed.

8. A contract service provision system according to claim 6, The information to be displayed includes the target image and the quality of the basis for identifying the target image. The aforementioned image diagnostic support device is a contract service provision system that, in determining the areas to be improved, compares the performance values ​​of each classifier, determines the performance of the classifier, and displays the information of the determination result on the display device.

Images