Recognizer generation device and image diagnosis assistance device

Abstract

The recognizer generation device updates the recognizer, analyzes the recognition basis, and analyzes the cause of erroneous recognition, and uses the analysis results to update the recognizer, thereby making a recognizer capable of recognizing an object in an image with high precision. The recognizer generation device acquires an object image, extracts a feature amount of the object image from the object image using a recognizer, recognizes the object image from the feature amount using the recognizer and determines a recognition value, determines a recognition basis using the feature amount and the recognition value, and determines an improvement site using the recognition basis and a teaching image. The identifier is updated based on the improvement site.

Description

[0001] Introduction by reference

[0002] This application claims priority from Japanese patent application No. 2022-099357 filed on June 21, 2022, the contents of which are incorporated herein by reference. Technical Field

[0003] The present invention relates to identifier generation and image diagnosis assistance, for example, to an image processing technology using machine learning for detecting specific objects (eg, cancer cells, etc.) included in a captured image. Background Art

[0004] In recent years, image recognition technology using machine learning and the like has been studied. By using deep learning and the like, the detection accuracy of objects in an image has been improved. In order to develop a recognizer for detecting objects in an image, there is a technology proposed in Japanese Patent Publication No. 2016-143351, for example. In Japanese Patent Publication No. 2016-143351, a plurality of learning image data sets are set to perform machine learning and calculate the parameters of a neural network. Summary of the invention

[0005] Problems to be solved by the invention

[0006] However, even if the learning images are divided into a plurality of image groups and relearned to obtain parameters as in Japanese Patent Application Laid-Open No. 2016-143351, the image groups may contain images that do not contribute to the improvement of the recognition accuracy of the recognizer, and there is a problem that the recognition accuracy of the recognizer may not be improved. In addition, in Japanese Patent Application Laid-Open No. 2016-143351, the cause of the impairment of the recognition accuracy of the recognizer is unclear, and there is a problem that the accuracy of the recognizer may not be improved.

[0007] The present invention has been made in view of such circumstances, and provides a technique for realizing the production of an identifier capable of highly accurately identifying an object (for example, abnormal tissue, etc.) in an image.

[0008] Means for solving problems

[0009] In order to solve the above-mentioned problems, an identifier generation device of one embodiment of the present invention obtains an object image, uses an identifier to extract a feature value of the object image from the object image, uses the identifier to identify the object image from the feature value and determine an identification value, uses the feature value and the identification value to determine an identification basis, uses the identification basis and a teaching image to determine an improvement area, and updates the identifier based on the improvement area.

[0010] Further features related to the present invention will become apparent from the description of this specification and the accompanying drawings. In addition, the present invention is achieved through the combination of elements and a plurality of elements and through the detailed description below and the scope of the appended claims.

[0011] The descriptions in this specification are merely typical examples, and it should be understood that they do not limit the scope of patent protection or application examples of the present invention in any sense.

[0012] Effects of the Invention

[0013] According to one embodiment of the present invention, it is possible to create an identifier that can accurately identify an object (eg, abnormal tissue) in an image, and to present the identification basis of the identifier and the quality (numerical value) of the identification basis. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 This is a block diagram showing the functions of the identifier generation device according to the first embodiment.

[0015] Figure 2 It is a diagram showing a hardware configuration example of the identifier generation device according to the first embodiment or the image diagnosis support device according to the second embodiment.

[0016] Figure 3 This is a diagram for explaining an example of the operation of the feature extraction unit.

[0017] Figure 4 This is a diagram for explaining an example of the operation of the feature extraction unit.

[0018] Figure 5 This is a diagram for explaining an example of the operation of the recognition unit.

[0019] Figure 6 This is a diagram for explaining an example of the operation of the recognition unit.

[0020] Figure 7 This is a diagram for explaining an example of the operation of the recognition basis analysis unit.

[0021] Figure 8 This is a diagram for explaining an example of the operation of the automatic cause analysis unit.

[0022] Fig.9A This is a diagram for explaining an example of display of a recognition value (object possibility), recognition / determination result, etc., by an output unit.

[0023] Fig. 9B This is a diagram for explaining an example of display of a recognition value (recognition score), recognition basis, etc. in an output unit.

[0024] Fig. 10A This is a diagram for explaining an example of the operation of the output unit.

[0025] Fig. 10B This is an example of the judgment result showing the performance of the identifier.

[0026] Fig.11 This is a flowchart for explaining the overall operation of the identifier generation device according to the first embodiment.

[0027] Fig.12 This is a block diagram showing the functions of the image diagnosis support device according to the second embodiment.

[0028] Fig.13 This is a flowchart for explaining the overall operation of the image diagnosis support device according to the second embodiment.

[0029] Fig.14 This is a diagram showing a schematic configuration of a remote diagnosis support system equipped with an identifier generation device and an image diagnosis support device.

[0030] Fig.15 This is a diagram showing a schematic configuration of a network entrusted service providing system equipped with an identifier generating device and an image diagnosis supporting device. DETAILED DESCRIPTION

[0031] The embodiments of the present specification provide an identifier generation device and method thereof, which, during machine learning, updates the parameters of the identifier while analyzing the identification basis of the identifier, uses a teaching image and the identification basis to automatically analyze the cause of the identifier's misrecognition, and reflects the analysis results in the production of the identifier through recognition basis learning to update the identifier, thereby achieving the production of an identifier that can identify objects in an image (for example, abnormal tissue, etc.) with high precision.

[0032] Even when the learning image database contains images that do not contribute to the improvement of the recognition accuracy of the recognizer, by adding a structure that automatically analyzes the cause of misrecognition based on the recognition processing of the recognizer and the analysis of the recognition basis, and updating the recognizer by performing recognition basis learning that reflects the analysis results in the creation of the recognizer, it is possible to create a recognizer that can recognize objects in the image with high accuracy. In addition, the recognition basis of the recognizer and the quality (numerical value) of the recognition basis are prompted.

[0033] Hereinafter, the embodiments of the present specification will be described with reference to the accompanying drawings. In the accompanying drawings, functionally identical elements are sometimes shown with the same number. In addition, the accompanying drawings show specific embodiments and installation examples in accordance with the principles of the present invention, but these are drawings for understanding the present invention and are by no means intended to limit the interpretation of the present invention.

[0034] In the present embodiment, a sufficient detailed description is provided to enable those skilled in the art to implement the present invention, but it is also understood that other installations and methods are possible, and that changes in structure and construction and replacement of various elements can be made without departing from the scope and spirit of the technical idea of ​​the present invention. Therefore, the following description should not be interpreted as limited to this.

[0035] Furthermore, as described later, the embodiments of the present specification may be implemented by software running on a general-purpose computer, or may be implemented by dedicated hardware or a combination of software and hardware.

[0036] In the following, each processing in the implementation mode of this specification will be described with "each processing unit as a program (for example, an identification unit, etc.)" as the subject (action subject), but since the processing is determined by the processor (CPU, etc.) executing the program while using a memory and a communication port (communication control device), the processor can also be described as the subject.

[0037] (1) First Embodiment

[0038] <Functional Structure of Identifier Generator>

[0039] Figure 1 The block diagram shows the functional structure of the identifier generation device of the embodiment of the present specification. The identifier generation device 1 includes an input unit 10, a feature extraction unit 11, a recognition unit 12, a recognition basis analysis unit 13, an automatic cause analysis unit 14, a recognition basis learning unit 15, an output unit 16, a recording unit 17, a region segmentation unit 18, a control unit 19, a teaching image (correct answer pattern, etc.) 50, and a memory 90.

[0040] However, in the recognizer generation device 1, when a teaching image exists in advance, the region segmentation unit 18 is not a necessary structure. The recognizer generation device 1 can be installed in an image diagnosis support device in the image acquisition device, or can be installed in a server connected to the image acquisition device via a network as described later (third to fourth embodiments).

[0041] The recognition unit 12 , recognition basis analysis unit 13 , automatic cause analysis unit 14 , recognition basis learning unit 15 , output unit 16 , and recording unit 17 in the identifier generation device 1 may be implemented by a program or by a hardware module.

[0042] The image data is input to the input unit 10. For example, the input unit 10 may also obtain the image data in the image acquisition device ( Figure 1 The input image is a still image data encoded in JPG, Jpeg2000, PNG, BMP format, etc., captured at a predetermined time interval by a built-in camera or other shooting unit (not shown in the figure), and the image is used as the input image.

[0043] The input unit 10 may also extract static image data of frames at a predetermined interval from dynamic image data in MotionJPEG, MPEG, H.264, HD / SDI format, etc., and obtain the image as an input image. The input unit 10 may also use an image obtained by a shooting unit via a bus or a network as an input image. The input unit 10 may obtain an image already stored in a removable recording medium as an input image. The image input from the input unit 10 and the teaching label are output to the feature extraction unit 11.

[0044] The feature extraction unit 11 calculates a feature amount (values ​​of each parameter) and an output value of each layer of a network (for example, a convolutional neural network, etc.) constituting the feature extraction unit 11 based on an input image.

[0045] The recognition unit 12 receives the output value of the final layer of the network of the feature extraction unit 11 and calculates the recognition result of the identifier.

[0046] The recognition basis analyzing unit 13 calculates recognition basis (for example, contribution rate (real number) of each feature amount to recognition result, etc.) using the parameters of the feature extracting unit 11 and the recognition unit 12 and the output value of each layer of the input image.

[0047] When making the machine learning of the identifier, the automatic cause analysis unit 14 uses the recognition basis calculated by the recognition basis analysis unit 13 and the teaching image 50 to analyze the cause of the identifier's misrecognition, calculate the inconsistent parts (improved parts) between the recognition basis and the teaching image, the improved parts of the feature quantity within the network of the identifier, and the quality of the recognition basis (the consistency between the recognition basis and the teaching image).

[0048] The recognition basis learning unit 15 uses the quality of the recognition basis calculated by the automatic cause analysis unit 14, the parameters of the feature extraction unit 11 and the recognition unit 12, the output values ​​of each layer of the network of the recognizer for the input image, and the teaching labels (for example, 0: normal tissue, 1: abnormal tissue, etc.) to perform machine learning in a manner of recognizing objects in the input image as objects, for example, recognizing normal tissues and cells as normal tissues and cells, and recognizing abnormal tissues and cells in the input image as abnormal tissues and cells, and produces a recognizer (recognizing the various parameters of the feature extraction unit 11 and the recognition unit 12 required for recognition (filter coefficients, offset values, etc.)) based on the learning image.

[0049] The output unit 16 displays information such as the quality of the recognition result calculated by the recognition unit 12 , the recognition basis calculated by the recognition basis analysis unit 13 , and the recognition basis calculated by the automatic cause analysis unit 14 on an output device (display, etc.).

[0050] The recording unit 17 stores information such as values ​​calculated by each unit from the feature extraction unit 11 to the identification basis learning unit 15 in the memory 90 .

[0051] When a teaching image exists in advance, the area dividing unit 18 is not used. When a teaching image does not exist, the area dividing unit 18 divides the area in the target image and creates a teaching image (correct answer pattern).

[0052] The control unit 19 is realized by a processor and is connected to each element in the identifier generating device 1. The operation of each element of the identifier generating device 1 is performed by the autonomous operation of each component described above or by the instruction of the control unit 19.

[0053] Thus, in the identifier generation device 1 of the present embodiment, during machine learning, the feature extraction unit 11 and the recognition unit 12 use the learning image to update the parameters of the identifier, and the recognition basis analysis unit 13 uses the parameters of the feature extraction unit 11 and the recognition unit 12 and the output values ​​of each layer of the input image to analyze the recognition basis of the identifier. The automatic cause analysis unit 14 uses the teaching image and the recognition basis to automatically analyze the cause of the misrecognition of the identifier, and the recognition basis learning unit 15 performs recognition basis learning to reflect the analysis results in the production of the identifier to update the identifier. Thus, it is characterized in that a recognizer that can recognize objects (e.g., abnormal tissue, etc.) in an image with high accuracy is produced.

[0054] <Hardware Structure of Identifier Generator>

[0055] Figure 2 : is a diagram showing an example of the hardware structure of the identifier generation device 1 of the embodiment of this specification. The identifier generation device 1 includes a CPU (processor) 201 that executes various programs, a memory (main storage device) 202 that stores various programs, and an auxiliary storage device 203 (equivalent to the memory 90) that stores various data. The identifier generation device 1 also includes: an output device 204, which is used to output the recognition result, the recognition basis, the quality of the recognition basis, etc.; an input device 205, which is used for the user to input instructions, images, etc.; and a communication device 206, which is used to communicate with other devices. They are connected to each other via a bus 207.

[0056] The CPU 201 reads and executes various programs from the memory 202 as needed. The memory 202 stores the input unit 10, the feature extraction unit 11, the recognition unit 12, the recognition basis analysis unit 13, the automatic cause analysis unit 14, the recognition basis learning unit 15, the output unit 16, the recording unit 17, and the area segmentation unit 18 as programs.

[0057] The auxiliary storage device 203 stores the learning image, the evaluation image, the teaching image, the various parameters of the identifier generated by the feature extraction unit 11 and the recognition unit 12, the recognition basis (the contribution rate of each feature quantity to the recognition result (real number)) generated by the recognition basis analysis unit 13, the inconsistent parts (improvement parts) between the recognition basis and the teaching image generated by the automatic cause analysis unit 14 and the improved parts of the feature quantities in the network of the identifier and the quality of the recognition basis (the consistency between the recognition basis and the teaching image), the identifier updated by the recognition basis learning unit 15 (the various parameters of the feature extraction unit 11 and the recognition unit 12 required for recognition (filter coefficients, offset values, etc.)), the teaching image generated by the area segmentation unit 18, and other display information generated by the output unit 16.

[0058] The output device 204 is composed of devices such as a display, a printer, and a speaker. For example, the output device 204 displays the data generated by the output unit 16 on the display screen.

[0059] The input device 205 is composed of devices such as a keyboard, a mouse, and a microphone. The input device 205 inputs the user's instructions (including the decision to input each image such as a teaching image) into the identifier generation device 1 .

[0060] The communication device 206 is not an essential component of the identifier generation device 1. When a communication device is included in a personal computer or the like connected to the image acquisition device, the identifier generation device 1 may not have the communication device 206. The communication device 206 receives data (including images, identifiers, etc.) sent from other devices (e.g., servers) connected via a network and stores the data in the auxiliary storage device 203.

[0061] The identifier generation device of the present specification analyzes the identification basis of the identifier while updating the parameters of the identifier, uses the teaching image and the identification basis to automatically analyze the cause of the identifier's misrecognition, and reflects the analysis results in the identification basis learning for the production of the identifier to update the identifier, thereby producing an identifier that can identify objects in the image (for example, abnormal tissue, etc.) with high accuracy.

[0062] <Structure and operation of each part>

[0063] The structure and operation of each element will be described in detail below.

[0064] (i) Feature Extraction Unit 11

[0065] The feature extraction unit 11 obtains the feature quantity of the input image. As an example, Figure 3 An example of obtaining a feature quantity is shown. Figure 3CNN represents a convolutional neural network. For example, the feature extraction unit 11 uses a feature extractor FEA that performs the calculation of equation (1) to obtain a feature amount FAi of an object (eg, tissue, cell, etc.) in the input image A1.

[0066] The filter coefficient wj shown in formula (1) is a coefficient obtained through machine learning, etc., so that objects other than the objects to be detected are identified as objects other than the objects to be detected (for example, normal tissues and normal cells are identified as normal tissues and normal cells, etc.), and objects to be detected are identified as objects to be detected (for example, abnormal tissues and abnormal cells are identified as abnormal tissues and abnormal cells).

[0067] In formula (1), pj represents the pixel value, bi represents the offset value, m represents the number of filter coefficients, and h represents the nonlinear function. Figure 4 As shown, the feature extraction unit 11 uses formula (1) to obtain the calculation results of each filter 42 from the upper left to the lower right of the target image 41, thereby obtaining the feature quantity fi of any filter i. For example, the matrix of the feature quantity fi obtained by the feature extractor FEA is set as the feature quantity FAi of the input image A1. The method for making the feature extractor FEA will be described by the recognition unit 12 described later.

[0068] [Formula 1]

[0069]

[0070] (ii) Identification Unit 12

[0071] like Figure 5 As shown, the recognition unit 12 uses the feature quantity FAi (matrix f) of the feature extractor FEA obtained by the feature extraction unit 11 to calculate the value of the probability of the object to be detected (abnormal tissue probability, lesion probability, etc.) by the equation (2) through the logistic regression process 51. Based on the calculated value, the recognition unit 12 determines whether the object (e.g., tissue, cell) in the input image A1 is the object to be detected (abnormal tissue, etc.) or an object other than the object to be detected (normal tissue, etc.).

[0072] In formula (2), w represents a weight matrix, b represents an offset value, g represents a nonlinear function, and y represents a recognition result. As described later, the recognition unit 12 uses the learning image to obtain the weight of w and the offset value of b through machine learning. In addition, the weight of w and the offset value of b obtained during learning are used to recognize the target image and calculate the recognition result y.

[0073] [Formula 2]

[0074] y=g(w×f+b)…(2)

[0075] The recognition unit 12 uses formula (2) to determine that the object in the input image is an object other than the object to be detected (for example, the tissue or cell is a normal tissue or a normal cell, etc.), for example, through logistic regression processing 51, and uses the feature extraction unit 11 to learn the feature value of the object (for example, tissue or cell, etc.) through well-known machine learning techniques.

[0076] In addition, if the object in the input image is an object to be detected (e.g., tissue or cell is abnormal tissue or cell, etc.), the recognition unit 12 uses the feature extraction unit 11 to learn the feature quantity of the object (e.g., tissue or cell, etc.) by using the logistic regression processing 51 to determine that the object is an object to be detected (e.g., abnormal tissue or cell, etc.). For example, a convolutional neural network can also be used as a machine learning technology.

[0077] like Figure 6 As shown, when learning, the recognition unit 12 uses the input image A1 (for example, HE-stained image) through machine learning to judge the object to be detected as the object to be detected (for example, abnormal tissue and abnormal cells are judged as abnormal tissue and abnormal cells, etc.), and judge the object other than the object to be detected as the object other than the object to be detected (for example, normal tissue and normal cells are judged as normal tissue and normal cells, etc.), and uses the feature extraction unit 11 to generate a feature extractor FEA for calculating the feature value fi (set to FAi) of the input image A1 through equations (1) and (2).

[0078] When learning, the recognition unit 12 repeatedly performs recognition basis learning by the feature extraction unit 11, the recognition unit 12, and the recognition basis learning unit 15 described later using a plurality of learning images, obtains the weight w, filter coefficient wj, offset values ​​b and bi shown in equations (1) and (2), and uses the feature extraction unit 11 to create a feature extractor FEA that calculates the feature amount FAi of the input image A1 from the input image A1. In addition, the recognition unit 12 calculates the recognition result of the target image using the feature extractor FEA.

[0079] The identification unit 12 stores the obtained weights w, filter coefficients wj, and offset values ​​b and bi in the memory 90 .

[0080] (iii) Identification basis analysis unit 13

[0081] Using the result z before applying the nonlinear function h in equation (1), the identification analysis unit 13 calculates by equation (3) and equation (4): Figure 7The identification basis shown (the contribution rate (real number) of each feature value to the identification result).

[0082] exist Figure 7 In the example shown, the input image 71 includes an abnormal tissue region 73 and a normal tissue region 72. Regions other than region 73 and region 72 are also normal tissue regions. Identification basis 75 indicates an area of ​​abnormal tissue identification basis and an area of ​​normal tissue identification basis. Figure 7 In the example of FIG. 7 , the region based on the identification of abnormal tissue is indicated in black, and the region based on the identification of normal tissue is indicated in white or a dot pattern. Region 77 is the region based on the identification of abnormal tissue included in region 73 of abnormal tissue. Region 78 is the region based on the identification of normal tissue included in region 73 of abnormal tissue. Region 79 is the region based on the identification of abnormal tissue included in region 72 of normal tissue.

[0083] In formula (3), P is the input data from the previous layer composed of pj, L is the index of the layer to be calculated as the basis for identification, and C L Indicates the identification basis of the Lth layer, C L+1 Indicates the identification basis of the L+1th layer. In addition, W -1 is a matrix that replaces the arbitrary dimension of the matrix of the filter coefficients wj in equation (1) and reverses the direction of the arbitrary dimension. -1 This is a matrix in which the first dimension (number of feature quantities) and the second dimension (number of channels) of the matrix of the filter coefficient wj in equation (1) are swapped, and the directions of the third dimension (vertical dimension) and the fourth dimension (horizontal dimension) are reversed.

[0084] [Formula 3]

[0085]

[0086] In addition, the identification basis of the layer that obtains the maximum value among the layers of the identifier (for example, the Max Pooling layer, etc.) is set to 1 only for the pixels where ri of each region becomes the maximum value, and the others are set to 0. Therefore, for example, the identification basis is propagated only to the pixels whose identification basis input from the adjacent layer on the output layer side is 1.

[0087] In addition, if the input data of the final layer of the recognizer (such as the Logistic Regression layer, etc.) is set to P LR , the label is set as a teaching signal in the form of a one hot vector, then equation (4) is used to calculate the recognition basis and back-propagate to the previous layer of the recognizer.

[0088] [Formula 4]

[0089]

[0090] (iv) Automatic cause analysis unit 14

[0091] like Figure 8 As shown, the automatic cause analysis unit 14 automatically analyzes the area that is the cause of the misrecognition using the recognition basis calculated by the recognition basis analysis unit 13 and the manually generated teaching image or the teaching image generated by the area segmentation unit 18. The teaching image (correct answer pattern) 81 is a monochrome image of the same size as the input image 71, for example, taking a value of 0 or 1. For example, the area where the teaching image is 0 represents an area other than the object area (for example, a normal tissue area, etc.), and the area where 1 represents the object area (for example, an abnormal tissue area, etc.).

[0092] It is expected that the area outside the object area is identified based on the feature quantity in the area outside the object area, and it is not expected to use the object area as the basis for identification. In addition, it is expected that the object area (e.g., abnormal tissue area, etc.) is identified based on the feature quantity in the object area (e.g., abnormal tissue area, etc.), and it is not expected to use the area outside the object area (e.g., normal tissue area, etc.) as the basis for identification.

[0093] Therefore, the automatic cause analysis unit 14 compares the recognition basis 75 calculated by the recognition basis analysis unit 13 with the area outside the object area and the object area of ​​the teaching image (correct answer pattern) 81, and determines the recognition basis appearing in the inappropriate area as the cause of misrecognition. In addition, the automatic cause analysis unit 14 analyzes the cause of misrecognition by dividing it into the object area and the area outside the object area. Figure 8 In the example of FIG. 1 , the part 82 indicates a part in the abnormal tissue region where the basis for identifying the abnormal tissue region is insufficient, and the part 83 indicates a part in the normal tissue region where the basis for identifying the abnormal tissue region is sufficient.

[0094] The automatic cause analysis unit 14 calculates the misrecognition cause score e for the area other than the object area using equations (5) and (6): back and the misidentification cause fraction e for the object region fore Among them, C sum The recognition basis C for the object region in the input image calculated by the recognition basis analysis unit 13, and the mask represents the teaching image. However, the recognition basis C other than the object region may be used. sum However, in this case, the area where the teaching image is 0 represents the object area (for example, abnormal tissue area, etc.), and the area where 1 represents the area other than the object area (for example, normal tissue area, etc.).

[0095] [Formula 5]

[0096] e back =Csum ×(1-mask)…(5)

[0097] [Formula 6]

[0098] e fore =-C sum ×mask…(6)

[0099] There is recognition basis C in the area outside the object area sum In this case, a feature quantity that is not suitable for recognition may be used as a basis for recognition, which may cause the accuracy of the recognition device to decrease. Therefore, the automatic cause analysis unit 14 extracts the recognition basis C included in the area other than the object area. sum , calculate the fraction e of misidentification causes in areas outside the object area back The more positive identification evidence is contained in the area outside the object area, the more e back The larger the value.

[0100] It is preferred that the object region contains a feature quantity for identifying the object region, and the object region contains more positive identification basis C sum . Recognition in the object area is based on C sum If the value is insufficient, it may cause the object to be missed. Therefore, the automatic cause analysis unit 14 uses the recognition basis C included in the object area. sum The negative number is used to calculate the fraction e of the misidentification reason of the object area fore .

[0101] The more negative identification evidence and the smaller the value of identification evidence, the greater the proportion of e fore The larger the value, the greater the proportion of positive identification evidence. fore In addition, the automatic cause analysis unit 14 can also use equations (7) and (8) to calculate the quality of the recognition basis outside the object area (the consistency between the recognition basis and the teaching image) coin back , the quality of the object area is based on the identification fore . Where X represents the horizontal pixel number of the object image, and Y represents the vertical pixel number of the object image.

[0102] [Formula 7]

[0103]

[0104] [Formula 8]

[0105]

[0106] Using a fixed number of evaluation images, the coin shown in formula (9) is calculated for each recognizer all, by comparing coins all , to determine the performance of the recognizer. For example, coin all The larger the value of is, the better the performance of the identifier is.

[0107] [Formula 9]

[0108] coin all =coit back +coin fore …(9)

[0109] (v) Identification basis learning unit 15

[0110] The recognition basis learning unit 15 uses the score of the misrecognition cause calculated by the automatic cause analysis unit 14 and the recognition result y calculated by the recognition unit 12 to perform recognition basis learning for correcting the recognition basis of the area estimated to be the misrecognition cause. In the recognition basis learning, multiple costs are used. For example, three costs can be used. The recognition basis learning unit 15 first uses the Negative Log Likelihood (nll) of formula (10) used in ordinary machine learning, and second uses the cost cgl related to the misrecognition cause in the area outside the object area shown in formula (11) back , and the third one uses the cost cgl associated with the cause of misidentification within the object region as shown in formula (12) fore . In formula (10), label is a teaching label representing the area outside the object area and the object area (for example, 0 for the area outside the object and 1 for the object area).

[0111] [Formula 10]

[0112] nll=-{label×ln(y)}…(10)

[0113] [Formula 11]

[0114]

[0115] [Formula 12]

[0116]

[0117] The score e of the misrecognition cause calculated by the automatic cause analysis unit 14 back The recognition basis learning unit 15 calculates the e calculated by the automatic cause analysis unit 14. back The average value of is set as cost cgl back , so that e backThe value of is reduced and machine learning is performed to suppress the recognition basis in areas other than the object area. In addition, the score of the cause of misrecognition e fore = represents an area with a low value of recognition basis in the object area. Therefore, the recognition basis learning unit 15 calculates the value of e calculated by the automatic cause analysis unit 14. fore The average value of is set as cost cgl fore , to increase the recognition basis within the object area.

[0118] As shown in formula (13), the recognition basis learning unit 15 sets the sum of the three costs as the comprehensive cost, and performs machine learning in a manner to reduce the cost. It corrects the recognition basis of the area of ​​the cause of misrecognition obtained by the automatic cause analysis unit 14, and updates the parameters of the feature extraction unit 11 and the recognition unit 12 to produce an identifier that can recognize objects with high precision.

[0119] [Formula 13]

[0120] cost = nll + cgl back +cgl fore …(13)

[0121] (vi) Output unit 16

[0122] As an example, the output unit 16 uses the recognition result y obtained by the produced identifier to generate Fig.9A , Fig. 9B The GUI (Graphical User Interface) shown displays the result of object possibility determination (eg, lesion possibility determination), the recognition basis, and the quality of the recognition basis.

[0123] Fig.9A The screen shown is displayed, for example, when the identifier is used. Fig.9A An example of the case of breast is shown in the diagram showing the classification results of non-tumor and tumor (abnormal tissue, abnormal cells, etc.). Fig.9A The screen example shows a classification result 91, an object possibility (tumor possibility) 92, an image display button 93, and an identification reason display button 94. When the user presses the image display button 93, for example, the input image is displayed. When the user presses the identification reason display button 94, the identification reason is displayed.

[0124] exist Fig.9A In the example of , the recognition unit 12 classifies the input target image as containing a tumor as abnormal tissue / cells for the breast, and calculates the object possibility value of the tumor as 0.89.

[0125] Fig. 9B For example, it is displayed when the recognizer is being trained or used. Fig. 9BThis is an example of displaying an input image, a recognition basis for the input image, and the quality of the recognition basis. Fig. 9B An example is shown in which the quality of the non-tumor identification basis is calculated as 0.96 and the quality of the tumor identification basis is calculated as 0.93. Fig. 9B The screen can be based on Fig.9A The image display button 93 and the recognition reason display button 94 shown in the figure can be displayed by operating the image display button 93 and the recognition reason display button 94. Fig.9A The screen is displayed independently.

[0126] In addition, when the output unit 16 receives an unknown image from the input unit 10 and displays the recognition results of each recognizer, if an object in the image is determined to be an object (for example, abnormal tissue, abnormal cell), the output unit 16 displays the recognition results of each recognizer. Fig. 10A As shown in FIG. 1 , in order to indicate the part of the object to be detected (e.g., the part suspected of abnormal tissue or abnormal cells, etc.), a detection frame 102 may be drawn in the input object image 101. On the other hand, when it is determined to be normal tissue or normal cells, the detection frame 102 may not be drawn on the input object image 101, and the input object image 101 may be directly displayed. Fig. 10A As shown, the result of the determined object possibility (eg, tumor) is displayed.

[0127] In addition, as an example, Fig. 10B As shown, the output unit 16 uses the calculated performance value coin of each identifier all , determine the performance of each identifier, and display the determination result of the performance of the identifier. Fig. 10B For example, it is displayed when the recognizer is being trained or used. Fig. 10B This is an example in which the performance value of classifier A is set to 1.97, the performance value of classifier B is set to 1.65, and the performance determination result of the classifier is represented as classifier A.

[0128] The output unit 16 is not an essential component of the identifier generating device 1 . When the image diagnosis supporting device includes the output unit, the identifier generating device 1 may not include the output unit 16 .

[0129] (vii) Recording unit 17

[0130] The recording unit 17 stores the input image (learning image, evaluation image, etc.), the teaching image, the feature extraction unit 11, the parameters of the identifier generated by the recognition unit 12, the recognition result and the recognition value, and the recognition basis generated by the recognition basis analysis unit 13 (the contribution rate (real number) of each feature quantity to the recognition result) in the memory 90. The recording unit 17 also stores the inconsistency points (improvement points) between the recognition basis generated by the automatic cause analysis unit 14 and the teaching image, the improvement points of the feature quantity in the network of the identifier, and the quality of the recognition basis (the consistency between the recognition basis and the teaching image) in the memory 90. In addition, the recording unit 17 stores the identifier updated by the recognition basis learning unit 15 (the parameters of the feature extraction unit 11 and the recognition unit 12 required for recognition (filter coefficients, weights, offset values, etc.)), the teaching image generated by the area segmentation unit 18, the display information generated by the output unit 16, etc. in the memory 90.

[0131] The region segmentation unit 18 divides each pixel in the input image into regions (eg, object region, non-object region, etc.) using a region segmentation method such as U-Net, and creates a teaching image.

[0132] <Processing Order of Identifier Generator>

[0133] Fig.11 This is a flowchart for explaining the operation of the identifier generation device 1 according to the embodiment of the present specification. In the following, each processing unit (input unit 10, feature extraction unit 11, etc.) is described as an action subject, but the CPU 201 may be used as an action subject and replaced with each processing unit executed as a program by the CPU 201.

[0134] (i) Step 1101

[0135] The input unit 10 receives a learning image and outputs the input image to the feature extraction unit 11 .

[0136] (ii) Step 1102

[0137] The feature extraction unit 11 obtains the feature quantity FAi of the object (such as tissue, cell, etc.) in the input image A1 by using a filter and creates a feature extractor FEA through machine learning and the above-mentioned formula 1. The feature extraction unit 11 obtains the filter coefficient wj and the offset value bi for the feature quantity FAi.

[0138] (iii) Step 1103

[0139] The recognition unit 12 obtains the recognition result based on the feature quantity FAi through machine learning using Formula 2, calculates the recognition value of the object possibility (for example, the possibility of a lesion, etc.), and obtains the parameters (weight w, offset value b, etc.) of Formula 2 used to obtain the recognition value, so as to determine whether the object in the input image A1 is an object to be detected (for example, a normal cell, or an abnormal cell, etc.).

[0140] (iv) Step 1104

[0141] The recognition basis analyzing unit 13 obtains the recognition basis of the target image using equations 3 and 4.

[0142] (v) Step 1105

[0143] The automatic cause analysis unit 14 calculates the misrecognition cause score e by using the recognition basis calculated by the recognition basis analysis unit 13 and the teaching image through equations 5 and 6. back and e fore , the quality coin of the recognition basis is obtained through equations 7 and 8 back and coin fore .

[0144] (vi) Step 1106

[0145] The recognition based learning unit 15 calculates the comprehensive cost cost of equation 12 through equations 9, 10, and 11, and uses machine learning to update the filter coefficients wj, weights w, and offsets bi, b in a way that reduces the cost, thereby producing an identifier.

[0146] (vii) Step 1107

[0147] The recording unit 17 stores the parameters of Expressions 1 and 2 (filter coefficients wj, weights w, offset values ​​bi, b, etc.), the identification basis, and the quality of the identification basis in the memory 90 .

[0148] According to the first embodiment, during machine learning, the recognition basis of the recognizer is analyzed while the parameters of the recognizer are updated, and the teaching image and the recognition basis are used to automatically analyze the cause of the recognizer's misrecognition. The recognition basis learning is then performed to reflect the analysis result in the production of the recognizer to update the recognizer, thereby making it possible to produce a recognizer that can recognize objects in an image (for example, abnormal tissue, etc.) with high precision.

[0149] In addition, it is possible to present a recognition basis indicating the reason for recognition of the target image and a quality of the recognition basis indicating the accuracy of the recognition device.

[0150] (2) Second Embodiment

[0151] like Fig.12As shown, the image diagnosis support device 2 of the second embodiment includes many features similar to those of the first embodiment. Figure 1 The structure of the identifier generation device 1 is the same as that of the first embodiment, but unlike the first embodiment, it does not include the recognition basis learning unit 15, but includes a region segmentation unit 18, a feature extraction unit 21, a recognition unit 22, a recognition basis analysis unit 23, and an automatic cause analysis unit 24. Figure 1 Different structures are described.

[0152] The image diagnosis support device 2 of one embodiment of the present specification reads various parameters of the recognizer from the memory, and the region segmentation unit 18 creates a teaching image of the evaluation image by region segmentation. Furthermore, the recognition basis analysis unit 23 analyzes the recognition basis of the evaluation image. The automatic cause analysis unit 24 uses the teaching image and the recognition basis to automatically analyze the cause of misrecognition of the recognizer and analyze the quality of the recognition basis of the evaluation image, thereby prompting the recognition result of the object (such as tissue, cell, etc.) in the evaluation image, the recognition basis, and the quality of the recognition basis.

[0153] <Structure and operation of each part>

[0154] The following is about Figure 1 The structure and operation of each element are described in detail.

[0155] (i) Region division unit 18

[0156] The region segmentation unit 18 performs region segmentation on the evaluation image using a segmentation model (eg, U-Net) that segments the evaluation image into an object region and a region other than the object region, and creates a teaching image.

[0157] (ii) Feature Extraction Unit 21

[0158] The feature extraction unit 21 reads each parameter of the learned feature extractor from the memory 90 , and calculates the feature amount FAi of the evaluation image according to Formula 1 using the feature extractor.

[0159] (iii) Identification Unit 22

[0160] The recognition unit 22 reads each parameter of the learned classifier from the memory 90, uses the classifier to recognize the evaluation image based on the feature amount FAi according to Formula 2, and calculates the recognition result y and the recognition value.

[0161] (iv) Identification basis analysis unit 23

[0162] The recognition basis analyzing unit 23 obtains the recognition basis of the evaluation image using equations 3 and 4.

[0163] (v) Automatic cause analysis unit 24

[0164] The automatic cause analysis unit 24 calculates the inconsistent parts (improvement parts) between the recognition basis and the teaching image, the misrecognition cause score and the quality of the recognition basis (the consistency between the recognition basis and the teaching image) through Formula 5 and Formula 6, using the recognition basis calculated by the recognition basis analysis unit 23 and the teaching image (correct pattern) of the evaluation image obtained by the area segmentation unit 18.

[0165] <Hardware Structure of Image Diagnosis Support Device>

[0166] The hardware configuration example of the image diagnosis support device 2 according to the embodiment of the present specification has the following Figure 2 The structure is the same as that of the identifier generation device 1 of the first embodiment, but different from that of the first embodiment, the memory 202 stores a region division unit 18 , a feature extraction unit 21 , a recognition unit 22 , a recognition basis analysis unit 23 , and an automatic cause analysis unit 24 .

[0167] The auxiliary storage device 203 of the image diagnosis supporting device 2 stores the recognition result y obtained by the recognition unit 22 , the recognition basis obtained by the recognition basis analyzing unit 23 , the quality of the recognition basis obtained by the automatic cause analyzing unit 24 , and the like.

[0168] <Processing sequence of image diagnosis support device>

[0169] Fig.13 This is a flowchart for explaining the operation of the image diagnosis support device 2 according to the embodiment of the present specification. In the following, each processing unit (input unit 10, feature extraction unit 21, etc.) is described as an action subject, but the CPU 201 may be used as the action subject and replaced with each processing unit executed as a program by the CPU 201.

[0170] (i) Step 1301

[0171] The input unit 10 receives an evaluation image and outputs the input image to the feature extraction unit 21 .

[0172] (ii) Step 1302

[0173] The feature extraction unit 21 reads the filter coefficients wj and offset values ​​bi of the learned feature extractor FEA from the memory 90 and uses the filter to obtain the feature amount FAi of the object (eg, tissue, cell, etc.) in the input image A1 according to the above equation 1.

[0174] (iii) Step 1303

[0175] The recognition unit 22 reads the parameters of the learned recognizer (weight w, offset value b, etc.) from the memory 90, obtains the recognition result based on the feature value FAi through Formula 2, calculates the recognition value of the object possibility (for example, the possibility of lesions, etc.), and determines whether the object in the input image A1 is an object to be detected (for example, a normal cell, or an abnormal cell, etc.).

[0176] (iv) Step 1304

[0177] The recognition basis analyzing unit 23 obtains the recognition basis of the target image using equations 3 and 4.

[0178] (v) Step 1305

[0179] The automatic cause analysis unit 24 calculates the misrecognition cause score e by using the recognition basis calculated by the recognition basis analysis unit 23 and the teaching image of the target image obtained by the area segmentation unit 18 through equations 5 and 6. back and e fore , the quality coin of the recognition basis is obtained through equations 7 and 8 back and coin fore .

[0180] (vi) Step 1306

[0181] The recording unit 17 stores the recognition result, the recognition value, the recognition basis, and the quality of the recognition basis in the memory 90 .

[0182] According to the second embodiment, various parameters of the identifier are read from the memory, and a teaching image of the evaluation image is produced by region segmentation. Furthermore, while analyzing the recognition basis of the evaluation image, the teaching image and the recognition basis are used to analyze the quality of the recognition basis of the evaluation image based on the identifier, thereby being able to prompt the recognition results, recognition basis, and quality of the recognition basis of the object (such as tissue, cell, etc.) in the evaluation image.

[0183] In addition, by changing the recognizer to another recognizer of another facility, etc., the recognition basis of the evaluation image and the quality of the recognition basis are displayed, and the performance of the other manufactured recognizer can be determined. In addition, by switching multiple recognizers to display the recognition basis of the evaluation image and the quality of the recognition basis, it is possible to determine which recognizer can recognize the object in the evaluation image with high accuracy.

[0184] (3) Third Embodiment

[0185] Fig.14 14 is a functional block diagram showing the configuration of a remote diagnosis support system 1400 according to a third embodiment of the present specification. The remote diagnosis support system 1400 includes a server 1403 and an image acquisition device 1405. The server 1403 can be formed of, for example, one or more computers.

[0186] The image acquisition device 1405 is a device such as a personal computer equipped with a virtual slide or a camera, and includes a photographing unit 1401 for photographing image data and a display unit 1404 for displaying the recognition / determination result transmitted from the server 1403. Although not shown in the figure, the image acquisition device 1405 includes a communication device for transmitting image data to the server 1403 or receiving data transmitted from the server 1403.

[0187] The server 1403 includes: an image diagnosis support device 2 that uses the identifier generation device 1 of the first embodiment of the present specification to create an identifier for image data (input image, teaching image) transmitted from the image acquisition device 1405, and performs image processing using the created identifier; and a storage unit 1402 that stores the recognition / determination results, recognition basis, and quality of recognition basis output from the image diagnosis support device 2. Although not shown in the figure, the server 1403 includes a communication device that receives image data transmitted from the image acquisition device 1405 or transmits data such as recognition / determination results, recognition basis, and quality of recognition basis to the image acquisition device 1405.

[0188] The image diagnosis support device 2 uses the identifier obtained by the identifier generation device 1 to classify the objects (e.g., tissues, cells, etc.) in the image data captured by the imaging unit 1401 as to whether there are objects to be detected (e.g., abnormal tissues, abnormal cells (e.g., cancer), etc.). The display unit 1404 displays the data such as the recognition / determination results, the recognition basis, and the quality of the recognition basis transmitted from the server 1403 on the display screen of the image acquisition device 1405.

[0189] As the image acquisition device 1405, a regenerative medical device or an iPS cell culture device having an imaging unit, or an MRI or ultrasonic imaging device, etc. may be used.

[0190] According to the third embodiment, for objects (such as tissues, cells, etc.) in images transmitted from facilities at different locations, the parameters of the identifier obtained by the identifier generating device 1 are used to accurately classify whether the objects are objects to be detected (abnormal tissues, abnormal cells, etc.), and the recognition / judgment results, recognition basis, quality of the recognition basis, and other data are transmitted to facilities at different locations, and the recognition / judgment results, recognition basis, quality of the recognition basis, and other data are displayed on the display unit of the image acquisition device located at the facility, thereby providing a remote diagnosis support system.

[0191] (4) Fourth Embodiment

[0192] Fig.15This is a functional block diagram showing the structure of a network hosting service providing system 1500 according to the fourth embodiment of this specification. The network hosting service providing system 1500 includes a server 1503 and an image acquisition device 1505. The server 1503 can be composed of, for example, one or more computers.

[0193] The image acquisition device 1505 is, for example, a device such as a personal computer equipped with a virtual slide device or a camera. The image acquisition device 1505 includes a photographing unit 1501 for photographing image data, a storage unit 1504 for storing the identifier transmitted from the server 1503, and an image diagnosis support device 2. The image diagnosis support device 2 reads the identifier transmitted from the server 1503, and uses the identifier obtained by the identifier generation device 1 of the first embodiment of this specification to classify whether the object (e.g., tissue, cell, etc.) in the image newly photographed by the photographing unit 1501 of the image acquisition device 1505 is an object to be detected (e.g., abnormal tissue, abnormal cell, etc.). In addition, although not shown in the figure, the image acquisition device 1505 has a communication device for transmitting image data to the server 1503 or receiving data transmitted from the server 1503.

[0194] The server 1503 has a storage unit 1502 that creates an identifier from the identifier generation device 1 of the first embodiment of this specification for the image data transmitted from the image acquisition device 1505, and stores the identifier output from the identifier generation device 1. In addition, the server 1503 receives the input and designation of the teaching image from the operator, or generates the teaching image using the area segmentation unit 18. In addition, the server 1503 presents the operator with the outputs of the identifier generation device 1 (recognition / determination results, recognition basis, and the quality of the recognition basis, etc.). In addition, although not shown in the figure, the server 1503 has a communication device that receives the image data sent from the image acquisition device 1505 or sends the identifier to the image acquisition device 1505.

[0195] In addition, the identifier generation device 1 performs machine learning on objects (such as tissues, cells, etc.) in the image data captured by the imaging unit 1501 to determine objects to be detected (for example, normal tissues and cells are determined as normal tissues and cells, and abnormal tissues and cells are determined as abnormal tissues and cells, etc.), and creates an identifier that calculates the feature value of objects (such as tissues, cells, etc.) in images of facilities at different locations. The storage unit 1504 stores the identifiers and the like transmitted from the server 1503.

[0196] The image diagnosis assisting device 2 in the image acquisition device 1505 reads an identifier etc. from the storage unit 1504, and uses the identifier to classify whether an object (e.g., tissue, cell, etc.) in an image newly captured by the capturing unit 1501 of the image acquisition device 1505 is an object to be detected (e.g., abnormal tissue, abnormal cell, etc.), and displays data such as recognition / judgment results, recognition basis / quality of recognition basis on the display screen of the output device 204 of the image diagnosis assisting device 2.

[0197] As the image acquisition device 1505, a regenerative medical device or an iPS cell culture device having an imaging unit, or an MRI or ultrasonic imaging device, etc. may be used.

[0198] According to the fourth embodiment, machine learning is performed on objects (such as tissues, cells, etc.) in images transmitted from facilities at different locations to classify them as objects that should be detected (for example, normal tissues and cells are classified as normal tissues and cells, and abnormal tissues and cells are classified as abnormal tissues and cells, etc.) to produce identifiers, which are transmitted to facilities at different locations, are read into the identifiers by an image acquisition device located at the facilities, and objects (such as tissues, cells, etc.) in newly captured images are classified as objects that should be detected (for example, abnormal tissues, abnormal cells, etc.), thereby providing a network entrusted service provision system.

[0199] The above-described embodiments can be modified as follows. In the feature extraction unit 11, multiple feature quantities are obtained by using a filter through machine learning, but other feature quantities such as HOG can also be used to achieve the same effect. In addition, in the recognition unit 12, negative log likelihood is used as a loss function (cost), but square error, Hinge loss, etc. can also be used to achieve the same effect.

[0200] The present invention can also be implemented by the program code of the software that realizes the functions of the implementation mode. In this case, a storage medium having the program code recorded thereon is provided to a system or device, and the computer (or CPU, MPU) of the system or device reads the program code stored in the storage medium. In this case, the program code itself read out from the storage medium realizes the functions of the above-mentioned implementation mode, and the program code itself and the storage medium having the program code stored thereon constitute the present invention. As a storage medium for providing such a program code, for example, a floppy disk, a CD-ROM, a DVD-ROM, a hard disk, an optical disk, a magneto-optical disk, a CD-R, a magnetic tape, a non-volatile memory card, a ROM, etc. are used.

[0201] In addition, based on the instructions of the program code, the OS (operating system) running on the computer may perform part or all of the actual processing, and the functions of the above-mentioned embodiments may be realized through the processing. In addition, after the program code read from the storage medium is written into the memory on the computer, based on the instructions of the program code, the CPU of the computer may perform part or all of the actual processing, and the functions of the above-mentioned embodiments may be realized through the processing.

[0202] Moreover, the program code of the software that implements the functions of the implementation method can be released via the network and stored in a storage unit such as a hard disk or memory of the system or device or a storage medium such as CD-RW, CD-R, etc. When in use, the computer (or CPU, MPU) of the system or device reads and executes the program code stored in the storage unit or the storage medium.

[0203] Finally, the processes and techniques described herein are not inherently related to any particular device and can be installed by any corresponding combination of components. In addition, various types of general-purpose devices can be used according to the methods described herein. In order to perform the steps of the methods described herein, it is sometimes beneficial to build a dedicated device. In addition, various inventions can be formed by appropriate combinations of multiple constituent elements disclosed in the embodiments. For example, several constituent elements can also be deleted from all the constituent elements shown in the embodiments. Moreover, the constituent elements of different embodiments can also be appropriately combined. The present invention is described in association with specific examples, but these are not intended to be limited in all viewpoints but to be illustrated. It is known that for those skilled in the art, it is known that there are multiple combinations of hardware, software and firmware corresponding to the implementation of the present invention. For example, the described software can be implemented by a wide range of programs or scripting languages ​​such as assembly language, C / C++, perl, Shell, PHP, Java, etc.

[0204] Furthermore, in the above-mentioned embodiments, the control lines and information lines are those considered necessary for the description, and do not necessarily represent all the control lines and information lines in the product. All the structures may be connected to each other.

[0205] In addition, for those with ordinary knowledge in the technical field, other implementations of the present invention will be apparent from consideration of the description and embodiments of the present invention disclosed herein. The various modes and / or components of the described embodiments may be used alone or in any combination.

Claims

1. A recognizer generating device, characterized in that: The identifier generating device comprises: a processor that executes a program for performing image processing on an image of a subject; and a memory for storing the program and the result of the image processing, The memory stores the object image and the identifier, The processor performs the following processing: extracting a feature quantity of the object image from the object image using the identifier; identifying the object image from the feature amount using the identifier to determine an identification value; Determining identification basis using the feature quantity and the identification value; Determine the improvement area using the recognition basis and the teaching image; and The identifier is updated based on the improved areas.

2. The identifier generating device according to claim 1, characterized in that The processor analyzes a cause of misrecognition by the identifier in determining the improvement area, and generates an analysis result and a quality of recognition basis of the target image.

3. The identifier generating device according to claim 1, characterized in that: The processor uses a cost based on a recognition result of the recognizer and a cost based on the recognition basis and the teaching image in updating the recognizer.

4. An image diagnosis auxiliary device, characterized in that: The image diagnosis auxiliary device comprises: a processor that executes a program for performing image processing on an image of a subject; and a memory for storing the program and the result of the image processing, The memory stores the object image and the identifier, The processor performs the following processing: extracting a feature quantity of the object image from the object image using the identifier; identifying the object image from the feature amount using the identifier to determine an identification value; Determining identification basis using the feature quantity and the identification value; Determining an improvement area of ​​the identifier using the recognition basis and the teaching image; and Information based on the identification basis and the improvement areas is provided.

5. The image diagnosis assisting device according to claim 4, characterized in that: The processor analyzes the cause of misrecognition by the identifier in determining the improvement area, and generates analysis results and quality of recognition basis of the object image. The information presented includes the quality of the identification basis.

6. The image diagnosis assisting device according to claim 4, characterized in that: The information provided includes the quality of the object image and the identification basis of the object image, The processor compares the performance values ​​of the respective identifiers, determines the performance of the identifiers, and presents information on the determination result in determining the improvement area.

7. A method for generating a classifier, wherein a system generates a classifier for classifying a desired object in an object image, characterized in that: The system comprises: a processor that executes a program for performing image processing on an image of a subject; and a memory for storing the program and the result of the image processing, The memory stores the object image and the identifier, In the identifier generation method, the processor performs the following processing: extracting a feature quantity of the object image from the object image using the identifier; identifying the object image from the feature amount using the identifier to determine an identification value; Determining identification basis using the feature quantity and the identification value; Determining an improvement area of ​​the identifier using the recognition basis and the teaching image; and The identifier is updated based on the improved areas.

8. The identifier generation method according to claim 7, characterized in that: In determining the improvement area, the cause of the erroneous recognition of the identifier is analyzed, and the analysis result and the quality of the recognition basis of the target image are produced.

9. The identifier generation method according to claim 7, characterized in that: In updating the identifier, a cost based on a recognition result of the identifier and a cost based on the recognition basis and the teaching image are used.

10. An image diagnosis aid method, wherein a system classifies a desired object in an object image, characterized in that: The system comprises: a processor that executes a program for performing image processing on an image of a subject; and a memory for storing the program and the result of the image processing, The memory stores the object image and the identifier, In the image diagnosis auxiliary method, the processor performs the following processing: extracting a feature quantity of the object image from the object image using the identifier; Using the identifier, identifying the object image from the feature amount to determine an identification value; Determining identification basis using the feature quantity and the identification value; Determining an improvement area of ​​the identifier using the recognition basis and the teaching image; and Information based on the identification basis and the improvement areas is provided.

11. The image diagnosis auxiliary method according to claim 10, characterized in that: In determining the improvement area, the cause of the misrecognition of the identifier is analyzed, and the analysis result and the quality of the recognition basis of the object image are prepared. The information provided includes the quality of the identification basis.

12. The image diagnosis auxiliary method according to claim 10, characterized in that: The information provided includes the quality of the object image and the identification basis of the object image, In the determination of the improvement area, the performance values ​​of the respective identifiers are compared, the performance of the identifiers is determined, and information on the determination result is presented.

13. A remote diagnosis assistance system, characterized in that: The remote diagnosis assistance system comprises: an image acquisition device having a photographing device for photographing image data; The identifier generating device according to claim 1; and Image diagnostic aids, The image acquisition device sends the image data to the identifier generation device, The identifier generating means generates an identifier using the received image data, The image diagnosis auxiliary device performs the following processing: acquiring an image of the identifier and the object from the image acquisition device; extracting a feature quantity of the image of the object from the image of the object using the identifier; identifying the object from the feature quantity using the identifier to determine an identification value and an identification result; Determining identification basis using the feature quantity and the identification value; Determining an improvement area of ​​the identifier using the recognition basis and the teaching image; and sending the recognition result and information based on the recognition basis and the improvement area to the image acquisition device, The image acquisition device displays the image of the object, the recognition result, and information based on the recognition basis and the improvement area on a display device.

14. A system for providing entrusted services, characterized in that: The entrusted service providing system comprises: An image acquisition device, comprising a photographing device for photographing image data and an image diagnosis assisting device; and The identifier generating device according to claim 1, The image acquisition device sends the image data to the identifier generation device, The identifier generating means generates an identifier using the received image data, The identifier generating device sends the generated identifier to the image acquiring device, The image acquisition device stores the received identifier in a memory, The image diagnosis auxiliary device performs the following processing: obtaining an image of the identifier and the object from the camera; extracting a feature quantity of the image of the object from the image of the object using the identifier; identifying the object from the feature quantity using the identifier to determine an identification value and an identification result; Determining identification basis using the feature quantity and the identification value; Determining an improvement area of ​​the identifier using the recognition basis and the teaching image; and The image of the object, the recognition result, and information based on the recognition basis and the improvement area are displayed on a display device.

Images