Image processing device, image processing method, and program

Abstract

In order to improve the accuracy of classifying a sample cell as benign or malignant in pathological diagnosis, an image processing device (1) comprises: a first acquisition unit (1) for acquiring a target image including a sample cell as a subject; a second acquisition unit (12) for acquiring characteristic information representing the characteristics of a learning image used to train an estimation model trained so as to estimate, upon receiving input of an image including a cell as a subject, whether the cell is a benign cell or a malignant cell; and a generation unit (13) for referencing the characteristic information and generating an image in which the characteristics of the target image are changed.

Description

[Technical field]

[0001] The present invention relates to an image processing device, an image processing method, and a program. [Background technology]

[0002] A technology is disclosed that relates to a trained model that performs image recognition of images that include a specimen as a subject in pathological diagnosis.

[0003] Patent Document 1 discloses a technology that normalizes WSI data by preprocessing multiple WSIs (Whole Slide Images) with different dyes to reduce sample variation, and then applies the normalized WSI data to a machine learning model. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japan Special Publication No. 2021-524630 Summary of the Invention [Problem to be solved by the invention]

[0005] The technology of Patent Document 1 does not consider the relationship between the image for learning and the image input to the machine learning model for image recognition in pathological diagnosis. Therefore, the technology of Patent Document 1 has a problem that the accuracy of image recognition of the image input to the machine learning model may be low.

[0006] Such a problem can cause false positives in benign / malignant classification, where a sample cell is classified as benign or malignant. When a false positive occurs, a retest is required even though it is not necessary. In addition, when a false negative occurs, where a malignant cell is classified as benign, there is a risk that cancer will be overlooked. Therefore, in benign / malignant classification, it is preferable to have high accuracy in image recognition of the input image.

[0007] One aspect of the present invention has been made in consideration of the above-mentioned problems, and one example of an objective of the present invention is to provide a technique for improving the accuracy of classifying specimen cells into benign or malignant in pathological diagnosis. [Means for solving the problem]

[0008] An image processing device according to one aspect of the present invention comprises a first acquisition means for acquiring a target image including a specimen cell as a subject, a second acquisition means for acquiring characteristic information indicating characteristics of a training image used to train an estimation model trained to estimate whether the cell is a benign cell or a malignant cell by inputting an image including a cell as a subject, and a generation means for generating an image in which the characteristics of the target image have been modified by referring to the characteristic information.

[0009] An image processing method according to one aspect of the present invention includes an image processing device acquiring a target image including specimen cells as subjects, acquiring characteristic information indicating characteristics of a training image used to train an estimation model trained to estimate whether a cell is a benign cell or a malignant cell using an image including cells as subjects as input, and generating an image in which the characteristics of the target image have been modified by referring to the characteristic information.

[0010] A program according to one aspect of the present invention is a program that causes a computer to function as an image processing device, the program causing the computer to function as a first acquisition means that acquires a target image including a specimen cell as a subject, a second acquisition means that receives an image including a cell as a subject as input and acquires characteristic information indicating characteristics of a training image used to train an estimation model that has been trained to estimate whether the cell is a benign cell or a malignant cell, and a generation means that refers to the characteristic information and generates an image in which the characteristics of the target image have been modified. Effect of the Invention

[0011] According to one aspect of the present invention, it is possible to improve the accuracy of classifying specimen cells into benign or malignant in pathological diagnosis. [Brief description of the drawings]

[0012] [Figure 1] FIG. 1 is a block diagram showing a configuration of an image processing device according to a first exemplary embodiment of the present invention. [Diagram 2] FIG. 2 is a flow diagram of an image processing method according to the first exemplary embodiment of the present invention. [Diagram 3] FIG. 11 is a block diagram showing a configuration of a classification device according to an exemplary embodiment 2 of the present invention. [Figure 4] FIG. 11 is a flowchart showing an example of a processing flow in which a generating unit according to the second exemplary embodiment of the present invention generates an image in which the noise level of a target image has been changed. [Diagram 5] FIG. 11 is a flowchart showing an example of a processing flow in which a generating unit according to the second exemplary embodiment of the present invention generates an image in which the color tone of a target image is changed. [Figure 6] FIG. 11 is a diagram showing an example of a histogram of hue in the second exemplary embodiment of the present invention. [Figure 7] FIG. 11 is a flowchart showing an example of a processing flow in which a generation unit according to the second exemplary embodiment of the present invention changes the resolution of a target image. [Figure 8] FIG. 11 is a diagram showing an example of an image of a micrometer in the second exemplary embodiment of the present invention. [Figure 9]FIG. 11 is a flowchart showing another example of the processing flow in which the generation unit according to the second exemplary embodiment of the present invention changes the resolution of a target image. [Figure 10] FIG. 2 is a block diagram showing an example of a hardware configuration of an image processing device and a classification device in each exemplary embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] Example 1 A first exemplary embodiment of the present invention will be described in detail with reference to the drawings. This exemplary embodiment is a basic form of the exemplary embodiments described later.

[0014] (Overview of image processing device 1) The image processing device 1 according to this exemplary embodiment is an image processing device that generates an image in which the characteristics of a target image including a specimen cell as a subject have been changed. As an example, the image processing device 1 receives an image including a cell as a subject as an input, and generates an image in which the characteristics of the target image have been changed in accordance with the characteristics of a learning image used in training an estimation model that has been trained to estimate whether the cell is a benign cell or a malignant cell.

[0015] The image characteristics refer to the properties and characteristics of an image, such as the noise level indicating the magnitude of noise contained in the image, the color tone consisting of the hue, brightness, and saturation of the image, the resolution indicating the number of pixels contained in a given length in the image, and information indicating the size of a given object contained as a subject in the image.

[0016] Although the specific configuration of the estimation model does not limit the present exemplary embodiment, as an example, a convolution neural network (CNN), a recurrent neural network (RNN), or a combination thereof may be used. A non-neural network model such as a random forest or a support vector machine may also be used.

[0017] (Configuration of image processing device 1) The configuration of an image processing device 1 according to this exemplary embodiment will be described with reference to Fig. 1. Fig. 1 is a block diagram showing the configuration of an image processing device 1 according to this exemplary embodiment.

[0018] 1, the image processing device 1 includes a first acquisition unit 11, a second acquisition unit 12, and a generation unit 13. The first acquisition unit 11, the second acquisition unit 12, and the generation unit 13 are configured to respectively realize a first acquisition means, a second acquisition means, and a generation means in this exemplary embodiment.

[0019] The first acquisition unit 11 acquires a target image including a specimen cell as a subject. The first acquisition unit 11 supplies the acquired target image to the generation unit 13.

[0020] The second acquisition unit 12 receives an image including a cell as a subject, and acquires characteristic information indicating characteristics of a learning image used for training a trained estimation model to estimate whether the cell is a benign cell or a malignant cell. The second acquisition unit 12 supplies the acquired characteristic information to the generation unit 13.

[0021] The generating unit 13 refers to the characteristic information supplied from the second acquiring unit 12 and generates an image in which the characteristic of the target image supplied from the first acquiring unit 11 is changed.

[0022] As described above, the image processing device 1 according to this exemplary embodiment has a configuration including a first acquisition unit 11 that acquires a target image including specimen cells as a subject, a second acquisition unit 12 that receives an image including cells as a subject as input and acquires characteristic information indicating the characteristics of a learning image used to train an estimation model trained to estimate whether the cell is a benign cell or a malignant cell, and a generation unit 13 that generates an image in which the characteristics of the target image have been modified by referring to the characteristic information.

[0023] In other words, the image processing device 1 according to this exemplary embodiment generates an image in which the characteristics of a target image are modified to match the characteristics of the learning image used to train the estimation model, and therefore, according to the image processing device 1 according to this exemplary embodiment, it is possible to obtain the effect of improving the accuracy of classifying specimen cells as benign or malignant in pathological diagnosis.

[0024] (Flow of image processing method S1) The flow of the image processing method S1 according to this exemplary embodiment will be described with reference to Fig. 2. Fig. 2 is a flow diagram showing the flow of the image processing method S1 according to this exemplary embodiment.

[0025] (Step S11) In step S11, the first acquisition unit 11 acquires a target image including a specimen cell as a subject. The first acquisition unit 11 supplies the acquired target image to the generation unit 13.

[0026] (Step S12) In step S12, the second acquisition unit 12 receives an image including a cell as a subject, and acquires characteristic information indicating characteristics of a learning image used for training a trained estimation model to estimate whether the cell is a benign cell or a malignant cell. The second acquisition unit 12 supplies the acquired characteristic information to the generation unit 13.

[0027] (Step S13) In step S13, the generation unit 13 refers to the characteristic information supplied from the second acquisition unit 12, and generates an image in which the characteristic of the target image supplied from the first acquisition unit 11 is changed.

[0028] As described above, the image processing method S1 according to this exemplary embodiment includes a configuration including: acquiring a target image including a specimen cell as a subject; acquiring characteristic information indicating characteristics of a learning image used to train an estimation model trained to estimate whether the cell is a benign cell or a malignant cell by inputting an image including the cell as a subject; and generating an image in which the characteristics of the target image are changed by referring to the characteristic information. Therefore, the image processing method S1 according to this exemplary embodiment can achieve the same effect as the image processing device 1 described above.

[0029] Exemplary embodiment 2 A second exemplary embodiment of the present invention will be described in detail with reference to the drawings. Note that components having the same functions as those described in the first exemplary embodiment are given the same reference numerals, and the description thereof will be omitted as appropriate.

[0030] (Overview of Classification Device 2) The classification device 2 according to this exemplary embodiment is a device that classifies specimen cells as to whether they are benign cells or malignant cells.

[0031] As an example, the classification device 2 inputs an image containing a cell as a subject into an estimation model that has been trained to predict whether the cell is a benign cell or a malignant cell, and then classifies the specimen cell as either a benign cell or a malignant cell.

[0032] More specifically, the classification device 2 acquires a target image including specimen cells as subjects, and generates an image in which the characteristics of the target image match those of the learning image used to train the estimation model. The classification device 2 then inputs the generated image to the estimation model, and classifies the specimen cells as benign or malignant cells by referring to the results of the estimation model. That is, the classification device 2 includes the configuration of the image processing device 1 described above, and classifies the specimen cells as benign or malignant cells by inputting the image generated by the image processing device 1 to the estimation model.

[0033] As an example, the learning image is a digitized image containing specimen cells as a subject, and the target image is a photographed image of the specimen cells projected under a microscope. When the learning image is a digitized image, many learning images can be acquired, and the classification device 2 can improve the accuracy of the estimation model. When the target image is a photographed image of an image projected under a microscope, the classification device 2 can be used for cytology in rapid on-site evaluation (ROSE).

[0034] The image characteristics and the estimation model are as described above.

[0035] (Configuration of Classification Device 2) The configuration of the classification device 2 according to this exemplary embodiment will be described with reference to Fig. 3. Fig. 3 is a block diagram showing the configuration of the classification device 2 according to this exemplary embodiment.

[0036] As shown in FIG. 3, the classification device 2 includes a control unit 10, a storage unit 21, and an input / output unit 22.

[0037] The storage unit 21 stores data referenced by the control unit 10, which will be described later. Examples of the data stored in the storage unit 21 include characteristic information PI indicating characteristics of the learning images TP used in learning the estimation model, target images SP including specimen cells as subjects, and the learning images TP.

[0038] The input / output unit 22 is an interface that acquires data from other connected devices or outputs data to other connected devices. The input / output unit 22 supplies data acquired from other devices to the control unit 10 and outputs data supplied from the control unit 10 to other devices.

[0039] The input / output unit 22 may be a communication module that communicates with other devices via a network. Although the specific configuration of the network does not limit the present exemplary embodiment, as an example, a wireless LAN (Local Area Network), a wired LAN, a WAN (Wide Area Network), a public line network, a mobile data communication network, or a combination of these networks can be used.

[0040] (Control unit 10) The control unit 10 controls each component included in the classification device 2. As shown in Fig. 3, the control unit 10 also functions as a first acquisition unit 11, a second acquisition unit 12, a generation unit 13, and a classification unit 14. The first acquisition unit 11, the second acquisition unit 12, the generation unit 13, and the classification unit 14 are configured to respectively realize a first acquisition means, a second acquisition means, a generation means, and a classification means in this exemplary embodiment.

[0041] The first acquisition unit 11 acquires a target image SP that includes a specimen cell as a subject. The first acquisition unit 11 stores the acquired target image SP in the storage unit .

[0042] The second acquisition unit 12 receives an image including a cell as a subject, and acquires characteristic information PI indicating characteristics of the training image TP used in training the trained estimation model to estimate whether the cell is a benign cell or a malignant cell. The second acquisition unit 12 may be configured to acquire the characteristic information PI itself, or may acquire the characteristic information PI by acquiring one or more training images TP and deriving the characteristic information PI from the one or more training images TP. The second acquisition unit 12 stores the acquired characteristic information PI (and the training images TP) in the memory unit 21.

[0043] Here, examples of characteristic information PI include characteristic information PI that indicates characteristics obtained by averaging the characteristics in each of multiple learning images TP, and characteristic information PI that indicates characteristics that are the median of the variation in characteristics in each of multiple learning images TP.

[0044] As shown in FIG. 3, the second acquisition unit 12 includes a derivation unit 121.

[0045] The derivation unit 121 derives the characteristic information PI from the training image TP. As one example, the derivation unit 121 extracts characteristics of the training image TP and derives the characteristic information PI indicating the extracted characteristics. As another example, the derivation unit 121 extracts characteristics of each of a plurality of training images TP and derives the characteristic information PI indicating an average characteristic of each of the extracted characteristics or a characteristic that is a median value of the variation of each of the extracted characteristics.

[0046] The generating unit 13 generates an image in which the characteristics of the target image SP have been changed, with reference to the characteristic information PI. The generating unit 13 supplies the generated image to the classifying unit .

[0047] As one example, the generation unit 13 generates an image in which the noise level of the target image SP has been changed. As another example, the generation unit 13 generates an image in which the color tone of the target image SP has been changed. As yet another example, the generation unit 13 generates an image in which the resolution of the target image SP has been changed. As yet another example, the generation unit 13 generates an image in which the size of an object included in the target image SP has been changed.

[0048] The generating unit 13 may generate an image in which a plurality of characteristics of the target image SP are changed. As an example, the generating unit 13 generates an image in which the noise level, color tone, and resolution of the target image SP are changed.

[0049] In addition, when generating an image in which multiple characteristics of the target image SP have been changed, the generation unit 13 may generate an image in which the characteristics have been changed in a predetermined order, or may generate an image in which multiple characteristics of the target image SP have been changed regardless of the order.

[0050] As an example of a configuration for generating an image with characteristics changed in a predetermined order, the generation unit 13 first generates an image with a changed noise level of the target image SP. Next, the generation unit 13 performs processing to change the color of the image after the noise level change, and generates an image with the changed color. Then, the generation unit 13 performs processing to change the resolution of the image after the color change, and generates an image with the changed resolution.

[0051] The flow of the process executed by the generation unit 13 will be described later.

[0052] The classification unit 14 inputs the image supplied from the generation unit 13 into an estimation model, and classifies the specimen cells contained as subjects in the image supplied from the generation unit 13 as benign cells or malignant cells. The classification unit 14 outputs the classification result via the input / output unit 22.

[0053] As an example, when the estimation result output from the estimation model indicates that the specimen cell is a benign cell, the classification unit 14 classifies the specimen cell included as a subject in the image supplied from the generation unit 13 as a benign cell. On the other hand, when the estimation result output from the estimation model indicates that the specimen cell is a malignant cell, the classification unit 14 classifies the specimen cell included as a subject in the image supplied from the generation unit 13 as a malignant cell.

[0054] (Processing to generate an image with a changed noise level) An example of a process in which the generating unit 13 generates an image with a changed noise level of the target image SP will be described with reference to Fig. 4. Fig. 4 is a flow diagram showing an example of the flow of a process S2 in which the generating unit 13 according to this exemplary embodiment generates an image with a changed noise level of the target image SP. In the following, the characteristic information PI is information indicating the noise level in the learning image TP, and is information indicating the noise level in a background area that does not include an object as a subject, and a process in which the generating unit 13 calculates the noise level by applying a Gaussian filter will be described.

[0055] (Step S21) In step S21, the generating unit 13 extracts a background region, which is a region that does not include an object as a subject, from the target image SP. There is no particular limitation on the method by which the generating unit 13 extracts the background region, but one example is a method of extracting a region with a pixel value equal to or less than a predetermined pixel value as the background region.

[0056] (Step S22) In step S22, the generation unit 13 applies a Gaussian filter to the extracted background region. The sigma value of the Gaussian filter applied by the generation unit 13 is not particularly limited, but examples include values ​​such as 1.0 and 1.5.

[0057] (Step S23) In step S23, the generation unit 13 calculates the noise level of the background region after application of the Gaussian filter. As an example, when the characteristic information PI indicates the standard deviation of pixel values ​​as the noise level, the generation unit 13 calculates the standard deviation of pixel values ​​in the background region after application of the Gaussian filter as the noise level. Then, the generation unit 13 calculates the difference between the calculated standard deviation of pixel values ​​in the background region and the standard deviation indicated by the characteristic information PI.

[0058] (Step S24) In step S24, the generation unit 13 determines whether the calculated difference is within a predetermined range.

[0059] (Step S25) If it is determined in step S24 that the calculated difference is not within the predetermined range (step S24: NO), in step S25, the generation unit 13 changes the sigma value of the Gaussian filter to be applied.

[0060] As an example, if the standard deviation of pixel values ​​in the background region after application of the Gaussian filter is higher than the standard deviation indicated by the characteristic information PI, the generation unit 13 increases the sigma value by 10%. On the other hand, if the standard deviation of pixel values ​​in the background region after application of the Gaussian filter is lower than the standard deviation indicated by the characteristic information PI, the generation unit 13 decreases the sigma value by 10%. In other words, the generation unit 13 derives a sigma value that brings the difference between the noise level indicated by the characteristic information PI and the noise level in the background region of the target image SP into a predetermined range.

[0061] Then, the generation unit 13 returns to the process of step S22 to apply the Gaussian filter with the changed sigma value.

[0062] (Step S26) If it is determined in step S24 that the calculated difference is within a predetermined range (step S24: YES), in step S26, the generator 13 applies a Gaussian filter with a sigma value in which the difference between the noise level indicated by the characteristic information PI and the noise level in the background region of the target image SP is within a predetermined range to the target image SP to generate an image with an altered noise level. That is, the generator 13 generates an image with an altered noise level of the target image SP so that the noise level in the extracted background region of the target image SP approaches the noise level indicated by the characteristic information PI.

[0063] In this way, the generating unit 13 generates an image in which the noise level of the target image SP is changed so that the noise level in the background region of the extracted target image SP approaches the noise level indicated by the characteristic information PI. Therefore, the generating unit 13 can input an image having a noise level equivalent to the noise level of the learning image TP to the estimation model, thereby improving the accuracy of classification of specimen cells into benign or malignant in pathological diagnosis.

[0064] (Process 1 for generating an image with changed color) An example of the process in which the generation unit 13 generates an image in which the hue of the target image SP has been changed will be described with reference to Fig. 5. Fig. 5 is a flow diagram showing an example of the flow of process S3 in which the generation unit 13 according to this exemplary embodiment generates an image in which the hue of the target image SP has been changed. In the following, the characteristic information PI is information indicating the hue in the learning image TP, and is information indicating the hue of an object region in which an object is included as a subject, and as an example, a case will be described in which the characteristic information PI is information indicating the histograms of hue, saturation, and brightness in the object region of the learning image TP.

[0065] (Step S31) In step S31, the generating unit 13 extracts an object region, which is a region that includes an object as a subject, from the target image SP. There is no particular limitation on the method by which the generating unit 13 extracts the object region, but one example is a method of extracting a region with a pixel value equal to or greater than a predetermined pixel value as the object region.

[0066] (Step S32) In step S32, the generation unit 13 generates a histogram of the color tone of the extracted object region as information indicating the color tone of the extracted object region. As an example, the generation unit 13 generates a histogram of each of the hue, saturation, and brightness of the extracted object region.

[0067] (Step S33) In step S33, the generation unit 13 generates an image in which the color tone in the object region of the extracted target image SP is changed to approach the color tone indicated by the characteristic information PI. An example of the processing of the generation unit 13 in step S33 will be described with reference to Fig. 6. Fig. 6 is a diagram showing an example of a histogram of hue in this exemplary embodiment. Although the histogram of hue will be described below, the same applies to the histograms of saturation and brightness.

[0068] When the histogram of the hue indicated by the characteristic information PI is the histogram shown in the upper part of FIG. 6, the generation unit 13 calculates the average value mean pi , the left end of the 95% confidence interval a pi and rightmost b pi Calculate.

[0069] Next, if the histogram of hues in the object region of the target image SP generated in step S32 is the histogram in the middle of FIG. 6, the generation unit 13 similarly calculates the average value in the histogram of hues in the object region of the target image SP. sp1 , the left end of the 95% confidence interval a sp1 and rightmost b sp1 Calculate.

[0070] Then, the generation unit 13 calculates the changed value Out using the following formula (1).

[0071]

number

[0072] That is, the generation unit 13 uses the formula (1) to calculate a value Var obtained by subtracting the upper limit value from the lower limit value of the 95% confidence interval for a certain hue value In in the target image SP in the 95% confidence interval of the certain hue value. poi1 , which is the target histogram, Var tarto the target hue value by modifying it to match

[0073] The histogram of the hue in the object region of the converted image is shown in the lower part of Fig. 6. As shown in the lower part of Fig. 6, the average value of the histogram of the hue in the object region after conversion is sp2 , the left end of the 95% confidence interval a sp2 and rightmost b sp2 is the object region of the target image SP Hue The mean value of the histogram sp1 , the left end of the 95% confidence interval a sp1 and rightmost b sp1 Compared to the mean value of the histogram shown by the characteristic information PI, pi , the left end of the 95% confidence interval a pi and rightmost b pi It's getting closer.

[0074] In this way, the generating unit 13 generates an image in which the color tone in the object region of the target image SP is changed to approach the color tone indicated by the characteristic information PI. Therefore, the generating unit 13 can input an image having a color tone equivalent to that of the learning image TP to the estimation model, thereby improving the accuracy of classifying specimen cells into benign or malignant in pathological diagnosis.

[0075] (Process 2 for generating an image with different color tones) In the process of changing the color tone of the target image SP, the generation unit 13 may change the color tone of the background region of the target image SP in addition to the object region of the target image SP.

[0076] As an example, the generation unit 13 may change the pixel values ​​RGB of the background region of the target image SP to predetermined values ​​(for example, in the case of an image with 256 gradations, (R,G,B)=(230,230,230)).

[0077] As another example, the generation unit 13 may be configured to refer to the background region of the learning image TP. As one example, the generation unit 13 may obtain the average value of the pixel values ​​of the background region of the learning image TP, and change the pixel value of the background region of the target image SP to the average value. As another example, the generation unit 13 may extract the background region of each of the multiple learning images TP, and calculate the average value of the pixel values ​​of each extracted background region. Then, the generation unit 13 may change the pixel value of the background region of the target image SP to the calculated average value. In this case, the generation unit 13 may use the median of the variation of the pixel values ​​instead of the average value of the pixel values.

[0078] (Process 1 for generating images with changed resolution) An example of the process in which the generation unit 13 changes the resolution of the target image SP will be described with reference to Fig. 7. Fig. 7 is a flow diagram showing an example of the flow of process S4 in which the generation unit 13 changes the resolution of the target image SP according to this exemplary embodiment. In the following, the characteristic information PI is information indicating the resolution of the learning image TP, and the process in which the generation unit 13 refers to an image including a micrometer as a subject will be described.

[0079] (Step S41) In step S41, the generation unit 13 acquires an image of a micrometer captured at the same resolution as the target image SP. An example of the image of the micrometer will be described with reference to Fig. 8. Fig. 8 is a diagram showing an example of the image of the micrometer in this exemplary embodiment. The generation unit 13 acquires an image including the micrometer as a subject, as shown in image P1 in Fig. 8.

[0080] (Step S42) In step S42, the generation unit 13 calculates the resolution of the micrometer image.

[0081] As an example, the generation unit 13 first generates an image by binarizing the image P1, as shown in image P2 in Fig. 8. Next, the generation unit 13 performs a closing process by performing a dilation process and an erosion process on the image P2, as shown in image P3 in Fig. 8. Then, as shown in image P4 in Fig. 8, the generation unit 13 calculates the coordinates of the pixel at the left end of the micrometer (X L ,Y L ) and the coordinate of the rightmost pixel of the micrometer (X R ,Y R ) is detected and the resolution is calculated using the following equation (2).

[0082]

number

[0083] That is, the generating unit 13 calculates the resolution by calculating the number of pixels of the target image SP included in a predetermined length using the formula (2).

[0084] (Step S43) In step S43, the generation unit 13 generates an image by changing the calculated resolution to the resolution indicated by the characteristic information PI.

[0085] In this way, the generating unit 13 generates an image in which the resolution of the target image SP is changed so that the resolution indicated by the characteristic information PI is the same as the resolution of the target image SP. Therefore, an image in which the size of an object included as a subject in the learning image TP is equal to the size of an object included as a subject in the target image SP can be input to the estimation model, thereby improving the accuracy of classifying specimen cells as benign or malignant in pathological diagnosis.

[0086] (Process 2 for generating images with changed resolution) Another example of the process in which the generation unit 13 changes the resolution of the target image SP will be described with reference to Fig. 9. Fig. 9 is a flow diagram showing another example of the flow of the process S5 in which the generation unit 13 changes the resolution of the target image SP according to this exemplary embodiment. In the following, the characteristic information PI is information indicating the size of a predetermined object included as a subject in the learning image TP, and the generation unit 13 will be described as extracting the predetermined object included as a subject in the target image SP.

[0087] Here, the predetermined object is not particularly limited, but an object with small individual variation in size is preferable, and examples thereof include red blood cells and lymphocytes. With this configuration, the generation unit 13 uses red blood cells or lymphocytes with small individual variation as a standard for the size of the object included in the target image SP, so that the size of the object can be suitably calculated. The following describes the case where the predetermined object is a red blood cell.

[0088] (Step S51) In step S51, the generation unit 13 extracts red blood cells included as subjects in the target image SP as a predetermined object included as a subject in the target image SP. As an example of a method for the generation unit 13 to extract red blood cells included as subjects in the target image SP, a method using a known deep learning segmentation (DL segmentation) can be mentioned.

[0089] (Step S52) In step S52, the generation unit 13 calculates the size of the extracted red blood cell. The generation unit 13 may approximate the extracted red blood cell to an ellipse and calculate the major axis and minor axis.

[0090] In addition, when multiple red blood cells are extracted, as one example, the generation unit 13 may calculate the major axis and minor axis of the red blood cell with the largest area. As another example, the generation unit 13 may calculate the major axis and minor axis of the red blood cell that are the median of the variation in the areas of the multiple red blood cells. As yet another example, the generation unit 13 may calculate the average values ​​of the major axis and minor axis of the multiple red blood cells.

[0091] (Step S53) In step S53, the generating unit 13 generates an image by changing the resolution of the target image SP so that the calculated size (major axis and minor axis) of the red blood cell becomes the size (major axis and minor axis) indicated by the characteristic information PI.

[0092] In this way, the generating unit 13 generates an image in which the resolution of the target image SP is changed so that the size of the predetermined object indicated by the characteristic information PI is equivalent to the size of the predetermined object included as a subject in the target image SP. Therefore, an image in which the size of the object included as a subject in the learning image TP is equivalent to the size of the object included as a subject in the target image SP can be input to the estimation model, thereby improving the accuracy of classifying specimen cells as benign or malignant in pathological diagnosis.

[0093] (Effect of Classifier 2) As described above, the classification device 2 according to the present exemplary embodiment receives an image containing a cell as a subject as an input, and generates an image with modified characteristics of a target image SP containing a specimen cell as a subject by referring to characteristic information PI indicating characteristics of a training image TP used in training an estimation model trained to estimate whether the cell is a benign cell or a malignant cell. The classification device 2 according to the present exemplary embodiment then employs a configuration in which the generated image is input to the estimation model.

[0094] In other words, the classification device 2 according to this exemplary embodiment generates an image in which the characteristics of the target image SP are modified to match the characteristics of the training image TP used to train the estimation model, and then inputs the image into the estimation model. Therefore, according to the classification device 2 according to this exemplary embodiment, it is possible to obtain the effect of improving the accuracy of classifying sample cells into benign or malignant in pathological diagnosis.

[0095] [Software implementation example] Some or all of the functions of the image processing device 1 and the classification device 2 may be realized by hardware such as an integrated circuit (IC chip), or may be realized by software.

[0096] In the latter case, the image processing device 1 and the classification device 2 are realized, for example, by a computer that executes instructions of a program, which is software that realizes each function. An example of such a computer (hereinafter, referred to as computer C) is shown in FIG. 10. The computer C includes at least one processor C1 and at least one memory C2. The memory C2 stores a program P for operating the computer C as the image processing device 1 and the classification device 2. In the computer C, the processor C1 reads and executes the program P from the memory C2, thereby realizing each function of the image processing device 1 and the classification device 2.

[0097] The processor C1 may be, for example, a central processing unit (CPU), a graphic processing unit (GPU), a digital signal processor (DSP), a micro processing unit (MPU), a floating point number processing unit (FPU), a physics processing unit (PPU), a microcontroller, or a combination of these. The memory C2 may be, for example, a flash memory, a hard disk drive (HDD), a solid state drive (SSD), or a combination of these.

[0098] The computer C may further include a RAM (Random Access Memory) for expanding the program P during execution and for temporarily storing various data. The computer C may further include a communication interface for transmitting and receiving data to and from other devices. The computer C may further include an input / output interface for connecting input / output devices such as a keyboard, a mouse, a display, and a printer.

[0099] Furthermore, the program P can be recorded on a non-transitory tangible recording medium M that can be read by the computer C. Such a recording medium M can be, for example, a tape, a disk, a card, a semiconductor memory, or a programmable logic circuit. The computer C can obtain the program P via such a recording medium M. Furthermore, the program P can be transmitted via a transmission medium. Such a transmission medium can be, for example, a communication network or broadcast waves. The computer C can also obtain the program P via such a transmission medium.

[0100] [Additional Note 1] The present invention is not limited to the above-described embodiment, and various modifications are possible within the scope of the claims. For example, embodiments obtained by appropriately combining the technical means disclosed in the above-described embodiment are also included in the technical scope of the present invention.

[0101] [Additional Note 2] Some or all of the above-described embodiments can be described as follows. However, the present invention is not limited to the following described aspects.

[0102] (Appendix 1) An image processing device comprising: a first acquisition means for acquiring a target image including a specimen cell as a subject; a second acquisition means for acquiring characteristic information indicating characteristics of a learning image used to train an estimation model trained to estimate whether the cell is a benign cell or a malignant cell by inputting an image including a cell as a subject; and a generation means for generating an image in which characteristics of the target image have been changed by referring to the characteristic information.

[0103] (Appendix 2) The image processing device described in Appendix 1, wherein the characteristic information is information indicating the noise level in the learning image, and is information indicating the noise level in a background area that does not include an object as a subject, and the generation means extracts a background area from the target image and generates an image in which the noise level of the target image is changed so as to bring the noise level in the extracted background area of ​​the target image closer to the noise level indicated by the characteristic information.

[0104] (Appendix 3) The image processing device described in Appendix 2, wherein the generation means calculates a noise level by applying a Gaussian filter to a background region of the target image, and derives a sigma value such that a difference between the noise level indicated by the characteristic information and the noise level in the background region of the target image is within a predetermined range.

[0105] (Appendix 4) The image processing device of any one of claims 1 to 3, wherein the characteristic information is information indicating a color tone in the learning image, and is information indicating a color tone in an object region that contains an object as a subject, and the generation means extracts an object region from the target image and generates an image in which the color tone in the extracted object region of the target image is changed to approximate the color tone indicated by the characteristic information.

[0106] (Appendix 5) 5. The image processing device according to claim 4, wherein the characteristic information is information indicating histograms of hue, saturation, and brightness in an object region of the learning image.

[0107] (Appendix 6) An image processing device as described in any one of Appendices 1 to 5, wherein the characteristic information is information indicating a resolution of the learning image, and the generation means generates an image in which the resolution of the target image is changed to the resolution indicated by the characteristic information by referring to a micrometer image taken at the same resolution as the target image.

[0108] (Appendix 7) An image processing device as described in any one of Appendices 1 to 6, wherein the characteristic information is information indicating a size of a specified object included as a subject in the learning image, and the generation means extracts a specified object included as a subject in the target image, and generates an image in which the resolution of the target image is changed so that the size of the extracted object in the target image becomes the size indicated by the characteristic information.

[0109] (Appendix 8) 8. The image processing device according to claim 7, wherein the specified object is a red blood cell or a lymphocyte.

[0110] (Appendix 9) The image processing device according to any one of claims 1 to 8, wherein the second acquisition means is configured to acquire the learning image and to derive characteristic information from the learning image.

[0111] (Appendix 10) The image processing device according to any one of claims 1 to 9, wherein the second acquisition means acquires characteristic information indicating an average characteristic of each of a plurality of learning images, or characteristic information indicating a characteristic that is a median of the variation of the characteristics of each of a plurality of learning images.

[0112] (Appendix 11) An image processing device as described in any one of Appendices 1 to 10, wherein the learning image is a digitized image of an image including specimen cells as a subject, and the target image is an image obtained by photographing an image of the specimen cells viewed under a microscope.

[0113] (Appendix 12) A classification device comprising a classification means for inputting an image generated by the image processing device described in Supplementary Note 1 into the estimation model and classifying a specimen cell contained as a subject in the image generated by the image processing device described in Supplementary Note 1 as whether it is a benign cell or a malignant cell.

[0114] (Appendix 13) An image processing method including the steps of: an image processing device acquiring a target image including a specimen cell as a subject; acquiring characteristic information indicating characteristics of a training image used to train an estimation model that has been trained to estimate whether a cell is a benign cell or a malignant cell using an image including a cell as a subject as input; and generating an image in which the characteristics of the target image have been modified by referring to the characteristic information.

[0115] (Appendix 14) A program that causes a computer to function as an image processing device, the program causing the computer to function as: a first acquisition means that acquires a target image including a specimen cell as a subject; a second acquisition means that receives an image including a cell as a subject and acquires characteristic information indicating characteristics of a training image used to train an estimation model that has been trained to estimate whether the cell is a benign cell or a malignant cell; and a generation means that generates an image in which the characteristics of the target image have been modified by referring to the characteristic information.

[0116] [Additional Note 3] A part or all of the above-described embodiments can be further expressed as follows.

[0117] An image processing device comprising at least one processor that executes a first acquisition process for acquiring a target image including a specimen cell as a subject, a second acquisition process for receiving an image including a cell as a subject and acquiring characteristic information indicating characteristics of a learning image used in training an estimation model that has been trained to estimate whether the cell is a benign cell or a malignant cell, and a generation process for generating an image in which the characteristics of the target image have been modified by referring to the characteristic information.

[0118] The image processing device may further include a memory that stores a program for causing the processor to execute the first acquisition process, the second acquisition process, and the generation process. The program may be recorded in a computer-readable, non-transitory, tangible recording medium. [Explanation of symbols]

[0119] 1 Image processing device 2 Classification device 10 Control section 11 First Acquisition Section 12 Second Acquisition Section 13 Generation part 14 Classification section 121 Derivation part

Claims

1. a first acquisition means for acquiring a target image including a specimen cell as a subject; A second acquisition means for acquiring characteristic information indicating characteristics of a learning image used for training a trained estimation model by inputting an image including a cell as a subject, the image being trained to estimate whether the cell is a benign cell or a malignant cell; a generating means for generating an image in which characteristics of the target image have been changed by referring to the characteristic information; An image processing device comprising:

2. the characteristic information is information indicating a noise level in the learning image, and is information indicating a noise level in a background region that does not include an object as a subject, The generating means includes: Extracting a background region from the target image; An image is generated in which the noise level of the target image is changed so that the noise level in the extracted background region of the target image approaches the noise level indicated by the characteristic information. The image processing device according to claim 1 .

3. the characteristic information is information indicating a color tone in the learning image, and is information indicating a color tone in an object region that includes an object as a subject; The generating means includes: Extracting an object region from the target image; An image is generated in which the color tone in the extracted object region of the target image is changed so as to approach the color tone indicated by the characteristic information.

3. The image processing device according to claim 1 or 2.

4. the characteristic information is information indicating a resolution of the learning image, The generating means generates an image in which the resolution of the target image is changed to the resolution indicated by the characteristic information by referring to an image of a micrometer captured at the same resolution as the target image.

3. The image processing device according to claim 1 or 2.

5. the characteristic information is information indicating a size of a predetermined object included as a subject in the learning image, The generating means includes: Extracting a predetermined object included as a subject in the target image; An image is generated by changing the resolution of the target image so that the size of the object of the extracted target image becomes the size indicated by the characteristic information.

3. The image processing device according to claim 1 or 2.

6. The second acquisition means includes: Acquire the learning images; A derivation means for deriving characteristic information from the learning image is provided.

3. The image processing device according to claim 1 or 2.

7. The second acquisition means acquires characteristic information indicating an average characteristic of each of the plurality of learning images, or characteristic information indicating a median of a variation in the characteristic of each of the plurality of learning images.

3. The image processing device according to claim 1 or 2.

8. A classification means for inputting an image generated by the image processing device according to claim 1 into the estimation model and classifying specimen cells included as subjects in the image generated by the image processing device according to claim 1 as benign cells or malignant cells. A classification device comprising:

9. The image processing device acquiring a target image including a specimen cell as a subject; An image including a cell as a subject is input, and characteristic information indicating characteristics of a learning image used to train an estimation model trained to estimate whether the cell is a benign cell or a malignant cell is acquired; generating an image in which characteristics of the target image are changed by referring to the characteristic information; An image processing method comprising:

10. A program for causing a computer to function as an image processing device, The program causes the computer to a first acquisition means for acquiring a target image including a specimen cell as a subject; A second acquisition means for acquiring characteristic information indicating characteristics of a learning image used for training a trained estimation model by inputting an image including a cell as a subject, the image being trained to estimate whether the cell is a benign cell or a malignant cell; a generating means for generating an image in which characteristics of the target image have been changed by referring to the characteristic information; A program that functions as a

Images