Color correction device, display device, color correction method, and program

Abstract

This invention provides a color correction device, display device, color correction method, and program that can correct variations in the color tones of images. [Solution] The color correction device of the present disclosure includes: a color distribution figure generation unit that approximates the shape of a color distribution including a point cloud obtained by plotting the coordinate values ​​of a plurality of pixels included in an image on a chromaticity diagram and generates a color distribution figure on the chromaticity diagram; a vertex detection unit that detects a first vertex of the color distribution figure; a target point determination unit that determines a first target point for color correction of the image corresponding to the first vertex; and a color correction unit that performs the color correction by bringing the coordinate value of the first vertex closer to the coordinate value of the first target point on the chromaticity diagram.

Description

Technical Field

[0001] The present disclosure relates to a color correction device, a display device, a color correction method, and a program.

Background Art

[0002] Generally, a subject is photographed using an imaging device to generate an image. Even for images of the same subject, when displayed on different display units, the color tones of the subject may appear different from each other.

[0003] Patent Document 1 discloses a color correction method capable of reproducing the color tone of a subject displayed on a first display unit on a second display unit.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] As a technique used in pathological diagnosis, there is WSI (Whole Slide Imaging). WSI is a technique for digitally imaging a pathological specimen prepared by staining a specimen collected from a subject. By WSI, for example, a pathologist or a pathologist can remotely diagnose a large number of digital specimens.

[0006] When preparing a pathological specimen, if the staining method or the amount of dye used for staining is different, variations occur in the color tone of the prepared pathological specimen. When a plurality of pathological specimens with different color tones are digitized by WSI, variations in color tone also remain in each of the generated digital specimens.

[0007] A diagnostic support system is being developed in which AI (artificial intelligence) provides diagnostic assistance by training it using digital specimens digitized by WSI. However, if there is variation in the color tones of the digital specimens used as input to the AI, it can be difficult to train the AI ​​to provide highly accurate diagnostic support. In addition, if an image with a different color tone than the one used for training is input to a trained AI, the accuracy of the diagnostic support output by the AI ​​may not meet the expected accuracy.

[0008] Therefore, there is a demand for correcting variations in the color tones of digital images caused by differences in dyeing conditions and other factors.

[0009] This disclosure aims to provide a color correction device, display device, color correction method, and program that can correct variations in the color tone of an image. [Means for solving the problem]

[0010] A color correction device according to one aspect of the present disclosure includes: a color distribution figure generation unit that approximates the shape of a color distribution including a point cloud obtained by plotting the coordinate values ​​of a plurality of pixels included in an image on a chromaticity diagram and generates a color distribution figure on the chromaticity diagram; a vertex detection unit that detects a first vertex of the color distribution figure; a target point determination unit that determines a first target point for color correction of the image corresponding to the first vertex; and a color correction unit that performs color correction such that the coordinate value of the first vertex approaches the coordinate value of the first target point on the chromaticity diagram.

[0011] A display device according to one aspect of the present disclosure acquires the coordinate values ​​of the vertices of a color distribution figure that approximates a color distribution including a point cloud obtained by plotting the coordinate values ​​of a plurality of pixels included in an image on a chromaticity diagram, displays the color distribution and a point selection unit that accepts the selection of the vertices included in the color distribution on a display screen, and when the selection of a vertex is accepted, displays the position of the selected vertex in the image on the display screen.

[0012] A color correction method according to one aspect of the present disclosure is a color correction method performed by a computer, wherein the computer generates a color distribution figure that approximates a color distribution including a point cloud obtained by plotting the coordinate values ​​of a plurality of pixels included in an image on a chromaticity diagram, detects a first vertex of the color distribution figure, determines a first target point for color correction of the image corresponding to the first vertex, and performs the color correction to bring the coordinate value of the first vertex closer to the coordinate value of the first target point.

[0013] A program according to one aspect of the present disclosure is a program executed by a computer, which causes the computer to perform the following steps: generate a color distribution figure that approximates a color distribution including a point cloud obtained by plotting the coordinate values ​​of a plurality of pixels included in an image on a chromaticity diagram; detect a first vertex of the color distribution figure; determine a first target point for color correction of the image corresponding to the first vertex; and perform the color correction to bring the coordinate value of the first vertex closer to the coordinate value of the first target point. [Effects of the Invention]

[0014] According to this disclosure, variations in the color tone of images can be corrected. [Brief explanation of the drawing]

[0015] [Figure 1] A diagram showing an example of a color correction system configuration. [Figure 2] Block diagram illustrating an example of the functional configuration of a color correction device. [Figure 3] A diagram to explain color distribution [Figure 4] A diagram illustrating the generation of color distribution patterns by the color distribution pattern generation unit. [Figure 5] A diagram illustrating a specific example of color correction in a color correction device. [Figure 6] A diagram illustrating a specific example of color correction in a color correction device. [Figure 7] A flowchart illustrating an example of how a color correction device works. [Figure 8] Block diagram showing an example of the functional configuration of a display device. [Figure 9] Figure showing an example of the initial setting screen [Figure 10] Figure showing an example of the color distribution display screen [Figure 11] Figure showing an example of the vertex position display screen [Figure 12] Figure showing the state where the point selection part is displayed on the vertex position display screen [Figure 13] Figure showing the state where the specimen selection part is displayed on the vertex position display screen [Figure 14] Figure showing an example of the target image selection screen [Figure 15] Figure showing an example of the result confirmation screen [Figure 16] Figure showing an example of the preview screen [Figure 17] Figure showing an example of the hardware configuration of a computer for realizing the functions of the color correction device or the display device

Mode for Carrying Out the Invention

[0016] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings. However, detailed descriptions of well-known matters and duplicate descriptions for substantially the same configurations may be omitted. Also, for configurations common in the embodiments, the same reference numerals may be given, and duplicate descriptions may be omitted.

[0017] <Color correction system 100> A color correction system 100 including a color correction device 10 and a display device 20 according to an embodiment of the present disclosure will be described. FIG. 1 is a diagram showing an example of the configuration of the color correction system 100.

[0018] As shown in FIG. 1, the color correction system 100 includes a color correction device 10, a display device 20, and an imaging device 30. The color correction device 10 is communicably connected to the imaging devices 30 installed in each of a plurality of medical facilities.

[0019] Figure 1 shows three medical facilities A to C as examples of multiple medical facilities. Medical facility A is equipped with imaging device 30_A, medical facility B with imaging device 30_B, and medical facility C with imaging device 30_C. The number of medical facilities in this disclosure is not limited to three, and more medical facilities may be connected to the color correction device 10.

[0020] The imaging device 30 generates a digital specimen by photographing a pathological specimen prepared by staining biological tissue collected from a subject at a medical facility using WSI technology, and transmits it to the color correction device 10. In the following description, the digital specimen is assumed to be a digital image of a pathological specimen containing biological tissue stained with hematoxylin-eosin (HE) stain. Hereinafter, staining with HE stain will be referred to as HE staining. HE staining can distinguish between cell nuclei and non-nuclear tissue components in blue (blue-violet) and red with good contrast. For this reason, HE-stained pathological specimens and digital specimens obtained by digitizing such pathological specimens are easier for pathologists and other medical professionals to diagnose.

[0021] In this embodiment, a digital specimen is a digital image generated by fixing a specimen, which has been collected and stained from a subject, to a slide glass or the like and photographing it with the imaging device 30. The digital specimen is an example of an image in this disclosure.

[0022] In this disclosure, pathological specimens do not necessarily have to be stained by HE staining; they may be stained using one or more types of staining solutions to differentiate the target biological tissue. Pathological specimens in this disclosure may be stained, for example, by EVG (Elastica van Guieson) staining, MT (Masson's trichrome) staining, PAM (Methenamine silver periodate) staining, PAS (Schiff periodate) staining, DAB (Diaminobenzidine) staining, etc.

[0023] In this embodiment, it is assumed that the staining conditions during the preparation of pathological specimens differ among medical facilities, resulting in variations in the color of the digital specimens generated by the imaging device 30. Staining conditions include the staining method, or, if the same staining method is used, the concentration and ratio of the dye.

[0024] The digital specimen generated by the imaging device 30 is transmitted to the color correction device 10 for color correction. The color correction device 10 may also receive digital specimens generated by the imaging device 30 that have been color-corrected by other color correction devices (not shown) using known color charts or the like. The color correction using color charts and the color correction by the color correction device 10 are of different types. Color correction using color charts can be expected to correct in advance for deviations in brightness that may occur in digital specimens due to differences in shooting conditions in the imaging device 30, for example.

[0025] The color correction device 10 performs color correction on digital specimens generated by imaging devices 30 at multiple medical facilities.

[0026] The display device 20 is a device that displays information related to color correction based on various information received from the color correction device 10. The display device 20 also accepts user operations related to color correction and transmits information related to those operations to the color correction device 10.

[0027] In this embodiment, the user refers to the user of the color correction system 100. By using the color correction system 100, the user can collect digital specimens from multiple medical facilities and perform color correction on each of them. Furthermore, when performing color correction, the user can operate the display device 20 while viewing its display, thereby causing the color correction device 10 to perform the desired color correction.

[0028] In the color correction system 100 shown in Figure 1, at least the color correction device 10 and the display device 20 are computers, such as a PC (Personal Computer) or a tablet terminal. The functions of the color correction device 10 and the display device 20 are realized by the computer reading and executing a program that has been stored in advance.

[0029] <Configuration of color correction device 10> The color correction device 10 will now be described. Figure 2 is a block diagram illustrating an example of the functional configuration of the color correction device 10.

[0030] As shown in Figure 2, the color correction device 10 comprises a communication unit 11, a color distribution acquisition unit 12, a color distribution figure generation unit 13, a vertex detection unit 14, a target point determination unit 15, a color correction unit 16, and a registration unit 17. The color correction device 10 is a device that performs color correction on a digital sample acquired from the imaging device 30.

[0031] The communication unit 11 communicates with the imaging device 30 and display device 20 of each medical facility. The communication unit 11 acquires digital specimens to be color corrected from the imaging device 30. The communication unit 11 also transmits various information necessary for display to the display device 20 and receives information from the display device 20 regarding user operation of the color correction system 100.

[0032] The communication unit 11 may acquire multiple digital samples from the imaging device 30. In this case, the color correction device 10 may perform color correction processing on the acquired multiple digital samples in parallel or sequentially.

[0033] The color distribution acquisition unit 12 acquires the color distribution on a chromaticity diagram from the digital sample to be color corrected. In this disclosure, the color distribution acquisition unit 12 acquires the color distribution on a chromaticity diagram by converting the coordinate values ​​in the RGB color space of multiple pixels included in the digital sample to coordinate values ​​(chromaticity coordinates) in the XYZ color space and plotting them on an xy chromaticity diagram. In this specification, the xy chromaticity diagram is simply referred to as a chromaticity diagram. That is, the color distribution includes a point cloud obtained by plotting the coordinate values ​​of multiple pixels included in the digital sample on a chromaticity diagram. Each point included in the point cloud indicates the position on the chromaticity diagram of the color of each of the multiple pixels.

[0034] Figure 3 is a diagram illustrating color distribution. Figure 3A shows an example of a chromaticity diagram. A chromaticity diagram represents color in the XYZ color space (Yxy color space) using (x,y) planar coordinates. In a chromaticity diagram, color is represented by x and y, and the Y information, which indicates lightness, is omitted.

[0035] Figure 3B is an enlarged view of region R in Figure 3A. Figure 3B illustrates a point cloud, i.e., a color distribution, composed of points obtained by plotting the chromaticity coordinates of all pixels contained in the digital sample onto a chromaticity diagram. By plotting the color distribution of the digital sample onto a chromaticity diagram, it is possible to extract only the hue, excluding the lightness information.

[0036] Furthermore, digital specimens created by digitizing HE-stained pathological specimens consist of fewer colors compared to general digital color images, mainly red, blue, and white. Therefore, the point cloud (color distribution) obtained by plotting the chromaticity coordinates of all pixels constituting the digital specimen created by digitizing HE-stained pathological specimens fits within a relatively small region R on the chromaticity diagram, as shown in Figure 3B. The size and shape of the color distribution on the chromaticity diagram change depending on the amount and proportion of dye used in HE staining. Note that the point cloud is not shown in Figure 3A.

[0037] Even when using staining methods other than HE staining, such as EVG staining, MT staining, or PAM staining, the color distribution of the digital specimen will fall within a relatively small area on the chromaticity diagram, similar to the case with HE staining. However, because the hues that appear in the digital specimen differ depending on the staining method, the position of the area on the chromaticity diagram where the color distribution of the digital specimen falls will differ depending on the staining method.

[0038] The color distribution figure generation unit 13 generates a color distribution figure on the chromaticity diagram by approximating the shape of the color distribution in the chromaticity diagram. More specifically, the color distribution figure generation unit 13 generates a color distribution figure on the chromaticity diagram by approximating the figure formed by connecting points located on the outer edge of the point cloud included in the color distribution with straight lines to an n-sided polygon. The color distribution figure is an n-sided polygon (n is an integer of 3 or more). The value of n may be set to a default value in advance, for example, or it may be set as appropriate by the user via the display device 20. Note that if n is large, the accuracy of approximating to an n-sided polygon may decrease. For this reason, it is desirable to set the value of n to, for example, 7 or less.

[0039] Figure 4 is a diagram illustrating the generation of a color distribution figure by the color distribution figure generation unit 13. Figure 4A shows an example where the color distribution exemplified in Figure 3B is approximated, and n=3, i.e., a triangular color distribution figure is generated. Figure 4B shows an example where the color distribution exemplified in Figure 3B is approximated, and n=4, i.e., a quadrilateral color distribution figure is generated.

[0040] The color distribution figure generation unit 13 generates a color distribution figure by calculating the convex or concave hull of the color distribution. The convex hull of the color distribution is the smallest convex polygon on the chromaticity diagram that encloses all the points in the point cloud included in the color distribution. The concave hull of the color distribution is the smallest concave polygon on the chromaticity diagram that encloses all the points in the point cloud included in the color distribution. The color distribution figure generation unit 13 generates a color distribution figure by approximating the shape of the calculated convex or concave hull to an n-sided polygon.

[0041] In the case of n=3, i.e., when the color distribution figure generation unit 13 generates a triangle, since the triangle is always a convex polygon, the color distribution figure generation unit 13 only needs to generate a triangular color distribution figure by calculating the convex hull of the color distribution. In addition, since HE staining stains the target biological tissue in two colors, red and blue (three colors including white), the color distribution figure is easily approximated as a triangle. Even with staining methods other than HE staining, if the staining method stains in three colors, the color distribution figure is easily approximated as a triangle.

[0042] When n=4 or greater, the color distribution figure may be a polygon in which at least one vertex has an interior angle exceeding 180°, as illustrated in Figure 4B. Such a polygon is called a concave polygon. When n=4 or greater, the color distribution figure generation unit 13 can generate the color distribution figure (n-sided polygon) by calculating the convex or concave hull of the color distribution. Alternatively, when n=4 or greater, the color distribution figure generation unit 13 may generate the color distribution figure by calculating the convex hull assuming that the actual color distribution figure is convex, regardless of whether it is actually convex or concave.

[0043] As a method for calculating the convex and concave hulls on a chromaticity diagram, any known method for calculating the convex and concave hulls on a two-dimensional plane can be applied. For example, the alpha shape method can be applied as a method for calculating the concave hull.

[0044] The color distribution figure generation unit 13 may automatically determine whether the color distribution is convex or concave through calculation, or it may display the generated color distribution figure on the display device 20 and make the decision based on the user's selection.

[0045] The vertex detection unit 14 detects the vertices of the color distribution figure generated by the color distribution figure generation unit 13. More specifically, the vertex detection unit 14 detects the positions (coordinates) of the vertices of the color distribution figure on the chromaticity diagram. In this embodiment, the vertices detected by the vertex detection unit 14 are examples of the first vertex, second vertex, or third vertex of this disclosure.

[0046] Furthermore, the vertices of the color distribution figure detected by the vertex detection unit 14 include at least one point with a high saturation of a particular color among all the colors contained in the digital specimen. For example, in the case of a digital specimen created by digitizing a pathological specimen stained with HE (i.e., a digital specimen mainly composed of three colors: red, blue, and white), at least one of the n vertices of the n-sided color distribution figure corresponds to the pixel with the highest red saturation among all the pixels in the digital specimen, at least one corresponds to the pixel with the highest blue saturation among all the pixels in the digital specimen, and at least one corresponds to the pixel with the highest brightness among all the pixels in the digital specimen. Even when a staining method other than HE staining is used to stain the target biological tissue in two colors (three colors including white), similar to the example above, the n vertices of the n-sided color distribution figure include the pixels with the highest saturation of each of the three colors.

[0047] When the color distribution figure generation unit 13 generates a color distribution figure by calculating the convex hull of the color distribution, the vertex detection unit 14 detects one of the vertices of the convex hull as a vertex, which is the maximum point where the distance from the centroid of the convex hull is maximized. Also, when the color distribution figure generation unit 13 generates a color distribution figure by calculating the concave hull of the color distribution, the vertex detection unit 14 extracts n points from the point cloud included in the concave hull such that the area of ​​the figure formed by connecting n points with straight lines is maximized and the figure is a concave polygon, and detects one of the predetermined number of extracted points as a vertex.

[0048] The target point determination unit 15 determines a target point on the chromaticity diagram that will be the target for color correction for at least one of the vertices detected by the vertex detection unit 14. The target point determination unit 15 determines one of the vertices of a target figure that approximates the shape of the color distribution in the chromaticity diagram of the target image, which is the target for color correction of the digital sample, as the target point. The target point in this embodiment is an example of the first target point, second target point, or third target point in this disclosure.

[0049] In the following description, the vertices that the target point determination unit 15 uses to determine the target point may be referred to as target points. The number of target points is, for example, n. Alternatively, the number of target points may be set to an appropriate number of 1 to n by the user's operation, for example, depending on the degree of correction that the user wants to perform on the digital sample to be color corrected. The target point determination unit 15 may set one of the multiple target points as the white point of the D65 light source or the D50 light source.

[0050] The target point determination unit 15 may determine the target point using, for example, a target image selected by the user. The user's operation for determining the target image will be described in detail later in the description of the display device 20. Alternatively, the target point determination unit 15 may determine the target point using, for example, a specific target image pre-stored in the color correction device 10.

[0051] The target point determination unit 15 may select, from among the multiple vertices of the color distribution figure of the digital sample to be color corrected, vertices that are close to each other on the chromaticity diagram and multiple vertices of the target figure as target points and target points corresponding to those target points. Alternatively, the target point determination unit 15 may determine, based on user operation, which vertices of the target figure correspond to which vertices of the color distribution figure as target points.

[0052] The target point determination unit 15 may determine multiple target points for a single target point by using multiple target images for a single digital sample.

[0053] The color correction unit 16 performs color correction on the point cloud corresponding to each pixel of the digital sample on the chromaticity diagram, such that the coordinate value of the target point approaches the coordinate value of the target point corresponding to that target point. If multiple target images are used as targets for color correction of a single digital sample, the color correction unit 16 performs color correction for each target image.

[0054] More specifically, the color correction unit 16 includes a correction coefficient calculation unit 161 that calculates a correction coefficient that transforms the coordinate values ​​of the target point on the chromaticity diagram so that they approach the coordinate values ​​of the corresponding target point.

[0055] Here, the correction coefficient calculation unit 161 calculates one correction coefficient so that each target point approaches its corresponding target point when there are multiple target points. That is, if the first target point, second target point, and third target point are determined for the first target point, second target point, and third target point respectively, the correction coefficient calculation unit 161 calculates one correction coefficient so that the first target point approaches the first target point, the second target point approaches the second target point, and the third target point approaches the third target point. The correction coefficient calculated by the correction coefficient calculation unit 161 is an example of the first correction coefficient of this disclosure.

[0056] Furthermore, if multiple target images are set for a single digital sample, the correction coefficient calculation unit 161 calculates a correction coefficient for each target image.

[0057] The color correction unit 16 then generates a corrected image by multiplying the coordinate values ​​of all points corresponding to each pixel of the digital sample on the chromaticity diagram by a correction coefficient. The color correction unit 16 may also generate a corrected image by applying a LUT (lookup table) based on the correction coefficient. If multiple target images are set for a single digital sample, the color correction unit 16 may generate multiple corrected images by multiplying each target image by a correction coefficient calculated for each target image.

[0058] The corrected image, after color correction by the color correction unit 16, is displayed on the display device 20. In this way, the color correction unit 16 generates a corrected image in which the color of all pixels of the digital sample is corrected to approach the color of the target point. This allows for suitable color correction so that the hue of the digital sample being corrected approaches the hue of the target image.

[0059] Furthermore, the correction coefficient calculation unit 161 can appropriately adopt known calculation methods for calculating the correction coefficient. For example, the correction coefficient calculation unit 161 may calculate the correction coefficient on the xy plane, or it may use the xy coordinate values ​​in the chromaticity diagram (coordinate values ​​in the XYZ color space) as L * a * b *The correction coefficient may be calculated after converting to coordinate values ​​in color space. The correction coefficient calculation unit 161 may, for example, L * a * b * The correction coefficient may be calculated by optimizing the parameters of the transformation formula that converts the pixel values ​​(coordinate values ​​in the RGB color space) of the original digital sample to coordinate values ​​in the XYZ color space, so as to minimize the color difference in the color space.

[0060] Furthermore, the correction coefficient calculation unit 161 may calculate a correction coefficient based on the target point, the target point, and the reference point. The reference point is a point on the chromaticity diagram whose coordinate values ​​are known and which can serve as a reference for color correction. The reference point is, for example, a single color point such as red, blue, or green, and on the chromaticity diagram, it is the point with the highest red saturation (a single red point, i.e., the point corresponding to R=255, B=0, G=0 in the RGB color space), the point with the highest blue saturation (a single blue point, i.e., the point corresponding to R=0, B=255, G=0 in the RGB color space), and the point with the highest green saturation (a single green point, i.e., the point corresponding to R=0, B=0, G=255 in the RGB color space). Other examples of reference points include points with high saturation such as cyan, magenta, or yellow. The correction coefficient calculation unit 161 may calculate a correction coefficient such that the reference point does not change as much as possible before and after the correction. For example, the correction coefficient calculation unit 161 may calculate the correction coefficient by setting the reference point to the same color for both the target point and the reference point.

[0061] Figures 5 and 6 illustrate specific examples of color correction in the color correction device 10. Figure 5A shows an example of a digital sample to be color corrected (hereinafter referred to as the target image). Figure 5B shows an example of the color distribution on the chromaticity diagram and the color distribution figure of the target image in Figure 5A. Figure 5C shows an example of a target image to be color corrected. Figure 5D shows an example of the color distribution on the chromaticity diagram and the color distribution figure of the target image in Figure 5C.

[0062] Figure 6 shows an example of a corrected image (hereinafter referred to as the corrected image) after performing color correction to bring the color tone of the target image shown in Figure 5A closer to that of the target image shown in Figure 5C. Table T1 in Figure 6 shows the coordinate values ​​in the RGB color space of the target image, the coordinate values ​​in the RGB color space of the target image, and the color difference between them. Table T2 in Figure 6 shows the coordinate values ​​in the RGB color space of a pixel in the corrected image, the coordinate values ​​in the RGB color space of the corresponding pixel in the target image, and the color difference between them. As can be seen from the color differences shown in Table T1 and Table T2, the color difference between the corrected image and the target image is significantly smaller than the color difference between the target image and the target image. In this way, the color correction of the color correction device 10 can suitably perform color correction to bring the color tone of the target image closer to that of the target image.

[0063] The registration unit 17 registers the correction coefficient calculated for color correction of a specific digital specimen as a preset. The registered preset (correction coefficient) can be used to perform color correction on other digital specimens. However, it is desirable that other digital specimens to be color corrected using the correction coefficient calculated for the specific digital specimen have a similar color tone to the specific digital specimen. An example of an image with a similar color tone to a specific digital specimen is a digital specimen that has been digitally imaged from another pathological specimen stained under the same staining conditions as the pathological specimen used to generate the specific digital specimen. Having the same staining conditions means that the staining method is the same, and the amount and proportion of the staining solution are the same.

[0064] By registering presets in the registration unit 17, for example, when performing color correction on digital specimens with similar hues, color correction can be performed in a relatively short time, and the processing load related to color correction can be kept relatively low. This configuration of the color correction device 10 is particularly effective when a large number of digital specimens with similar hues are generated for color correction using the same equipment in a specific medical facility.

[0065] The registration unit 17 may register the calculated correction coefficients as separate presets when the target point determination unit 15 determines a target point for each of the multiple target images and the color correction unit 16 calculates a correction coefficient for each of the multiple target images. For example, if a user selects one of the presets via the display device 20, the color correction device 10 may generate a corrected image using the selected preset (correction coefficient). By displaying the corrected images generated for each preset on the display device 20, the user can intuitively understand which of the multiple presets to use to correct the digital sample (target image) to the desired hue.

[0066] <Example of operation of color correction device 10> Referring to Figure 7, an example of the operation of the color correction device 10 during color correction will be explained.

[0067] In step S1, the communication unit 11 acquires the target image from the imaging device 30.

[0068] In step S2, the color distribution acquisition unit 12 acquires the color distribution on the chromaticity diagram from the target image.

[0069] In step S3, the color distribution figure generation unit 13 generates a color distribution figure on the chromaticity diagram by approximating the shape of the color distribution in the chromaticity diagram. The color distribution figure generated in step S3 may or may not be displayed on the display device 20. If the color distribution figure is displayed on the display device 20, the user can confirm the shape of the color distribution figure of the target image.

[0070] In step S4, the vertex detection unit 14 detects the vertices of the color distribution figure.

[0071] In step S5, the target point determination unit 15 determines a target point on the chromaticity diagram for at least one vertex (target point) based on the target image that is the target of color correction.

[0072] In step S6, the correction coefficient calculation unit 161 calculates a correction coefficient that transforms the coordinate values ​​of the target point on the chromaticity diagram so that they approach the coordinate values ​​of the corresponding target point.

[0073] In step S7, the color correction unit 16 generates a corrected image by multiplying the coordinate values ​​on the chromaticity diagram of all pixels in the target image by a conversion coefficient. The display image generated in step S7 is displayed on, for example, the display device 20.

[0074] In step S8, the registration unit 17 registers the correction coefficients corresponding to the corrected image selected by the user as presets.

[0075] This operation allows for accurate color correction to bring the color tone of the target image closer to that of the desired image. In the example shown in Figure 7, steps S7 and S8 describe an example in which the corrected image is displayed on the display device 20 and the correction coefficients corresponding to the corrected image selected by the user are registered as presets, but this disclosure is not limited to this. For example, after calculating the correction coefficients in step S6, all generated correction coefficients may be registered as presets without generating a corrected image.

[0076] As described above, the color correction performed by the color correction device 10 can remove the brightness component by using the coordinate values ​​on the chromaticity diagrams of the target image and the image itself. This reduces the amount of computation performed by the color correction device 10 during color correction. Furthermore, by detecting the vertices of a color distribution figure that approximates the color distribution corresponding to the target image in the chromaticity diagram, the device can extract the point with the highest saturation of a particular color in the chromaticity diagram and correct that point to bring it closer to a similar point in the target image, thereby enabling highly accurate color correction.

[0077] Furthermore, by registering correction coefficients calculated based on a specific digital sample and target image as presets, when other digital samples with similar color tones to a specific digital sample are targeted for color correction, color correction can be performed using the preset correction coefficients. In this case, since it is not necessary to calculate the correction coefficient for each digital sample, the time and processing load related to color correction can be significantly reduced.

[0078] <How to use the color correction device 10> The usage of the color correction device 10, which produces such effects, will be explained with specific examples.

[0079] (First mode of use) For example, a pathologist working at medical facility A may make a diagnosis using a digital specimen generated at another medical facility B. If the staining conditions for the original pathological specimens used to create the digital specimens differ between medical facility A and medical facility B, the color tones of the digital specimens may differ significantly.

[0080] In such cases, the pathologist can make a highly accurate diagnosis because they are familiar with the digital specimen S_A generated at medical facility A, but they may take longer to diagnose the digital specimen S_B generated at medical facility B because they are unfamiliar with it.

[0081] Here, by using the color correction device 10 to perform color correction on the digital specimen S_B with the digital specimen S_A as the target image, a corrected image can be generated in which the color tone of the digital specimen S_B is brought closer to the color tone of the digital specimen S_A. This prevents situations where diagnosis is delayed due to differences in the color tone of the digital specimens.

[0082] Furthermore, by registering correction coefficients as presets for when digital specimen S_B is the target of color correction and digital specimen S_A is the target image, the color correction device 10 can generate corrected images for all digital specimens S_B relatively quickly and with low load, even when a large number of digital specimens S_B are sent from, for example, medical facility B, by using the presets.

[0083] The above example described the case where there are two medical facilities, A and B, but the same effect can be obtained even when there are three or more medical facilities. When there are three or more medical facilities, a preset can be registered for each medical facility, so it becomes very useful because color correction can be easily performed regardless of which medical facility requests a diagnosis from later on.

[0084] This method can achieve similar results not only in cases involving multiple medical facilities, but also in situations where multiple departments within the same medical facility have different staining conditions.

[0085] (Second mode of use) Artificial intelligence (AI) is being developed that uses digital specimens as training data to enable it to output diagnostic support information when digital specimens are input. In order to train the AI ​​to output highly accurate support results, it is effective to use a large amount of digital specimens as training data. Collecting a large amount of digital specimens is effective by collecting them from multiple medical facilities, but it is difficult to train the AI ​​to output highly accurate diagnostic support information if the training data contains a mixture of digital specimens with different color tones.

[0086] By using the color correction device 10 to color-correct multiple digital specimens acquired from multiple medical facilities to approximate the color of a digital specimen generated at a specific medical facility, a large number of digital specimens with similar color tones can be easily prepared. Using such a large number of digital specimens with similar color tones as training data, the AI ​​can be trained to output highly accurate diagnostic support information. Thus, the color correction device 10 is extremely useful for preparing large amounts of training data.

[0087] By registering correction coefficients as presets for multiple medical facilities, it becomes easier to prepare a larger number of digital samples. Furthermore, by using corrected images that match the color tones of digital samples generated at a specific medical facility as training data, it is possible to perform learning processing that is specific to that medical facility.

[0088] Furthermore, by registering correction coefficients calculated using digital specimens with distinctive color tones as target images as presets, and by preparing a large number of digital specimens with distinctive color tones as training data, the robustness of the AI ​​can be enhanced. Specifically, for example, in digital specimens generated from HE-stained pathological images, a correction coefficient can be calculated and registered using a digital specimen with a particularly strong blue color as the target image. By using this preset and training the AI ​​with digital specimens with a strong blue color as training data, the AI ​​can be trained to output diagnostic support information with high accuracy even when digital specimens with a strong blue color are input. By performing similar processing for red and white, the AI ​​can be trained to output diagnostic support information with relatively high accuracy regardless of the type of digital specimen input.

[0089] <Configuration of display device 20> Next, the configuration of the display device 20 will be described. Figure 8 is a block diagram showing an example of the functional configuration of the display device 20.

[0090] The communication unit 21 communicates with the color correction device 10. The communication unit 21 receives various information necessary for display transmitted from the color correction device 10 and transmits information related to user operations to the color correction device 10.

[0091] The display processing unit 22 performs various display processing for color correction based on various information received from the color correction device 10. Details of the display processing performed by the display processing unit 22 will be described later. The display processing unit 22 performs display processing on various display devices, such as liquid crystal displays and organic EL (Electro-Luminescence) displays.

[0092] The operation reception unit 23 accepts various operations from the user. The operation reception unit 23 accepts user operations via various operation devices, such as buttons, keyboards, mice, trackballs, and touchpads. A so-called touch panel may be configured by superimposing a display device, where the display processing unit 22 performs display processing, and a touchpad, which acts as an operation device. Details of the operations accepted by the operation reception unit 23 will be explained later in the description of the display processing.

[0093] <Details of display processing> The display processing performed by the display processing unit 22 will be described in detail below.

[0094] When the color correction device 10 acquires a digital sample from the imaging device 30, the display processing unit 22 displays an initial setup screen 510 on the display device for performing initial settings related to color correction in the color correction device 10. Figure 9 shows an example of the initial setup screen 510.

[0095] In the example shown in Figure 9, the initial setup screen 510 includes a reading unit 511, a target selection unit 512, an initial setup unit 513, and a setup completion unit 514.

[0096] The reading unit 511 is a button that accepts user input to read information about new digital specimens that are candidates for color correction. When the user operates the reading unit 511 via the operation reception unit 23, the display device 20 obtains information about the new digital specimens to be color corrected from, for example, the color correction device 10 and displays it on the target selection unit 512.

[0097] The target selection unit 512 displays a list of digital samples that are candidates for color correction. The target selection unit 512 also accepts an operation by the user to select a digital sample from among multiple digital samples to be color corrected by the color correction device 10. When the user selects one of the multiple digital samples displayed on the target selection unit 512 via the operation reception unit 23, the display device 20 transmits information indicating the digital sample selected by the user to the color correction device 10. In the example shown in Figure 9, only two digital samples are displayed on the target selection unit 512, but in reality, more digital samples may be displayed on the target selection unit 512. The user may select multiple digital samples using the target selection unit 512. This allows the color correction device 10 to use the digital samples selected by the user as candidates for color correction points and start generating color distribution figures, detecting vertices, etc.

[0098] In the example shown in Figure 9, the target selection unit 512 displays a string indicating the date of shooting along with the candidate digital specimens. In addition to the shooting date, metadata of the images may also be displayed in the target selection unit 512. Furthermore, the target selection unit 512 may display the model name of the device used to photograph the digital specimens, the name of the facility where the staining or photography was performed, etc. In addition, the target selection unit 512 may indicate whether or not each digital specimen has undergone color correction using a color chart.

[0099] The initial setting unit 513 is an area that displays information regarding the initial settings of color correction. The initial setting unit 513 also accepts user operations to perform the initial settings of color correction. In the example shown in Figure 9, three elements are shown as initial setting elements: "Generate an n-sided color distribution figure," "Detect while ignoring convex and concave parts," and "Detection power of the distribution." "Generate an n-sided color distribution figure" accepts an operation to determine whether the color correction device 10 should generate an n-sided color distribution figure when generating a color distribution figure on the chromaticity diagram. Note that it may be configured to allow selection of multiple n at once. "Detect while ignoring convex and concave parts" accepts an operation to determine whether the color correction device 10 should detect the vertices of the color distribution figure on the chromaticity diagram as convex polygons, even if the shape of the color distribution figure is approximated as a concave polygon. "Detection power of the distribution" accepts an operation to select the degree to which outliers should be ignored when calculating the convex hull of the color distribution figure. When the detection power is set to a large value, the color correction device 10 calculates the convex hull using the point cloud further inside the color distribution figure.

[0100] When a user performs initial settings in the initial settings unit 513 via the operation reception unit 23, the display device 20 transmits information regarding the initial settings corresponding to the user's operation to the color correction device 10. This allows the color correction device 10 to start color correction based on the user's initial settings.

[0101] Note that the elements of the initial settings in the initial setting unit 513 shown in Figure 9 are just examples, and this disclosure is not limited thereto. The initial settings are performed in order to accurately detect the color distribution in the color distribution device 10, and initial settings may be performed for elements other than those shown in Figure 9.

[0102] The setting completion unit 514 is a button that accepts an operation from a user who has completed the selection of the color correction target in the target selection unit 512 and the initial settings in the initial settings unit 513, indicating that the initial settings are complete. When the user operates the setting completion unit 514, the color correction device 10 performs the generation of a color distribution figure and vertex detection on the selected digital sample (target image).

[0103] The display device 20 obtains information about the color distribution from the color correction device 10, which performed the generation of a color distribution figure and vertex detection, including a point cloud in which the coordinate values ​​of multiple pixels included in the selected digital sample are plotted on a chromaticity diagram.

[0104] Based on information regarding the color distribution, the display device 20 displays the color distribution display screen 520 on the display device. Figure 10 shows an example of the color distribution display screen 520. The color distribution display screen 520 includes a color distribution display unit 521, a target image display unit 522, a sample selection unit 523, and a determination unit 524.

[0105] The color distribution display unit 521 displays a chromaticity diagram that includes a point cloud (color distribution) in which the coordinate values ​​of the pixels of the target image are plotted. The target image display unit 522 displays a list of digital specimens (target images) that are subject to color correction by the color correction device 10. Along with the target images, the target image display unit 522 may also display a string indicating the date of capture, image metadata, the model name of the device that captured the digital specimen, and the name of the facility that performed the staining or capture. Furthermore, the target image display unit 522 may also indicate whether or not each target image has been color corrected using a color chart.

[0106] On the color distribution display screen 520, the user can select one of the digital samples displayed on the target image display unit 522 by operating the sample selection unit 523 (described later) via the operation reception unit 23. The color distribution display unit 521 displays the color distribution of the target image selected by the user using the sample selection unit 523.

[0107] The sample selection unit 523 is a display that indicates whether the user has selected one of the multiple target images displayed on the target image display unit 522. In the example shown in Figure 10, the sample selection unit 523 is composed of a thick line surrounding one of the target images displayed on the target image display unit 522. The user can select the desired image by moving the sample selection unit 523 via the operation reception unit 23 so that the thick line surrounds one of the images. In this disclosure, the sample selection unit 523 does not have to be in this shape; it may be any shape that clearly allows the user to understand which image they are selecting, such as an arrow.

[0108] The sample selection unit 523 may be configured such that, for example, it is not displayed in the color distribution display screen 520 in the initial state, and only appears in the color distribution display screen 520 when, for example, the user operates the operation reception unit 23 while the color distribution display screen 520 is displayed on the display device.

[0109] The user can visually confirm the shape of the color distribution for each target image by referring to the color distribution displayed on the color distribution display unit 521. The user may also perform an operation via the operation reception unit 23 to select whether the color distribution figure approximating the outline of the color distribution is a convex polygon or a concave polygon. If the user performs an operation to select the concavity or concavity of the color distribution figure, the display device 20 transmits information regarding the result of the selection to the color correction device 10. Based on the user's selection, the color correction device 10 can then detect the vertices of the color distribution figure by calculating either a convex or concave hull. However, this disclosure is not limited to this embodiment; the color correction device 10 may automatically determine the concavity or concavity based on the shape of the color distribution figure and detect the vertices, regardless of the user's selection.

[0110] Furthermore, the color distribution table screen 520 may also include a selection unit for the user to refer to the color distribution displayed on the color distribution display unit 521 and, if it determines that the referenced image is not suitable for color correction, to exclude that image from the target images.

[0111] Furthermore, the color distribution display screen 520 may be configured so that a user who has referred to the color distribution displayed on the color distribution display unit 521 can re-select the value of n (a value indicating the vertices of the polygon that approximates the color distribution figure in color correction) that was selected on the initial setting screen 510 via the operation reception unit 23.

[0112] The decision unit 524 is a button that accepts an operation from a user who has referred to the color distribution display screen 520 to advance the color correction process of the color correction device 10 to the next step. When a user who has referred to the color distribution figure of the target image displayed on the color distribution display screen 520 and has decided to proceed with the process operates the decision unit 524, the color correction device 10 proceeds to the next step, the detection of vertices of the color distribution figure. Once the detection of vertices in the color correction device 10 is complete, the display device 20 receives information from the color correction device 10 regarding the coordinate values ​​of the vertices (target points) of the color distribution figure.

[0113] Based on information regarding the coordinate values ​​of the vertices, the display device 20 causes the vertex position display screen 530 to be displayed on the display device. Figure 11 shows an example of the vertex position display screen 530. The vertex position display screen 530 includes a vertex position display unit 531 and a digital sample display unit 532.

[0114] The vertex position display unit 531 displays a chromaticity diagram plotting the positions of the vertices. In the example shown in Figure 11, multiple vertices detected in multiple digital samples are plotted on a single chromaticity diagram. However, this disclosure is not limited to this, and the vertex position display unit 531 may, for example, display only the vertices detected from any part of the digital samples to be color corrected.

[0115] The digital sample display unit 532 is an area that displays a list of multiple digital samples in which the vertices displayed on the vertex position display unit 531 have been detected. In the example in Figure 11, the digital sample display unit 522 allows selection of detected vertices for both the case of n=3 and the case of n=4, and the state in which the case of n=4 has been selected is shown. In the digital sample display unit 522 in Figure 11, the tabs labeled "n=3" and "n=4" are the areas where selection is accepted. If the user performs an operation via the operation reception unit 23 to select, for example, the tab labeled "n=3", the vertices displayed on the chromaticity diagram of the vertex position display unit 531 will also be the vertices corresponding to the case of n=3.

[0116] On the vertex position display screen 530, the user can select any of the vertices displayed on the vertex position display screen 531 by operating the point selection unit 533 (described later) via the operation reception unit 23. The vertex position display screen 530 may also be configured to enlarge a portion of the chromaticity diagram based on the user's operation. This makes it easier for the user to select any of the vertices even when vertex positions are close to each other in a portion of the chromaticity diagram.

[0117] Figure 12 shows the point selection unit 533 displayed on the vertex position display screen 530. The point selection unit 533 is a display object that allows the user to freely move within the vertex position display unit 531 via the operation reception unit 23 to select one of the vertices displayed in the vertex position display unit 531. The point selection unit 533 may be configured not to be displayed in the vertex position display screen 530 in the initial state, for example, and to only be displayed in the vertex position display screen 530 when the user operates the operation reception unit 23 while the vertex position display screen 530 is displayed on the display device.

[0118] In the example shown in Figure 12, the point selection unit 533 has a wedge shape, and the user can select a desired vertex by moving the point selection unit 533 via the operation reception unit 23 so that the tip of the wedge shape points to one of the vertices. In this disclosure, the point selection unit 533 does not need to have a wedge shape; any shape that clearly indicates the vertex to be indicated, such as an arrow, is acceptable.

[0119] When a user selects a vertex using the point selection unit 533, the digital sample display unit 532 displays a corresponding pixel indicator unit 534 that indicates which pixel of which digital sample the selected vertex corresponds to. In the example shown in Figure 12, the corresponding pixel indicator unit 534 has a wedge shape similar to that of the point selection unit 533, but the disclosure is not limited to this, and any shape that clearly indicates the location to be indicated, such as an arrow, may be used. Furthermore, the point selection unit 533 and the corresponding pixel indicator unit 534 may have different shapes from each other.

[0120] The vertex position display screen 530 may also include a vertex display area for displaying a list of vertices selected by the user using the point selection unit 533. The user may also be allowed to further select any of the vertices displayed in the vertex display area.

[0121] In the example shown in Figure 12, the digital sample display unit 532 displays a thumbnail-sized digital sample, and the corresponding pixel indicator unit 534 is displayed on the thumbnail-sized digital sample. To allow the user to more accurately understand which pixel of the digital sample corresponds to the selected vertex, the digital sample display unit 532 may be capable of displaying the digital sample in a magnified view, and the corresponding pixel indicator unit 534 may be displayed on the magnified digital sample. Alternatively, when the user selects a vertex using the point selection unit 533 on the vertex position display screen 530, the display device 20 may display a magnified digital sample containing the pixels corresponding to the selected vertex on a separate screen. In this case, the corresponding pixel indicator unit 534 only needs to be displayed on the magnified digital sample on the separate screen.

[0122] The user can accurately determine which vertex selected by the point selection unit 533 corresponds to which pixel of which digital sample, using the corresponding pixel indicator unit 534. This allows the user to confirm whether the vertices of the color distribution figure detected during the color correction of the digital sample performed by the color correction device 10 contain the points with the highest saturation of a particular color (red, blue, white, etc.) on the chromaticity diagram.

[0123] Furthermore, on the vertex position display screen 530, the user can select one of the digital specimens displayed on the digital specimen display unit 532 by operating the specimen selection unit 535, which will be described later, via the operation reception unit 23. As shown in Figure 13, in addition to the digital specimen, the digital specimen display unit 532 may also display color patches representing the color information of the detected vertices. The number of color patches displayed should be equal to the number of vertices in the color distribution figure, that is, the value of n set in the initial setup screen 510. For example, in the case of a digital specimen stained with HE, if vertex detection is successful, at least three patches corresponding to red, blue, and white will be displayed.

[0124] Furthermore, the user may be allowed to select one of the color patches displayed on the digital specimen display unit 532. Since selecting a color patch is equivalent to selecting a vertex, the user can select a vertex in a more intuitive way on the vertex position display screen 530.

[0125] Figure 13 shows the sample selection unit 535 displayed on the vertex position display screen 530. The sample selection unit 535 may be configured not to be displayed on the vertex position display screen 530 in the initial state, for example, but to be displayed only when the user operates the operation reception unit 23 while the vertex position display screen 530 is displayed on the display device.

[0126] The specimen selection unit 535 is a display that indicates whether the user has selected one of the multiple digital specimens displayed on the digital specimen display unit 532. In the example shown in Figure 13, the specimen selection unit 535 is composed of a thick line surrounding one of the digital specimens displayed on the digital specimen display unit 522. The user can select a desired image by moving the specimen selection unit 535 via the operation reception unit 23 so that the thick line surrounds one of the images. In this disclosure, the specimen selection unit 535 does not have to be in this shape; it may be any shape that clearly indicates the image being indicated, such as an arrow.

[0127] When a user selects a digital sample using the sample selection unit 535, the vertex position display unit 531 shows the positions of the vertices of the color distribution figure of the selected digital sample on the chromaticity diagram. Since the number of vertices in the color distribution figure is n, which is set in the initial setting screen 510 or the color distribution display screen 520, the vertex position display unit 531 displays n points, each representing a different vertex. Alternatively, when a user selects a digital sample using the sample selection unit 535, as shown in Figure 13, the vertex position display unit 531 may display a point cloud (color distribution) obtained by plotting the chromaticity coordinates of at least some of the pixels of the selected digital sample, and then display the vertices of the color distribution figure.

[0128] The user can accurately determine the position of the vertices in the digital sample selected by the sample selection unit 535 using the vertex position display unit 531. This allows the user to confirm whether the vertices of the color distribution figure detected by the color correction device 10 include the point with the highest saturation of a particular color (red, blue, white, etc.) on the chromaticity diagram.

[0129] The determination unit 536 is a button that, after the user has selected a vertex and confirmed the vertex position for each digital sample, accepts an operation to proceed with color correction in the color correction device 10 via the operation reception unit 23. When the determination unit 536 is operated, the display device 20 transmits information to the color correction device 10 indicating that an operation to proceed with color correction has been accepted. This allows the color correction device 10 to proceed to the next step in the color correction process.

[0130] Furthermore, the vertex position display screen 530 may be configured to accept an operation from a user who has checked the vertex position to set another point adjacent to that vertex on the chromaticity diagram as a new vertex if the user determines that the vertex is not suitable as a vertex in the color distribution figure. In this case, the display device 20 transmits information to the color correction device 10 indicating that the vertex is not suitable. As a result, the color correction device 10 can discard the current vertex and set another point adjacent to that vertex on the chromaticity diagram as a new vertex.

[0131] Next, the display device displays a target image selection screen 540, which accepts the selection of a target image to match the color tone of the digital sample to be color-corrected, to the display device 20. Figure 14 shows an example of the target image selection screen 540.

[0132] The target image selection screen 540 includes a target point candidate display unit 541 and a target point setting unit 542. The target point candidate display unit 541 displays a list of target image candidates and also displays the color patches of the target point candidates included in the target image candidates. The target image candidates may be stored in the display device 20 in advance, for example, or they may be transmitted to the display device 20 from the color correction device 10 as appropriate. In the example shown in Figure 14, a direct specification button is installed inside the target point candidate display unit 541. The direct specification button is a button for specifying a target point candidate, for example, by directly specifying coordinate values.

[0133] The target point setting unit 542 is an area for selecting a target point from among the color patches of target point candidates included in multiple target images displayed in the target point candidate display unit 541. The user sets a target point by referring to the color patches of target point candidates displayed in the target point candidate display unit 541 and selecting the desired target point candidate from the list. As a specific example of operation, as shown by the white arrow in Figure 14, the user drags the color patch of a target point candidate to the corresponding location in the target point setting unit 542 via the operation reception unit 23, thereby setting the dragged target point candidate as the target point. The user can set n target points by repeating this operation n times. Figure 14 shows an example where n=3. In addition, the target point setting unit 542 also displays the color patch of the target point, and the user can set the target point while referring to the color patch of the target point as well.

[0134] As shown in Figure 14, the target image selection screen 540 allows the user to set a desired target point that will bring the coordinate values ​​of the target point closer to the desired target point while viewing the color patches of the candidate target points. It is not necessary for all n target points to be selected from the same target image; the user can set target points with the desired coordinate values ​​as appropriate while viewing the color patches of candidate target points from multiple target images.

[0135] When the user selects n target points using the target point setting unit 542, the display device 20 transmits information about the selected target points to the color correction device 10. This allows the color correction device 10 to calculate a correction coefficient based on the selected target points.

[0136] The target image selection screen 540 may be configured to allow the user to select multiple target images at once. In this case, the color correction device 10 may perform color correction on the digital sample to be color corrected to match the hue of each of the selected target images, either in parallel or sequentially.

[0137] The back button 543 is a button that accepts the user's operation to interrupt the setting of the target point and return to the previous screen (such as the vertex position display screen 530). The confirm button 544 is a button that accepts the user's operation to advance the color correction process of the color correction device 10 to the next step after the setting of the target point in the target point setting unit 542 is complete.

[0138] When the confirmation button 544 is pressed, the display device 20 transmits information about the set target point to the color correction device 10. The color correction device 10 calculates a correction coefficient to bring the target point of the digital sample closer to the corresponding target point. If multiple target images are selected, the color correction device 10 calculates a correction coefficient for each target image. The color correction device 10 transmits to the display device 20 information about the target point determined based on the target image and information about the calculated correction coefficient.

[0139] The color correction device 10 does not necessarily have to determine all of the multiple target points based on the target image selected by the user. For example, the color correction device 10 may determine the target point corresponding to white based on the pixel with the highest brightness in the target image, or it may determine it to the point with the highest brightness coordinates (255,255,255) in the RGB color space. Furthermore, when the color correction device 10 directly specifies the pixel value (coordinate) of the target point, it is not limited to the RGB color space, but may also directly specify coordinates in the XYZ color space, for example.

[0140] The display device 20, having received information regarding the correction coefficient from the color correction device 10, displays a result confirmation screen 550 on the display device for the user to confirm the correction result using the said correction coefficient. Figure 15 shows an example of the result confirmation screen 550. In the example shown in Figure 15, the result confirmation screen 550 includes a target point color display unit 551, a corrected color display unit 552, a color difference display unit 553, a chromaticity diagram display unit 554, a preview button 555, a registration button 556, and a cancel button 557.

[0141] The target point color display unit 551 displays the color (pre-correction color) of the vertices (target points) of the digital sample to be color corrected. In the example shown in Figure 15, an example is shown where the number of vertices (n) is 3. Therefore, the target point color display unit 551 includes a section where the color of the first target point is displayed as "vertex 1", a section where the color of the second target point is displayed as "vertex 2", and a section where the color of the third target point is displayed as "vertex 3".

[0142] The corrected color display unit 552 displays the color of the target point after color correction has been performed based on the target point (corrected color). In the example shown in Figure 15, the corrected color display unit 552 includes a section where the corrected color of the first target point is displayed as "Vertex 1", a section where the corrected color of the second target point is displayed as "Vertex 2", and a section where the corrected color of the third target point is displayed as "Vertex 3".

[0143] The color difference display section 553 is a field that displays the color difference between the pre-correction and post-correction values ​​for each vertex. The target point color display section 551, the post-correction color display section 552, and the color difference display section 553 allow the user viewing the result confirmation screen 550 to compare and confirm the color of the target point of the digital sample subject to color correction with the color of the result after color correction.

[0144] The chromaticity diagram display unit 554 displays a chromaticity diagram. The chromaticity diagram display unit 554 shows the positions of the target points before and after correction, as shown in the target point color display unit 551 and the corrected color display unit 552. In the example shown in Figure 15, the position of the target points before correction is shown as ○, and the position of the target points after correction is shown as △. The chromaticity diagram display unit 554 may also display which vertex position each position corresponds to.

[0145] Furthermore, on the results confirmation screen 550, in addition to the corrected color display unit 552, or instead of the corrected color display unit 552, a color patch of the target point corresponding to the target point (the color patch of the target point selected in Figure 14) may be displayed. The color patch of the target point corresponding to the target point may be displayed on the chromaticity diagram in the chromaticity diagram display unit 554, for example. In this case, the color difference display unit 553 may display the color difference between the target point and the target point before correction. Also, the color difference display unit 553 may display at least one of three types of color differences: the color difference between the target point and the target point after correction, the color difference between the target point and the target point before correction, and the color difference between the target point and the target point after correction. If multiple types of color differences are displayed on the color difference display unit 553, it may further display which two points each color difference represents.

[0146] The preview button 555 is a button that accepts user input to display the preview screen 560. When the user operates the preview button 555 via the operation reception unit 23, the display device 20 displays the preview screen 560 on the display device.

[0147] The preview screen 560 displays a corrected image generated by color-correcting a digital sample to be color-corrected so that the color of the target point displayed in the target point color display unit 551 approaches the color of the target point displayed in the corrected color display unit 552. Figure 16 shows an example of the preview screen 560.

[0148] The preview screen 560 includes a target image display unit 561, a target image display unit 562, a corrected image display unit 563, and a back button 564. The target image display unit 561 displays the digital sample (target image) that is subject to color correction. The target image display unit 562 displays the target image used for color correction. If the target point is selected from multiple target images, the target image display unit 562 may display one target image as a representative, or it may display all target images corresponding to all target points within the target image display unit 562. If the coordinates of the target point are specified directly by the user, for example, the target image does not need to be displayed in the target image display unit 562.

[0149] The corrected image display unit 563 displays a corrected image obtained by color-correcting the digital sample displayed in the target image display unit 561 using a correction coefficient calculated using the target image displayed in the target image display unit 562. The back button 564 is a button that, when operated by the user via the operation reception unit 23, causes the display device 20 to return the display device to the result confirmation screen 550.

[0150] In the example shown in Figure 16, the preview screen 560 is a separate screen from the results confirmation screen 550, but the disclosure is not limited to this. For example, the preview screen 560 may be displayed as part of the results confirmation screen 550. Alternatively, the preview screen 560 may be overlaid on the results confirmation screen 550.

[0151] The preview screen 560 shown in Figure 16 is an example. For example, the preview screen 560 may allow for simultaneous magnification of corresponding areas of the target image and the corrected image, and display them side by side. This is preferable as it allows the user to check how the details of the image have changed due to the correction.

[0152] Returning to the explanation of Figure 15, the registration button 556 is a button that accepts an operation to register the correction coefficient used to bring the color of the corrected target point closer to the color displayed on the corrected color display unit 552 as a preset in the color correction device 10. When the user operates the registration button 556 via the operation reception unit 23, the display device 20 transmits information to the color correction device 10 indicating that permission for registration has been obtained from the user. As a result, the color correction device 10 can register the selected correction coefficient as a preset.

[0153] The cancel button 557 is a button that the user operates when they do not want to register the correction coefficient used to correct the color to the color displayed on the corrected color display unit 552 as a preset in the color correction device 10. When the user operates the registration button 556 via the operation reception unit 23, the display device 20 transmits information to the color correction device 10 indicating that permission for registration was not obtained from the user. In this case, the color correction device 10 discards the corresponding correction coefficient.

[0154] Subsequently, if the display device 20 has already received a correction result from the color correction device 10 using a correction coefficient calculated from another target image, it may update the result confirmation screen 550 to allow the user to confirm the correction result. Alternatively, the display device 20 may allow the user to select a new target image by displaying a target image selection screen 540 on the display device, and then transmit information about the selected target image to the color correction device 10 to calculate a correction coefficient based on the new target image. The display device 20 may also choose not to display in the target image selection unit 542 the target image used to calculate a correction coefficient that the user did not authorize registration of.

[0155] The result confirmation screen 550 and preview screen 560 allow the user to easily and accurately understand what kind of corrected image will be obtained if color correction is performed on the currently selected target image. Furthermore, after confirming the corrected image on the preview screen 560, the user can decide on the result confirmation screen 550 whether to register the correction coefficients that generate the corrected image as a preset. This allows the color correction device 10 to register correction coefficients that can correct to the user's desired color tones as presets.

[0156] The initial setup screen 510, color distribution display screen 520, vertex position display screen 530, target image selection screen 540, result confirmation screen 550, and preview screen 560 described above are examples of display screens in this disclosure.

[0157] <Variation> The embodiments of this disclosure have been described above. This disclosure is not limited to the embodiments described above, and various modifications are possible.

[0158] In the above-described embodiment, it was explained that the color correction device 10 may receive a digital sample generated by the imaging device 30 that has been subjected to known color correction using a color chart or the like by another color correction device (not shown). In this example, a different type of color correction using a color chart or the like is performed before the color correction in the color correction device 10.

[0159] This disclosure is not limited thereto. For example, a color correction device may perform a second color correction using a pre-acquired color chart after performing a first color correction using a correction coefficient calculated based on a target image.

[0160] Specifically, the process is as follows: The imaging device captures a predetermined color chart along with the pathological specimen, and the color correction device acquires an image of the color chart along with the digital specimen. The color correction device calculates a first correction coefficient based on the target image using the method described in the above-described embodiment. The color correction unit of the color correction device then performs color correction (first color correction) on the digital specimen using the first correction coefficient to generate a first corrected image.

[0161] The color correction unit creates a color chart profile based on a previously acquired image of the color chart and the colorimetric values ​​of the color chart. Then, the color correction unit applies the created profile to the first corrected image to perform a second color correction and generate a second corrected image.

[0162] In this embodiment, the second color correction is expected to correct for deviations in the brightness direction that may occur in digital specimens due to differences in shooting conditions in the imaging device. In the color correction by the color correction device 10 described above (first color correction), differences in hue excluding the brightness component of the digital specimen are corrected, but by adding a second color correction that can also correct in the brightness direction, more accurate color correction can be achieved.

[0163] <Example hardware configuration> In the embodiments described above, the color correction device 10 and the display device 20 were described as computers. Below, an example of a computer hardware configuration for realizing the functions of the color correction device 10 or the display device 20 will be described.

[0164] Figure 17 shows an example of a computer hardware configuration for realizing the functions of the color correction device 10 or the display device 20.

[0165] Computer 1000 includes an input device 1001, an output device 1002, a CPU 1003, a ROM (Read Only Memory) 1004, a RAM (Random Access Memory) 1005, a storage device 1006, a reader 1007 for reading information from various storage media, and a transceiver 1008, with each part connected by a bus 1009.

[0166] The reading device 1007 reads a program from a recording medium that contains a program for realizing the functions of each component of the color conversion system, and stores it in the storage device 1006. Alternatively, the transmitting / receiving device 1008 communicates with a system device connected to the network and stores a program downloaded from the system device in the storage device 1006.

[0167] Then, the CPU 1003 copies the program stored in the memory device 1006 to the RAM 1005, and sequentially reads and executes the instructions contained in that program from the RAM 1005, thereby realizing the functions of the color correction device 10 or the display device 20. [Industrial applicability]

[0168] This disclosure is useful for a color correction device that can perform color correction on an image. [Explanation of Symbols]

[0169] 100 Color Correction System 10 Color Correction Device 11 Communications Department 12 Color distribution acquisition section 13. Color Distribution Figure Generation Unit 14 Vertex detection unit 15. Target Point Determination Section 16 Color Correction Section 161 Correction coefficient calculation unit 17 Registration Department 20 Display device 21 Communications Department 22 Display Processing Unit 23 Operation Reception Section 30,30_A,30_B,30_C Imaging device

Claims

1. A color distribution figure generation unit approximates the shape of a color distribution including a point cloud obtained by plotting the coordinate values ​​of multiple pixels contained in an image on a chromaticity diagram, and generates a color distribution figure on the chromaticity diagram. A vertex detection unit for detecting the first vertex of the color distribution figure, A target point determination unit that determines a first target point for color correction of the image corresponding to the first vertex, A color correction unit that performs color correction to bring the coordinate values ​​of the first vertex closer to the coordinate values ​​of the first target point on the chromaticity diagram, A color correction device, including one.

2. The system further includes a correction coefficient calculation unit that calculates a first correction coefficient that brings the coordinate values ​​of the first vertex closer to the coordinate values ​​of the first target point. The color correction device according to claim 1.

3. The correction coefficient calculation unit calculates the first correction coefficient based on the first vertex, the first target point, and a reference point located outside the color distribution figure on the chromaticity diagram. The color correction device according to claim 2.

4. On the chromaticity diagram, each of the multiple reference points is located in a position that encloses the color distribution figure. The color correction device according to claim 3.

5. The vertex detection unit further detects a second vertex, The target point determination unit further determines a second target point for color correction of the second vertex, The color correction unit further performs color correction to bring the coordinate values ​​of the second vertex closer to the coordinate values ​​of the second target point. The color correction device according to claim 2.

6. The correction coefficient calculation unit calculates the first correction coefficient that brings the coordinate value of the first vertex closer to the coordinate value of the first target point, and brings the coordinate value of the second vertex closer to the coordinate value of the second target point. The color correction device according to claim 5.

7. The vertex detection unit further detects a third vertex, The target point determination unit further determines a third target point for color correction of the third vertex, The color correction unit further performs color correction to bring the coordinate values ​​of the third vertex closer to the coordinate values ​​of the third target point. The color correction device according to claim 6.

8. The third target point is the white point of a D65 light source or a D50 light source. The color correction device according to claim 7.

9. The target point determination unit sets one of the vertices of a target figure that approximates the shape of the color distribution in the chromaticity diagram of the target image, which is the target of the color correction of the image, as the first target point. The color correction device according to claim 1.

10. The above image is of biological tissue stained with hematoxylin-eosin staining solution. The color correction device according to claim 1.

11. The aforementioned image has been color-corrected using a color chart before the color correction performed by the color correction unit. The color correction device according to claim 1.

12. The color distribution figure generation unit generates the color distribution figure by calculating the convex hull of the color distribution, The vertex detection unit detects one of the vertices of the convex hull that is a local maximum point where the distance from the centroid of the convex hull is maximized, as the first vertex. The color correction device according to claim 1.

13. The color distribution figure generation unit generates the color distribution figure by calculating the concave hull of the color distribution, The vertex detection unit extracts a predetermined number of points from the point cloud contained in the concave hull such that the area of ​​a figure formed by connecting a predetermined number of points with straight lines is maximized and the figure is a concave polygon, and detects one of the extracted predetermined number of points as the first vertex. The color correction device according to claim 1.

14. The color correction unit further performs a second color correction on the color-corrected image after the color correction has been performed by applying a second correction coefficient obtained based on the captured image of the color chart and the colorimetric values ​​of the color chart. The color correction device according to claim 3.

15. The coordinate values ​​of the vertices of a color distribution figure that approximates the color distribution, which includes a point cloud formed by plotting the coordinate values ​​of multiple pixels contained in the image onto a chromaticity diagram, are obtained. The color distribution and a point selection unit that accepts the selection of vertices included in the color distribution are displayed on the display screen. When the selection of the aforementioned vertex is accepted, the position of the selected vertex in the image is displayed on the display screen. Display device.

16. The display screen displays at least one of the following: the color distribution figure in a single chromaticity diagram, the positions of the vertices in the color distribution figure, and a sample selection unit that accepts the selection of any of the multiple images. When the selection of the aforementioned image is received, the vertices of the color distribution figure of the selected image are displayed on the single chromaticity diagram. The display device according to claim 15.

17. The color of the vertex selected in the above image, The position of the selected vertex color on the chromaticity diagram, When color-correcting the aforementioned image, the color of the target point to bring the vertices closer together, The position of the target point on the chromaticity diagram, A preview button that accepts requests for preview display is displayed on the aforementioned display screen, When the aforementioned request is received, the color-corrected image that has undergone the color correction is displayed on the display screen. The display device according to claim 15.

18. A color correction method performed by a computer, The aforementioned computer, A color distribution figure is generated that approximates the color distribution, including a point cloud formed by plotting the coordinate values ​​of multiple pixels contained in the image onto a chromaticity diagram. The first vertex of the aforementioned color distribution figure is detected, Determine a first target point for color correction of the image that corresponds to the first vertex, The color correction is performed to bring the coordinate values ​​of the first vertex closer to the coordinate values ​​of the first target point. Color correction methods.

19. A program executed by a computer, A procedure for generating a color distribution figure that approximates the color distribution, which includes a point cloud obtained by plotting the coordinate values ​​of multiple pixels contained in an image on a chromaticity diagram, and A procedure for detecting the first vertex of the aforementioned color distribution figure, A procedure for determining a first target point for color correction of the image, corresponding to the first vertex, A procedure for performing color correction to bring the coordinate values ​​of the first vertex closer to the coordinate values ​​of the first target point, A program that causes the aforementioned computer to execute.

Images