Domain selection support method, domain selection support device, and program
The method of dividing stained images into regions and assigning display colors based on cell clustering addresses the challenge of selecting suitable analysis regions, improving visibility and efficiency in immunostaining analysis.
Patent Information
- Application Number
- JP2023045668
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- Filing Date
- 2023-03-22
- Publication Date
- 2026-11-19
- Estimated Expiration
- 2043-03-22
AI Technical Summary
Selecting a suitable region for analysis in stained images, particularly in immunostaining, is challenging due to the visibility issues of stained cells and noise in tissue samples, making it difficult for operators to visually identify regions suitable for analysis.
A method and device that divide stained images into regions, assign display colors based on cell clustering, and generate a selection support image to enhance visibility, allowing for easy region selection.
Facilitates the easy identification of regions suitable for analysis by enhancing the visibility of cell clusters, reducing the time and effort required for selecting analysis regions.
Smart Images

Figure 0007953857000001 
Figure 0007953857000002 
Figure 0007953857000003
Abstract
Description
Technical Field
[0001] The present invention relates to a technique for supporting selection of an analysis region in a stained image obtained by an immunostaining method.
Background Art
[0002] In recent years, in medical research and biological research, single-cell level analysis of biological specimens such as tissue specimens and cell specimens has been carried out for the purposes of elucidating disease mechanisms, elucidating biological mechanisms, and elucidating drug action mechanisms. For example, in immunostaining analysis, quantitative and qualitative analysis is performed by staining biological substances such as proteins and measuring the staining state.
[0003] Patent Document 1 proposes a technique of dividing a specimen image of one cell into a plurality of sections and scoring the malignancy of a disease such as cancer based on the expression level and distribution of biomarkers in each section. Further, Patent Document 2 proposes a technique of identifying pixels showing blood flow characteristics on a diagnostic image from the blood flow characteristics obtained by measuring temporal changes of a contrast agent inside a lesion in dynamic examinations performed by CT, MRI or the like.
Prior Art Literature
Patent Literature
[0004]
Patent Document 1
Patent Document 2
Summary of the Invention
Problems to be Solved by the Invention
[0005] Incidentally, in the analysis of tissue samples, attempting to analyze the entire sample on a single-cell basis would require a considerable amount of time. Therefore, it is conceivable to extract a partial region of the sample and perform analysis on that region. When extracting such a partial region, it is necessary to appropriately select a region that best represents the characteristics of the sample and is suitable for analysis (i.e., a region that appropriately contains the various cells to be analyzed).
[0006] However, even if an operator visually inspects an image of a tissue sample, if the number of stained cells (hereinafter also referred to as "stained cells") is small, there is a risk that the stained cells may not be visible at a display magnification that shows the entire tissue. In addition, noise other than stained cells (for example, dark black or white pixels caused by the structure of the tissue or cells) may reduce the visibility of stained cells. Therefore, it is not easy for an operator to select a region suitable for analysis from an image of a tissue sample by visually inspecting the image.
[0007] This invention has been made in view of the above problems, and aims to easily select regions in stained images that are suitable for analysis. [Means for solving the problem]
[0008] One aspect of the present invention is a region selection support method for assisting the selection of an analysis region in a stained image by immunohistochemistry, comprising: a) dividing a stained image of a specimen by immunohistochemistry into a plurality of divided regions; b) assigning a display color to each of the plurality of divided regions from a group of display colors including a plurality of colors with different attractive properties, such that the attractiveness increases as the clustering of stained cells in the divided region increases; c) generating a selection support image in which each of the plurality of divided regions is filled with the display color assigned in step b); and d) displaying the selection support image on a screen.
[0009] Aspect 2 of the present invention is a region selection support method according to aspect 1, wherein in step b), the clustering of stained cells in the divided region is determined based on the staining intensity in the divided region.
[0010] A third aspect of the present invention is a region selection support method according to the second aspect, wherein the calculation of the clustering of stained cells in the divided region comprises: e) obtaining the staining intensity of each of a plurality of sub-regions arranged in a matrix that constitute the divided region; and f) performing image reduction using the maximum value among the plurality of sub-regions as a representative value of the divided region.
[0011] Aspect 4 of the present invention is a region selection support method according to aspect 1, wherein in step b), the clustering of stained cells in the divided region is determined based on the number of positive cells in the divided region.
[0012] Aspect 5 of the present invention is a region selection support method according to any one of aspects 1 to 4, wherein in step d), other selection support images generated in the same manner as the selection support image from the stained image or other stained images of the specimen by immunohistochemistry are displayed side by side on the screen in alignment with the selection support image.
[0013] Aspect 6 of the present invention is a region selection support method according to aspect 5, wherein the display magnification of the selection support image and the display magnification of the stained image or the other selection support image can be changed in conjunction.
[0014] Aspect 7 of the present invention is a region selection support method according to aspect 5 (or aspect 5 or 6), wherein the display range of the selection support image on the specimen and the display range of the stained image or the other selection support image on the specimen can be changed in conjunction.
[0015] Aspect 8 of the present invention is a region selection support method according to aspect 5 (which may be any one of aspects 5 to 7), wherein a position indicator unit displayed on the selection support image to indicate a position on the image and a position indicator unit displayed on the stained image or the other selection support image to indicate a position on the image are movable in conjunction with each other.
[0016] Aspect 9 of the present invention is a region selection support method according to aspect 5 (which may be any one of aspects 5 to 8), wherein in step d), the scale of the selection support image relative to the specimen is the same as the scale of the stained image or the other selection support image relative to the specimen.
[0017] Aspect 10 of the present invention is a region selection support method according to any one of aspects 1 to 4 (or any one of aspects 1 to 9), further comprising the step of dividing another stained image of the specimen by immunohistochemistry into a plurality of divided regions in the same manner as the division method in step a), prior to step b), wherein the assignment of display colors in step b) is performed based on the clustering of stained cells in the stained image and the other stained image in the divided regions.
[0018] Aspect 11 of the present invention is a region selection support method according to any one of aspects 1 to 4 (or any one of aspects 1 to 10), wherein a binary image showing positive and negative regions is generated by thresholding other stained images of the specimen by immunohistochemistry, and in step d), the selection support image is masked by the positive region or the negative region.
[0019] Aspect 12 of the present invention is a region selection support method according to any one of aspects 1 to 4 (or any one of aspects 1 to 11), wherein a boundary line of a positive or negative region is obtained by thresholding another stained image of the specimen by immunohistochemistry, and in step d), the boundary line is superimposed on the selection support image.
[0020] Aspect 13 of the present invention is a region selection support device that supports selection of an analysis region in a stained image obtained by an immunostaining method, comprising: an image division unit that divides a stained image of a specimen obtained by the immunostaining method into a plurality of divided regions; a display color assignment unit that assigns, to each of the plurality of divided regions, one display color from a display color group including a plurality of colors with different attractiveness, such that the attractiveness increases as the aggregation of stained cells in the divided region increases; a selection support image generation unit that generates a selection support image in which each of the plurality of divided regions is filled with the display color assigned by the display color assignment unit; and a screen that displays the selection support image.
[0021] Aspect 14 of the present invention is a program that supports selection of an analysis region in a stained image obtained by an immunostaining method, wherein when the program is executed by a computer, the following steps are performed: a) a step of dividing the stained image of the specimen obtained by the immunostaining method into a plurality of divided regions; b) a step of assigning, to each of the plurality of divided regions, one display color from a display color group including a plurality of colors with different attractiveness, such that the attractiveness increases as the aggregation of stained cells in the divided region increases; c) a step of generating a selection support image in which each of the plurality of divided regions is filled with the display color assigned in step b); and d) a step of displaying the selection support image on a screen. [Advantageous Effects of Invention]
[0022] According to the present invention, a region suitable for analysis in a stained image can be easily selected. [Brief Description of Drawings]
[0023] [Figure 1] It is a diagram illustrating a configuration of an image display device according to one embodiment. [Figure 2] It is a block diagram illustrating a functional configuration of the image display device. [Figure 3] It is a diagram illustrating a stained image. [Figure 4] It is a diagram illustrating a selection support image. [Figure 5] It is a diagram illustrating a display flow of the selection support image. [Figure 6] This diagram shows a part of the display flow for selection support images. [Figure 7] This diagram shows a part of the display flow for selection support images. [Figure 8] This figure shows an example of a screen display. [Figure 9] This figure shows an example of a screen display. [Figure 10] This figure shows an example of a screen display. [Figure 11] This figure shows an example of a screen display. [Figure 12] This figure shows an example of a screen display. [Figure 13] This figure shows an example of a screen display. [Figure 14] This figure shows an example of a screen display. [Figure 15] This figure shows an example of a screen display. [Figure 16] This figure shows an example of a screen display. [Figure 17] This figure shows an example of a screen display. [Figure 18] This figure shows an example of a screen display. [Figure 19] This figure shows an example of a screen display. [Figure 20] This diagram shows a part of the display flow for selection support images. [Modes for carrying out the invention]
[0024] Figure 1 shows the configuration of a computer that functions as a region selection support device 1 according to one embodiment of the present invention. The region selection support device 1 is used to assist in selecting a region suitable for analysis in a stained image of a biological specimen obtained by immunohistochemistry.
[0025] Figure 1 shows the configuration of a computer that functions as a region selection support device 1. The computer has the configuration of a general computer system, including a CPU 51, ROM 52, RAM 53, fixed disk 54, display 55, input unit 56, reader 57, communication unit 58, GPU 59, and bus 50. The CPU 51 performs various arithmetic operations. The GPU 59 performs various arithmetic operations related to image processing. The ROM 52 stores the basic program. The RAM 53 stores various information. The fixed disk 54 stores information. The display 55 is a display unit that displays various information such as images.
[0026] The input unit 56 includes a keyboard 56a and a mouse 56b that accept input from the operator. The reader 57 reads information from a computer-readable recording medium 571 such as an optical disk, magnetic disk, magneto-optical disk, or memory card. The display 55, keyboard 56a, mouse 56b, and reader 57 are connected to the bus 50 via an interface I / F. The communication unit 58 sends and receives signals to and from external devices of the area selection support device 1. The bus 50 is a signal circuit that connects the CPU 51, GPU 59, ROM 52, RAM 53, fixed disk 54, display 55, input unit 56, reader 57, and communication unit 58.
[0027] In the area selection support device 1, program 572 is read in advance from the recording medium 571 via the reader 57 and stored in the fixed disk 54. Program 572 may also be stored in the fixed disk 54 via a network. The CPU 51 and GPU 59 perform arithmetic processing using RAM 53 and the fixed disk 54 according to program 572. The CPU 51 and GPU 59 function as the arithmetic unit in the area selection support device 1. Other configurations besides the CPU 51 and GPU 59 that function as the arithmetic unit may also be employed.
[0028] Figure 2 is a block diagram showing the functional configuration of the region selection support device 1, which is realized when the aforementioned computer performs calculations and other processing according to program 572. These functional configurations include a storage unit 501, an image division unit 502, a display color assignment unit 503, a selection support image generation unit 504, a display control unit 505, an analysis image acquisition unit 506, a mask generation unit 507, and a boundary line acquisition unit 508. All or part of these functions may be realized by dedicated electrical circuits. Alternatively, these functions may be realized by multiple computers.
[0029] Of the functional configurations shown in Figure 2, the storage unit 501 is mainly realized by RAM 53 and a fixed disk 54. The image splitting unit 502, display color assignment unit 503, selection support image generation unit 504, display control unit 505, analysis image acquisition unit 506, mask generation unit 507, and boundary acquisition unit 508 are realized by CPU 51, GPU 59, ROM 52, RAM 53, fixed disk 54, and their peripheral configurations.
[0030] The memory unit 501 pre-stores stained images of biological specimens (hereinafter also simply referred to as "specimens"). Figure 3 shows an example of a stained image 91 of a specimen 90 stored in the memory unit 501. The stained image 91 is an image obtained by imaging the specimen 90 after it has been stained by immunohistochemistry. The stained image 91 is, for example, a single-stain image obtained by staining the specimen 90 with a single staining method. This staining method is, for example, a staining method using a CD45 antibody that detects the CD45 protein, which is one of the CD classifications. In this case, in the stained image 91, among the many cells contained in the specimen 90, the cells 96 detected by the CD45 antibody are shown stained in a single color. The staining of the specimen 90 may be done by various staining methods using antibodies, or by various staining methods that do not use antibodies.
[0031] The memory unit 501 shown in Figure 2 also stores in advance a group of display colors to be used in the selection support image described later. This group of display colors includes multiple colors (i.e., display colors) with different levels of attention-grabbing properties. "Attractiveness" refers to the degree to which an image attracts the attention of an operator or other person viewing it. For example, warm colors are more attractive than cool colors, and the higher the brightness, the higher the attractiveness. Furthermore, the "multiple colors" mentioned above does not mean that the group consists only of the same color with different shades (i.e., a single color), but rather includes multiple types of colors with different hues. For example, the group of display colors includes "yellow," "green," "light blue," and "navy blue" in order of increasing attractiveness. In this embodiment, as described above, the stained image 91 is a monochromatic image (i.e., an image displayed in a single color), so the group of display colors including multiple colors has higher discriminability than the group of colors (i.e., a single color) that make up the stained image 91.
[0032] The image segmentation unit 502 divides the stained image 91 into a plurality of segmented regions 95. In the example shown in Figure 3, the stained image 91 is divided into a plurality of segmented regions 95 arranged in a matrix in the vertical and horizontal directions in the figure. The shape of each segmented region 95 is, for example, a square. Each segmented region 95 is, for example, larger than each pixel that makes up the stained image 91. In other words, each segmented region 95 contains multiple pixels. Also, each segmented region 95 is, for example, larger than each cell 96 included in the stained image 91. Note that each segmented region 95 may have various shapes other than a square (for example, a rectangle). Furthermore, in the stained image 91, segmented regions 95 do not need to be generated in areas other than the specimen 90. In the actual stained image 91, lines indicating the boundaries of the plurality of segmented regions 95 are not included, so when the stained image 91 is displayed on the display 55 (see Figure 1), these boundaries are not displayed.
[0033] The display color assignment unit 503 determines the clustering of stained cells 96 (hereinafter also referred to as "stained cells 96") in each of the multiple divided regions 95. Based on the clustering of stained cells 96 in each divided region 95, the display color assignment unit 503 assigns one display color from the above-mentioned group of display colors to each divided region 95. The attractiveness of the display color assigned to a divided region 95 increases as the clustering of stained cells 96 in the divided region 95 increases. In addition, the display color assignment unit 503 assigns the background color "black" to regions other than the specimen 90.
[0034] The selection support image generation unit 504 generates a selection support image corresponding to the stained image 91. Figure 4 shows an example of a selection support image 81. The selection support image 81 is an image in which each divided region 95 of the stained image 91 is filled with a display color assigned by the display color assignment unit 503.
[0035] For example, the segmented region 95 containing the area with the highest clustering of stained cells 96 (see Figure 3) is filled in "yellow" and constitutes the first region 811 in the selection support image 81. In Figure 4, for illustrative purposes, the two first regions 811 are shown as white areas without parallel diagonal lines. The segmented region 95 containing the area with the second highest clustering of stained cells 96 is filled in "green" and constitutes the second region 812 in the selection support image 81. In Figure 4, the second region 812 is marked with parallel diagonal lines that are relatively spaced far apart. The segmented region 95 containing the area with the third highest clustering of stained cells 96 is filled in "light blue" and constitutes the third region 813 in the selection support image 81. In Figure 4, the third region 813 is marked with parallel diagonal lines that are spaced far apart far apart than those in the second region 812. The segmented region 95, which contains the area with the lowest clustering of stained cells 96, is filled in "dark blue" and constitutes the fourth region 814 in the selection support image 81. In Figure 4, the fourth region 814 is marked with parallel diagonal lines with the smallest spacing. In the example shown in Figure 4, the spacing of the parallel diagonal lines decreases as the visibility of the displayed color decreases.
[0036] The display control unit 505 shown in Figure 2 displays the selection support image 81 on the screen of the display 55 (see Figure 1). The display control unit 505 can also change the display magnification of the selection support image 81 displayed on the screen. It can also change the display range of the selection support image 81 on the sample 90.
[0037] The analysis image acquisition unit 506 uses the selection support image 81 displayed on the display 55 to select a region on the stained image 91, extracts it from the stained image 91 as an image for detailed analysis, and stores it in the storage unit 501.
[0038] Next, the process of displaying the selection support image 81 by the region selection support device 1 will be explained with reference to Figures 5 to 7. First, the image division unit 502 (see Figure 2) divides the stained image 91 into a plurality of division regions 95 as shown in Figure 3 (step S11). Subsequently, the display color assignment unit 503 (see Figure 2) determines the clustering tendency of stained cells 96 for each of the plurality of division regions 95. Then, a single display color is assigned to each division region 95 such that the visibility increases as the clustering tendency of stained cells 96 increases (step S12).
[0039] In step S12, the clustering of stained cells 96 in the divided region 95 can be determined by various methods. For example, the clustering of stained cells 96 in the divided region 95 can be determined based on the number of positive cells in the divided region 95. Specifically, first, the location of the stained cells 96 in the stained image 91 is identified (Figure 6: step S21). Known techniques such as image segmentation using deep learning can be used to identify the location of the stained cells 96. Next, for each identified stained cell 96, feature quantities related to a predetermined analysis method (e.g., average brightness within the cell region) are calculated (step S22).
[0040] Next, it is determined whether each stained cell 96 is positive or not by comparing the above-mentioned feature quantity of each stained cell 96 with a predetermined threshold (step S23). Then, the number of stained cells 96 determined to be positive (i.e., positive cells) in each divided region 95 is determined (step S24). This allows us to obtain a number of positive cells that matches the above analysis method. Subsequently, for each divided region 95, one display color is assigned from the above-mentioned group of display colors so that the visibility increases as the number of positive cells increases (i.e., the clustering increases) (step S25).
[0041] Furthermore, the clustering of stained cells 96 in a divided region 95 may be determined based on the staining intensity in that divided region 95. The average staining intensity value of a divided region 95 depends on the density of stained cells 96 in the divided region 95, if the divided region 95 contains multiple stained cells 96. Therefore, when the staining intensity in the divided region 95 is high, the clustering of stained cells 96 is also high, and when the staining intensity in the divided region 95 is low, the clustering of stained cells 96 is also low.
[0042] Figure 7 shows an example of calculating the clustering of stained cells 96 based on staining intensity. In the example shown in Figure 7, first, noise reduction processing (e.g., blurring) is performed on the stained image 91, and then color separation processing is performed to extract one color corresponding to the staining of the stained cells 96. Next, the divided region 95 is divided into multiple sub-regions arranged in a matrix (e.g., four 2x2 sub-regions), and the staining intensity of each sub-region (so-called kernel) is determined for the extracted one color (step S31). Next, maximum value pooling is performed, and the maximum value (i.e., the largest staining intensity) among the staining intensities of the multiple sub-regions is used as the representative value of the staining intensity of the divided region 95, and the image is reduced (step S32). After that, for each divided region 95, one display color is assigned from the above-mentioned display color group so that the visibility increases as the staining intensity increases (i.e., the clustering increases) (step S33).
[0043] In the method shown in steps S31 to S33, since it is not necessary to identify individual stained cells 96 in the stained image 91, the amount of processing required for assigning display colors to each segmented region 95 can be reduced compared to the method shown in steps S21 to S25, and processing using stained images 91 acquired at low magnification can be performed. For example, according to the method shown in steps S31 to S33, even when using a stained image 91 with a resolution where the size of one pixel is about the same as the size of a cell (for example, a pixel pitch of 10 μm), the display colors can be suitably assigned to improve the visibility of regions with high clustering of stained cells (i.e., regions suitable for analysis). Furthermore, in the method shown in steps S31 to S33, the display colors can be suitably assigned even when the number of stained cells 96 contained in the stained image 91 is small.
[0044] When step S12 is completed, the selection support image generation unit 504 (see Figure 2) generates a selection support image 81 (see Figure 4), in which each of the multiple divided regions 95 is filled with the display color assigned in step S12 (step S13). Then, the display control unit 505 (see Figure 2) displays the selection support image 81 on the screen of the display 55 (step S14).
[0045] In the region selection support device 1, the display magnification of the selection support image 81 on the display 55 and the display range of the selection support image 81 on the sample 90 (i.e., the range of the sample 90 that is displayed on the screen as the selection support image 81) can be changed. The operator changes the display mode of the selection support image 81 in various ways as needed and specifies the region for analysis (hereinafter also called the "analysis region") on the selection support image 81 (step S15). The analysis image acquisition unit 506 extracts the region on the stained image 91 corresponding to the analysis region and sends it to the storage unit 501 as an image for detailed analysis (step S16).
[0046] The display control unit 505 can either display only the selection support image 81 on the screen of the display 55, or it can display various other images on the screen of the display 55 along with the selection support image 81. For example, in step S14, as shown in Figure 8, the selection support image 81 and the staining image 91 may be displayed side by side on the screen 551 of the display 55 so as not to overlap each other. In the example shown in Figure 8, the selection support image 81 and the staining image 91 are displayed at the same scale and in a relative position. That is, the selection support image 81 and the staining image 91 are corrected as needed so that the size and position of the sample 90 in the selection support image 81 and the size and position of the sample 90 in the staining image 91 are approximately the same. This allows the operator to easily compare the selection support image 81 with the staining image 91.
[0047] In the region selection support device 1, it is preferable that the display magnification of the selection support image 81 on the screen 551 and the display magnification of the stained image 91 can be changed in conjunction. Furthermore, it is preferable that the display range of the selection support image 81 on the specimen 90 and the display range of the stained image 91 on the specimen 90 (i.e., the range of the specimen 90 that is displayed on the screen 551 as the stained image 91) can be changed in conjunction.
[0048] For example, if an operator positions a mouse pointer (not shown) over the selection support image 81 and changes the display magnification of the selection support image 81 by operating the mouse wheel or the like, the display magnification of the stained image 91 on the screen 551 is also changed in conjunction to match the display magnification of the selection support image 81, as shown in Figure 9. If the display magnification of the selection support image 81 is increased from its initial state, only a portion of the selection support image 81 and a portion of the stained image 91 will be displayed on the screen 551. Similarly, if the operator changes the display magnification of the stained image 91, the display magnification of the selection support image 81 is also changed in conjunction.
[0049] This allows the operator to compare the selection support image 81 and the stained image 91 at the same display magnification without having to adjust the display magnification for each of the selection support image 81 and the stained image 91.
[0050] Furthermore, as shown in Figure 9, the operator positions the mouse pointer over the selection support image 81 while the selection support image 81 and the stained image 91 are enlarged compared to their initial state (see Figure 8), and changes the display range of the selection support image 81 on the sample 90 (see Figure 8) by operating the mouse buttons, etc. (i.e., changes the position of the display range of the selection support image 81 on the sample 90). Accordingly, as shown in Figure 10, the display range of the stained image 91 on the sample 90 on the screen 551 is also changed in conjunction to match the display range of the selection support image 81. Similarly, if the operator changes the display range of the stained image 91 on the sample 90, the display range of the selection support image 81 on the sample 90 is also changed in conjunction.
[0051] This allows the operator to immediately observe a region of interest in one of the selection support images 81 and the staining image 91 without having to search for that region again in the other image.
[0052] In the region selection support device 1, as shown in Figures 8 to 10, position indicators 553 indicating the position on each image may be displayed on the selection support image 81 and the staining image 91, respectively. The position indicators 553 are, for example, cross marks or rectangular frames that can be moved by a mouse 56b or the like. Preferably, the two position indicators 553 displayed on the selection support image 81 and the staining image 91 can move in conjunction with the movement of the mouse 56b or the like. This allows the operator to easily focus on approximately the same position indicated by the position indicators 553 on the selection support image 81 and the staining image 91.
[0053] The operator specifies the analysis region on the selection support image 81 while changing the display modes of the selection support image 81 and the stained image 91 as needed. Specifically, the operator sets the analysis window 87, as illustrated in Figure 11, to a desired position on the selection support image 81 using the mouse 56b or the like. In the region selection support device 1, the analysis window 97 is also displayed on the stained image 91 in conjunction with the setting of the analysis window 87 on the selection support image 81. The size of the analysis window 97 and its relative position to the sample 90 on the stained image 91 are the same as the size of the analysis window 87 and its relative position to the sample 90 on the selection support image 81. In the example shown in Figure 11, the shape of the analysis window 87 is rectangular, and its size can be changed. When the size or position of the analysis window 87 is changed, the size and position of the analysis window 97 are also changed in conjunction.
[0054] Once the settings for analysis windows 87 and 97 (i.e., the specification of the analysis region) are complete, the analysis image acquisition unit 506 (see Figure 2) extracts the region included in the analysis window 97 of the stained image 91 and sends it to the storage unit 501 as an image for detailed analysis. This allows the operator to easily confirm the distribution of clustering of stained cells 96 in the selection support image 81 and select a region in the stained image 91 suitable for analysis based on that distribution. For example, the operator can confirm a region with high clustering of stained cells 96 in the selection support image 81 and select a region in the stained image 91 that corresponds to that region and is suitable for analysis.
[0055] Alternatively, the operator may set the analysis window 97 directly on the stained image 91 instead of performing operations on the selection support image 81 to set the analysis window 87. In this case, the analysis window 87 linked to the analysis window 97 may be displayed on the selection support image 81. Furthermore, the settings of the analysis windows 87 and 97 may be performed on screen 551, where the display magnification and display range of the selection support image 81 and the stained image 91 have been changed from their initial state, as shown in Figures 9 and 10.
[0056] In step S14, as shown in Figure 12, other staining images 92 different from the staining image 91, and other selection support images 82 generated from the other staining images 92 in the same manner as the selection support image 81, may be displayed on the screen 551 side by side so as not to overlap with the staining image 91 and the selection support image 81.
[0057] Stained image 92 is a single-stain image obtained by staining the same specimen 90 as stained image 91 using a different staining method (for example, immunohistochemistry using CD20 antibody). Stained images 91 and 92 are obtained, for example, by sequentially performing staining and imaging with a first antibody, destaining the specimen 90, and then staining and imaging with a second antibody on a single specimen 90. Alternatively, multiple types of single-stain images may be obtained by staining a single specimen 90 using multiple types of antibodies together, and then color-separating the multicolor image obtained by imaging according to the color corresponding to each antibody.
[0058] In selection support image 82, similar to selection support image 81, the segmented region containing the area with the highest clustering of stained cells 96 is filled in "yellow," constituting the first region 821 in selection support image 82. In Figure 12, for illustrative purposes, the first region 821 is shown as a white area without parallel diagonal lines. The segmented region containing the area with the second highest clustering of stained cells 96 is filled in "green," constituting the second region 822 in selection support image 82. In Figure 12, the second region 822 is marked with parallel diagonal lines that are relatively spaced far apart. The segmented region containing the area with the third highest clustering of stained cells 96 is filled in "light blue," constituting the third region 823 in selection support image 82. In Figure 12, the third region 823 is marked with parallel diagonal lines that are spaced far apart far apart than those in the second region 822. The segmented region containing the least clustering of stained cells 96 is filled in "dark blue" and constitutes the fourth region 824 in the selection support image 82. In Figure 12, the fourth region 824 is marked with the smallest spacing of parallel diagonal lines. In the example shown in Figure 12, the spacing of the parallel diagonal lines decreases as the visibility of the displayed color decreases.
[0059] In the example shown in Figure 12, the selection support image 81, staining image 91, selection support image 82, and staining image 92 are displayed aligned to each other and at the same scale relative to the specimen 90. That is, the selection support images 81, 82 and the staining images 91, 92 are displayed with necessary corrections so that the size and position of the specimen 90 in the selection support image 81, the staining image 91, the selection support image 82, and the staining image 92 are approximately the same. This allows the operator to easily compare the selection support images 81, 82 and the staining images 91, 92.
[0060] The alignment of the selection support images 81, 82 and the staining images 91, 92 may be performed automatically, for example, by known image registration techniques, or manually by an operator. When manual alignment is performed, the operator first sets reference points for each of the selection support images 81, 82 and the staining images 91, 92 by operating a mouse 56b (see Figure 1), etc. These reference points are set to what are thought to be the same position on the specimen 90 in each of the selection support images 81, 82 and the staining images 91, 92. Then, the display range of each image is adjusted so that the reference points are located at a predetermined position (for example, the center of each image).
[0061] In the region selection support device 1, it is preferable that the display magnification of the selection support image 81 and staining image 91 on the screen 551 and the display magnification of the selection support image 82 and staining image 92 can be changed in conjunction. Furthermore, it is preferable that the display range of the selection support image 81 and staining image 91 on the specimen 90 and the display range of the selection support image 82 and staining image 92 on the specimen 90 (i.e., the range of the specimen 90 displayed on the screen 551 as the selection support image 82 and staining image 92) can be changed in conjunction.
[0062] For example, if an operator positions a mouse pointer (not shown) over the selection support image 81 and changes the display magnification of the selection support image 81 by operating the mouse wheel or the like, the display magnifications of the selection support image 82 and the staining images 91 and 92 on the screen 551 are also changed in conjunction to match the display magnification of the selection support image 81, as shown in Figure 13. If the display magnification of the selection support image 81 is increased from its initial state, only a portion of the selection support images 81 and 82 and a portion of the staining images 91 and 92 will be displayed on the screen 551. Similarly, if the operator changes the display magnification of the selection support image 82, the staining image 91, or the staining image 92, the display magnifications of the other three images are changed in conjunction.
[0063] This allows the operator to compare the selection support images 81 and 82 and the staining images 91 and 92 at the same display magnification without having to adjust the display magnification for each of them.
[0064] Furthermore, as shown in Figure 13, with the selection support images 81 and 82 and staining images 91 and 92 enlarged from their initial state (see Figure 12), the operator positions the mouse pointer over the selection support image 81 and changes the display range of the selection support image 81 on the sample 90 (see Figure 12) by operating the mouse buttons, etc. Accordingly, as shown in Figure 14, the display ranges of the selection support image 82 and staining images 91 and 92 on the sample 90 on the screen 551 are also changed in conjunction to match the display range of the selection support image 81. Similarly, if the operator changes the display range of the selection support image 82, staining image 91, or staining image 92 on the sample 90, the display ranges of the other three images on the sample 90 are also changed in conjunction.
[0065] This allows the operator to immediately observe a region of interest in one of the selection support images 81, 82 and staining images 91, 92, without having to search for that region again in the other three images.
[0066] In the region selection support device 1, as shown in Figures 12 to 14, position indicators 553 indicating the position on each image may be displayed on the selection support images 81, 82 and the staining images 91, 92. The position indicators 553 are, for example, cross marks or rectangular frames that can be moved by a mouse 56b or the like. Preferably, the four position indicators 553 displayed on the selection support images 81, 82 and the staining images 91, 92 are able to move in conjunction with the movement of the mouse 56b or the like. This allows the operator to easily focus on approximately the same position indicated by the position indicators 553 in the selection support images 81, 82 and the staining images 91, 92.
[0067] The operator specifies the analysis region on the selection support image 81 or selection support image 82 while changing the display modes of the selection support images 81, 82 and the staining images 91, 92 as needed. For example, the operator specifies the analysis region by setting the analysis window 87, as illustrated in Figure 15, at a desired position on the selection support image 81 using the mouse 56b or the like (step S15). In the region selection support device 1, in conjunction with the setting of the analysis window 87 on the selection support image 81, the analysis window 88 is displayed on the selection support image 82, and the analysis windows 97 and 98 are displayed on the staining images 91 and 92.
[0068] The size of the analysis window 88 and its relative position to the sample 90 on the selection support image 82, the size of the analysis window 97 and its relative position to the sample 90 on the stained image 91, and the size of the analysis window 98 and its relative position to the sample 90 on the stained image 92 are the same as the size of the analysis window 87 and its relative position to the sample 90 on the selection support image 81. In the example shown in Figure 15, the shape of the analysis window 87 is rectangular, and its size can be changed. When the size or position of the analysis window 87 is changed, the size and position of the analysis windows 88, 97, and 98 are also changed in conjunction.
[0069] Once the settings for analysis windows 87, 88, 97, and 98 (i.e., the specification of the analysis regions) are complete, the analysis image acquisition unit 506 (see Figure 2) extracts the region included in analysis window 97 of the stained image 91 and / or the region included in analysis window 98 of the stained image 92, and sends it to the storage unit 501 as an image for detailed analysis (step S16). This allows the operator to easily confirm the distribution of clustering of stained cells 96 in the selection support images 81 and 82, and select regions in the stained images 91 and 92 that are suitable for analysis based on that distribution. For example, the operator can confirm the region with the highest clustering of stained cells 96 (the region shown in "yellow" in the above example) on screen 551 in both selection support image 81 and selection support image 82, and select a region in the stained images 91 and 92 that is suitable for analysis from the region corresponding to that region.
[0070] Alternatively, instead of setting the analysis window 87 by performing operations on the selection support image 81, the operator may set the analysis window 88 on the selection support image 82. In this case as well, analysis windows 87, 97, and 98, which are linked to the analysis window 88, will be displayed on the selection support image 81, the stained image 91, and the stained image 92, respectively.
[0071] Furthermore, instead of setting the analysis window 87 by performing operations on the selection support image 81, the operator may directly set the analysis window 97 on the stained image 91, or directly set the analysis window 98 on the stained image 92. In these cases, analysis windows 87 and 88 linked to the analysis window 97 or analysis window 98 may be displayed on the selection support images 81 and 82. Also, the settings of the analysis windows 87, 88, 97, and 98 may be performed on screen 551 where the display magnification and display range of the selection support images 81 and 82 and the stained images 91 and 92 have been changed from their initial state, as shown in Figures 13 and 14.
[0072] As described above, when specifying the analysis region based on both stained images 91 and 92, in step S14, it is not necessarily required that the selection support image 81 and the selection support image 82 be displayed side by side on the screen 551. For example, as shown in Figures 16 and 17, the masks 85a and 85b generated by the mask generation unit 507 (see Figure 2) based on the stained image 92 (see Figure 12) may be superimposed on the selection support image 81, so that only a part of the selection support image 81 is visible. The mask generation unit 507 generates a binary image showing positive and negative regions by, for example, thresholding the stained image 92 aligned with the stained image 91. Then, one of the positive and negative regions is designated as the transparent region and the other as the mask region (i.e., the opaque region), thereby generating the masks 85a and 85b.
[0073] In the mask 85a illustrated in Figure 16, only the region containing the highest clustering of stained cells 96 in the stained image 92 (i.e., the region corresponding to the first region 821 shown in Figure 12) is considered a positive region, and this positive region is designated as the permeable region, which is the opening 851a. In addition, the region other than the opening 851a (i.e., the negative region) is filled with the background color "black" and designated as the mask region.
[0074] In the example shown in Figure 16, a mask 85a generated based on the stained image 92 is displayed superimposed on a selection support image 81 generated based on the stained image 91. In the selection support image 81, only the area overlapping with the positive area of the stained image 92 is visible. Therefore, the operator can select an analysis area by specifying the analysis area within the visible area, thereby limiting the analysis to the positive area in the stained image 92 and considering the clustering of stained cells 96 in the stained image 91. For example, by specifying the analysis area on the first area 811 of the selection support image 81 within the opening 851a, the analysis area can be limited to the positive area in the stained image 92 and selected as an analysis area where the clustering of stained cells 96 is high in the stained image 91.
[0075] In the mask 85b illustrated in Figure 17, the region containing the area with the highest clustering of stained cells 96 in the stained image 92, and the region containing the area with the second highest clustering (i.e., the regions corresponding to the first region 821 and the second region 822 shown in Figure 12) are defined as positive regions, and these positive regions are designated as the permeable region, which is the opening 851b. In addition, regions other than the opening 851b (i.e., negative regions) are filled with the background color "black" and designated as mask regions.
[0076] In the example shown in Figure 17, the mask 85b generated based on the stained image 92 is superimposed on the selection support image 81 generated based on the stained image 91. In this case as well, the operator can select the analysis region by specifying the analysis region in a visible area on the selection support image 81 (i.e., the area overlapping with the opening 851b of the mask 85b), thereby limiting the analysis to the positive area in the stained image 92 and considering the clustering of stained cells 96 in the stained image 91.
[0077] In the examples shown in Figures 16 and 17, the selection support image 81 is masked by the region identified as negative in the stained image 92. However, as described above, it may also be masked by the region identified as positive in the stained image 92. In this case, the operator can select the analysis region by limiting it to the negative region in the stained image 92 and considering the clustering of stained cells 96 in the stained image 91.
[0078] In step S14, instead of the masks 85a and 85b described above, a boundary line 86a acquired by the boundary line acquisition unit 508 (see Figure 2) based on the stained image 92 (see Figure 12) may be superimposed on the selection support image 81, as shown in Figure 18. The boundary line acquisition unit 508 generates a binary image showing positive and negative regions by, for example, thresholding the stained image 92, and acquires the boundary line 86a between the positive and negative regions. In the example shown in Figure 18, only the region containing the highest clustering of stained cells 96 in the stained image 92 (i.e., the region corresponding to the first region 821 shown in Figure 12) is considered a positive region, and the other regions are considered negative regions. The outer edge of the positive region is then defined as the boundary line 86a.
[0079] The operator can specify the analysis area inside the boundary line 86a, thereby limiting it to the positive region in the stained image 92 and selecting the analysis area considering the clustering of stained cells 96 in the stained image 91. Alternatively, the operator can specify the analysis area outside the boundary line 86a, thereby limiting it to the negative region in the stained image 92 and selecting the analysis area considering the clustering of stained cells 96 in the stained image 91. Furthermore, the operator can specify the analysis area at a position overlapping with the boundary line 86a, thereby limiting it to the boundary between the positive and negative regions in the stained image 92 and selecting the analysis area considering the clustering of stained cells 96 in the stained image 91.
[0080] Furthermore, in step S14, as shown in Figure 19, the boundary lines 86b and 86c acquired by the boundary line acquisition unit 508 (see Figure 2) based on the stained image 92 may be superimposed on the selection support image 81. The boundary line acquisition unit 508 generates an image showing the positive region, the negative region, and the boundary region between the positive region and the negative region (hereinafter also referred to as the "boundary region") by, for example, thresholding the stained image 92, and acquires the boundary line 86b between the positive region and the boundary region, and the boundary line 86c between the boundary region and the negative region.
[0081] In the example shown in Figure 19, the region containing the highest clustering of stained cells 96 in the stained image 92 (i.e., the region corresponding to the first region 821 shown in Figure 12) is defined as the positive region, the region containing the second highest clustering of stained cells 96 (i.e., the region corresponding to the second region 822 shown in Figure 12) is defined as the boundary region, and the other regions are defined as the negative regions. The outer edge of the positive region is defined as boundary line 86b, and the inner edge of the negative region is defined as boundary line 86c.
[0082] The operator can specify the analysis area inside boundary line 86b, thereby limiting the analysis to the positive region in stained image 92 and selecting the analysis area considering the clustering of stained cells 96 in stained image 91. Alternatively, the operator can specify the analysis area between boundary line 86b and boundary line 86c, thereby limiting the analysis to the boundary region in stained image 92 and selecting the analysis area considering the clustering of stained cells 96 in stained image 91. Furthermore, the operator can specify the analysis area outside boundary line 86b, thereby limiting the analysis to the negative region in stained image 92 and selecting the analysis area considering the clustering of stained cells 96 in stained image 91.
[0083] The assignment of display colors to each segmented region 95 in step S12 does not necessarily have to be based solely on the clustering of stained cells 96 in a single stained image 91, but may also be based on the clustering of stained cells 96 in each of multiple stained images obtained by staining the same specimen 90 using multiple different staining methods.
[0084] For example, if the stained images 91 and 92 described above are used for assigning display colors, first, before step S11, the stained images 91 and 92 are aligned. This alignment may be performed automatically by known image registration techniques, as described above, or it may be performed manually by an operator. Also, if the scales of the stained images 91 and 92 relative to the specimen 90 are different, the images are processed so that the scales are the same.
[0085] Next, as described above, the stained image 91 is divided into multiple divided regions 95 as shown in Figure 3 (Figure 5: Step S11). The stained image 92 is then divided into multiple divided regions 95 in a manner similar to that of the stained image 91 (Figure 20: Step S41). That is, the stained image 92 has multiple divided regions 95 of the same size and arrangement as the multiple divided regions 95 set in the stained image 91. Note that Step S41 may be performed in parallel with Step S11, or before or after Step S11.
[0086] Once steps S11 and S41 are completed, the display color assignment unit 503 (see Figure 2) determines the clustering characteristics of stained cells 96 for each segmented region 95 of the stained image 91 and each segmented region 95 of the stained image 92. Then, a display color is assigned to each segmented region 95 such that the visibility increases as the clustering characteristics of stained cells 96 increase (step S12).
[0087] The assignment of display colors in step S12 is based on the clustering of stained cells 96 in stained image 91 and stained image 92 within the divided region 95. For example, in each divided region 95, the display color may be assigned based on the lower clustering of stained cells 96 in stained image 91 and the lower clustering of stained cells 96 in stained image 92. Alternatively, in each divided region 95, the display color may be assigned based on the higher clustering of stained cells 96 in stained image 91 and the higher clustering of stained cells 96 in stained image 92. In either case, in step S14, a selection support image that takes into account the clustering of stained cells 96 in multiple types of stained images can be displayed.
[0088] Alternatively, in step S12, a composite stained image may be prepared by Max-combining multiple types of staining images pixel by pixel (i.e., an image in which the pixel value of each pixel is set to the largest pixel value among the multiple types of staining images), and this composite stained image may be divided into multiple divided regions 95. Then, display colors may be assigned based on the clustering of stained cells 96 in the composite stained image in each divided region 95. In this case, by dividing the composite stained image into multiple divided regions 95, for example, steps S11 and S41 described above can be performed simultaneously. According to this method, in the selection support image displayed in step S14, regions in which the clustering of stained cells 96 is high in any of the multiple types of staining images can be displayed with high visibility.
[0089] As described above, the region selection support method for assisting the selection of analysis regions in a stained image by immunohistochemistry comprises the steps of: dividing the stained image 91 of the specimen 90 by immunohistochemistry into a plurality of divided regions 95 (step S11); assigning a display color to each of the plurality of divided regions 95 from a group of display colors including multiple colors with different attractiveness, such that the attractiveness increases as the clustering of stained cells 96 in the divided region 95 increases (step S12); generating a selection support image 81 by filling each of the plurality of divided regions 95 with the display color assigned in step S12 (step S13); and displaying the selection support image 81 on a screen 551 (step S14). This makes it easy to visually identify the degree of clustering of stained cells 96 at each position in the stained image 91. As a result, the operator can easily select a region in the stained image 91 that is suitable for analysis based on the clustering of stained cells 96 as the analysis region.
[0090] As described above, in step S12, it is preferable that the clustering of stained cells 96 in the divided region 95 is determined based on the staining intensity in the divided region 95. In this case, since it is not necessary to identify individual cells in the stained image 91, the amount of processing required for assigning display colors in step S12 is reduced, and processing using the stained image 91 acquired at low magnification becomes possible. As a result, the time required for step S12 can be shortened.
[0091] As described above, the calculation of the clustering of stained cells 96 in the divided region 95 preferably comprises the steps of obtaining the staining intensity of each of the multiple sub-regions constituting the divided region 95 (step S31), and performing image reduction using the maximum value among the multiple sub-regions as a representative value for the divided region 95 (step S32). This makes it possible to improve the visibility of the divided region 95 in the selection support image 81, even when the number of stained cells 96 included in the divided region 95 is small.
[0092] As described above, in step S12, it is also preferable that the clustering of stained cells 96 in the divided region 95 be determined based on the number of positive cells in the divided region 95. This allows for accurate improvement of the attractiveness of regions with a high concentration of positive cells.
[0093] As described above, in step S14, it is preferable that other selection support images 82 generated in the same manner as selection support image 81 from the stained image 91 or other stained images 92 of the specimen 90 obtained by immunohistochemistry are displayed side by side on the screen 551, aligned with selection support image 81. When selection support image 81 and stained image 91 are displayed side by side, the structure of the tissue in the specimen 90 is generally easier to understand in the stained image 91, so the operator can select the analysis area while looking at the actual dispersion pattern of stained cells 96 in the stained image 91. In addition, regarding artifacts that may be difficult to judge from selection support image 81 alone (for example, those where staining is nonspecifically enhanced due to tissue distortion, etc.), the operator can make an accurate judgment by looking at selection support image 81 and stained image 91. When selection support image 81 and other selection support images 82 are displayed side by side, the operator can easily select an analysis region that takes into account the clustering of stained cells 96 in both stained image 91 and other stained image 92 by comparing selection support image 81 and other selection support images 82.
[0094] As described above, preferably, the display magnification of the selection support image 81 and the display magnification of the stained image 91 or other selection support image 82 can be changed in conjunction. This allows the operator to easily compare multiple images at the same display magnification without having to adjust the display magnification of the selection support image 81 and the stained image 91 or other selection support image 82 separately.
[0095] As described above, preferably, the display range of the selection support image 81 on the specimen 90 and the display range of the stained image 91 or other selection support image 82 on the specimen 90 can be changed in conjunction. This allows the operator to immediately observe a region of interest in one of the selection support image 81 and stained image 91 or other selection support image 82 without having to search for that region again in the other image.
[0096] As described above, preferably, the position indicator 553 displayed on the selection support image 81 to indicate a position on the image and the position indicator 553 displayed on the stained image 91 or other selection support image 82 to indicate a position on the image are movable in conjunction with each other. This allows the operator to easily focus on approximately the same position indicated by the position indicator 553 in the selection support image 81 and the stained image 91 or other selection support image 82.
[0097] As described above, in step S14, it is preferable that the scale of the selection support image 81 relative to the sample 90 is the same as the scale of the stained image 91 or other selection support image 82 relative to the sample 90. This allows the operator to easily compare the selection support image 81 with the stained image 91 or other selection support image 82.
[0098] The above-described region selection support method preferably further includes a step (step S41) prior to step S12, in which other stained images 92 of the specimen 90 obtained by immunohistochemistry are divided into a plurality of divided regions 95 in the same manner as the division method in step S11. Furthermore, the assignment of display colors in step S12 is preferably performed based on the clustering characteristics of the stained cells 96 in the stained image 91 and the other stained images 92 in the divided regions 95. This allows the operator to easily select an analysis region that takes into account the clustering characteristics of the stained cells 96 in both the stained image 91 and the stained image 92.
[0099] As described above, preferably, a binary image showing positive and negative regions is generated by thresholding other stained images 92 of the specimen 90 by immunohistochemistry, and in step S14, the selection support image 81 is masked by the positive or negative region. This allows the operator to select an analysis region that is limited to the positive or negative region in the stained image 92, while also considering the clustering of stained cells 96 in the stained image 91.
[0100] As described above, preferably, the boundaries 86a, 86b, and 86c of the positive or negative regions are obtained by thresholding other stained images 92 of the specimen 90 by immunohistochemistry, and in step S14, the boundaries 86a, 86b, and 86c are superimposed on the selection support image 81. This allows the operator to select the analysis region considering the clustering of stained cells 96 in the stained image 91, while also considering the positive and negative regions in the stained image 92.
[0101] The above-described region selection support device 1 comprises an image division unit 502, a display color assignment unit 503, a selection support image generation unit 504, and a screen 551. The image division unit 502 divides the stained image 91 of the specimen 90 obtained by immunohistochemistry into a plurality of divided regions 95. The display color assignment unit 503 assigns a display color to each of the plurality of divided regions 95 from a group of display colors containing multiple colors with different attractive properties, such that the attractiveness increases as the clustering of stained cells 96 in the divided region 95 increases. The selection support image generation unit 504 generates a selection support image 81 by filling each of the plurality of divided regions 95 with the display color assigned by the display color assignment unit 503. The screen 551 displays the selection support image 81. As a result, similar to the above, the operator can easily select a region in the stained image 91 that is suitable for analysis based on the clustering of stained cells 96 as the analysis region.
[0102] Furthermore, when the above-described program 572 is executed by the computer, the following steps are performed: (step S11) dividing the stained image 91 of the specimen 90 by immunohistochemistry into a plurality of divided regions 95; (step S12) assigning a display color to each of the plurality of divided regions 95 from a group of display colors including multiple colors with different attractiveness, such that the attractiveness increases as the clustering of stained cells 96 in the divided region 95 increases; (step S13) generating a selection support image 81 by filling each of the plurality of divided regions 95 with the display color assigned in step S12; and (step S14) displaying the selection support image 81 on the screen 551. As a result, similar to the above, the operator can easily select a region in the stained image 91 that is suitable for analysis based on the clustering of stained cells 96 as the analysis region.
[0103] Various modifications are possible to the above-described region selection support method, region selection support device 1, and program 572.
[0104] For example, the number of images displayed on screen 551 of display 55 is not limited to the example above and can be changed in various ways. Also, the arrangement of images on screen 551 can be changed in various ways.
[0105] In the area selection support device 1, when multiple displays 55 are arranged vertically and / or horizontally and used as a single display unit, the screen 551 described above is a compilation of the screens of the multiple displays 55.
[0106] The aforementioned boundary lines 86a, 86b, and 86c may be displayed superimposed on the selection support image 81 together with the mask 85a or mask 85b.
[0107] In the selection support images 81 and 82, contour lines corresponding to the degree of clustering of stained cells 96 in each segmented region 95 may be displayed using the aforementioned display colors, which are assigned such that their attractiveness increases as the clustering of stained cells 96 in each segmented region 95 increases. In this case, the outer edges of a group of adjacent segmented regions 95 that are assigned the same display color will be displayed in that color. Furthermore, the outer edge of a segmented region 95 that is assigned a different display color from all adjacent segmented regions 95 will be displayed in the display color assigned to that segmented region 95. Note that when the above contour lines are displayed in the selection support images 81 and 82, the filling of each segmented region 95 may be omitted. Even in this case, the degree of clustering of stained cells 96 at each position in the stained images 91 and 92 can be easily visualized, much as described above.
[0108] As described above, when multiple images (for example, selection support images 81, 82 and staining images 91, 92) are displayed on screen 551, the display magnification of these multiple images may be changed individually without being linked. Furthermore, the display range of these multiple images on the specimen 90 may be changed individually without being linked. In addition, the position indicators 553 displayed on each of these multiple images may be moved individually without being linked. Note that the display of the position indicators 553 may be omitted.
[0109] The stained image stored in the memory unit 501 of the region selection support device 1 may be, for example, a multicolor image in which multiple types of antibodies are used together on a single sample 90. In this case, for example, by color-separating the multicolor image, a single-stain image corresponding to a specific antibody is obtained, and this single-stain image is designated as the stained image 91. Alternatively, the multicolor image may be designated as the stained image 91, stained cells 96 corresponding to a specific antibody may be detected from the multicolor image to determine their clustering, and the selection support image 81 may be generated based on this clustering.
[0110] The configurations in the above embodiments and each modified example may be combined as appropriate, as long as they do not contradict each other. [Explanation of symbols]
[0111] 1 Area selection support device 81,82 Selection support images 86a,86b,86c border 90 specimens 91,92 Stained images 95 Split area 96 Stained cells 502 Image division section 503 Display Color Assignment Section 504 Selection Support Image Generation Unit 551 screens 553 Position indicator 572 Programs S11-S16, S21-S25, S31-S33, S41 Step
Claims
1. A region selection support method that assists in selecting analysis regions in stained images obtained by immunohistochemistry, a) A step of dividing the stained image of the specimen by immunohistochemistry into multiple divided regions, b) A step of assigning one display color to each of the plurality of divided regions from a group of display colors including multiple colors with different attractive properties, such that the attractiveness increases as the clustering of stained cells in the divided region increases, c) A step of generating a selection support image in which each of the plurality of divided regions is filled with the display color assigned in step b), d) A step of displaying the selection support image on the screen, A method for supporting domain selection, comprising the following features.
2. A method for supporting area selection according to claim 1, In step b) above, the aggregation of stained cells in the divided region is determined based on the staining intensity in the divided region, and is a region selection support method.
3. A method for supporting area selection according to claim 2, The calculation of the clustering of stained cells in the aforementioned divided region is as follows: e) A step of obtaining the staining intensity of each of the multiple sub-regions arranged in a matrix that constitute the divided region, f) A step of reducing the image using the maximum value among the multiple sub-regions' staining intensities as the representative value of the divided region, A method for supporting domain selection, comprising the following features.
4. A method for supporting area selection according to claim 1, In step b) above, the clustering of stained cells in the divided region is determined based on the number of positive cells in the divided region, and this is a region selection support method.
5. A method for supporting area selection according to any one of claims 1 to 4, A region selection support method in which, in step d) above, other selection support images generated in the same manner as the selection support image from the stained image or other stained images of the specimen by immunohistochemistry are displayed side by side on the screen in alignment with the selection support image.
6. A method for supporting area selection according to claim 5, A region selection support method in which the display magnification of the selection support image and the display magnification of the stained image or the other selection support image can be changed in conjunction.
7. A method for supporting area selection according to claim 5, A region selection support method in which the display range of the selection support image on the specimen and the display range of the stained image or the other selection support image on the specimen can be changed in conjunction.
8. A method for supporting area selection according to claim 5, A region selection support method in which a position indicator unit displayed on the selection support image to indicate a position on the image and a position indicator unit displayed on the stained image or the other selection support image to indicate a position on the image are able to move in conjunction with each other.
9. A method for supporting area selection according to claim 5, A region selection support method in which, in step d) above, the scale of the selection support image relative to the specimen and the scale of the stained image or the other selection support image relative to the specimen are the same.
10. A method for supporting area selection according to any one of claims 1 to 4, g) The step further comprises, prior to step b), dividing other stained images of the specimen by immunohistochemistry into a plurality of divided regions in a manner similar to the division method in step a), The assignment of display colors in step b) is a region selection support method performed based on the clustering characteristics of stained cells in the respective stained images and other stained images in the divided region.
11. A method for supporting area selection according to any one of claims 1 to 4, A region selection support method wherein a binary image showing positive and negative regions is generated by thresholding other stained images of the specimen by immunohistochemistry, and in step d), the selection support image is masked by the positive region or the negative region.
12. A method for supporting area selection according to any one of claims 1 to 4, A region selection support method in which a boundary line of a positive or negative region is obtained by thresholding other stained images of the specimen by immunohistochemistry, and in step d), the boundary line is superimposed on the selection support image.
13. A region selection support device that assists in selecting analysis regions in stained images obtained by immunohistochemistry, An image division unit that divides the stained image of a specimen obtained by immunohistochemistry into multiple divided regions, A display color assignment unit assigns one display color to each of the aforementioned plurality of divided regions from a group of display colors including multiple colors with different attractive properties, such that the attractiveness increases as the clustering of stained cells in the divided region increases. A selection support image generation unit generates a selection support image in which each of the plurality of divided regions is filled with a display color assigned by the display color assignment unit, A screen that displays the aforementioned selection support image, A region selection support device equipped with the following features.
14. A program that assists in selecting analysis regions in stained images obtained by immunohistochemistry, When the program is executed by the computer, a) A step of dividing the stained image of the specimen by immunohistochemistry into multiple divided regions, b) A step of assigning one display color to each of the plurality of divided regions from a group of display colors including multiple colors with different attractive properties, such that the attractiveness increases as the clustering of stained cells in the divided region increases, c) A step of generating a selection support image in which each of the plurality of divided regions is filled with the display color assigned in step b), d) A step of displaying the selection support image on the screen, A program in which this takes place.
Citation Information
Patent Citations
Image diagnosis support device, image diagnosis support method, and image diagnosis support program
JP2017153852A
Threshold determination method
JP2022123488A
Diagnosis-assistance-information generation method, image processing device, diagnosis-assistance-information generation system, and image processing program
WO2015190225A1
Image processing device, image processing method, and program
WO2017150194A1
