Region selection assistance method, region selection assistance device, and program

Abstract

A region selection assistance method comprises: a step (step S11) for dividing a stained image of a specimen obtained by an immunostaining method into a plurality of divided regions; a step (step S12) in which, for each of the plurality of divided regions, one display color is assigned from a display color group including a plurality of colors having different visibility, such that the visibility becomes higher as the aggregation of the stained cells in the divided regions becomes higher (step S12); a step (step S13) for generating a selected auxiliary image in which each of the plurality of divided regions is filled with the display color assigned in step S12; and displaying the selected auxiliary image on a screen (step S14). As a result, an operator can easily select a region suitable for analysis in the dyed image as an analysis region.

Description

Technical Field

[0001] This invention relates to a technique for assisting in the selection of regions to be resolved in stained images obtained by immunostaining.

[0002] [Reference to relevant applications]

[0003] This application claims the priority of Japanese Patent Application JP2023-045668, filed on March 22, 2023, all disclosures of which are incorporated herein by reference. Background Technology

[0004] In recent years, medical and biological research has focused on elucidating disease mechanisms, biological mechanisms, and drug action mechanisms by analyzing single-cell units of biological specimens, such as tissue or cell specimens. For example, in immunostaining analysis, quantitative / qualitative analysis is performed by staining biological substances such as proteins and measuring the staining state.

[0005] International Patent Application Publication No. 2015 / 190225 (Reference 1) proposes a technique that divides a single cell specimen image into multiple partitions and scores the malignancy of diseases such as cancer based on the expression levels and distribution of biomarkers in each partition. Additionally, Japanese Patent Application Publication No. 2017-153852 (Reference 2) proposes a technique that, during dynamic examinations performed via CT, MRI, etc., determines pixels representing blood flow characteristics on diagnostic images based on the time-dependent changes in contrast agents within lesions.

[0006] However, in the analysis of tissue specimens, analyzing the entire specimen at the single-cell level would be very time-consuming. Therefore, it is considered to extract a portion of the specimen and analyze that portion. When extracting this portion, it is necessary to appropriately select a region suitable for analysis that appropriately represents the characteristics of the specimen (i.e., a region that appropriately includes the various cells to be analyzed).

[0007] However, even when an operator directly views an image of a tissue specimen, if the number of stained cells (hereinafter also referred to as "stained cells") is small, stained cells may not be visually distinguishable at a magnification that shows the overall tissue. Furthermore, the visual distinguishability of stained cells may also be reduced due to noise outside the stained cells (e.g., dark or white pixels caused by the structure of the tissue or cells). Therefore, it is not easy for an operator directly viewing an image of a tissue specimen to select a suitable area for resolution. Summary of the Invention

[0008] This invention relates to a region selection aid method for selecting a region in a stained image obtained by immunostaining, with the aim of easily selecting a region in the stained image suitable for resolution.

[0009] Method 1 of the present invention is a region selection assistance method for assisting in the selection of a region to be resolved in a stained image obtained by immunostaining, wherein the region selection assistance method comprises the following steps: (a) dividing the stained image of the specimen obtained by immunostaining into a plurality of segmented regions; (b) assigning a display color from a set of multiple display colors containing different colors of salience to each of the plurality of segmented regions, such that the salience increases as the aggregation of stained cells in the segmented region increases; (c) generating a selection assistance image, which is an image obtained by filling each of the plurality of segmented regions with the display color assigned in step (b); and (d) displaying the selection assistance image on a screen.

[0010] According to the present invention, it is possible to easily select regions in a stained image suitable for analysis.

[0011] In the region selection auxiliary method of the present invention, in step (b), the aggregation of stained cells in the segmented region is determined based on the staining intensity in the segmented region in step (b).

[0012] In the region selection assistance method of Method 2, the third aspect of the present invention calculates the aggregation of stained cells in the segmented region and includes the following steps: (e) obtaining the staining intensity of each of a plurality of small regions configured in a matrix that constitute the segmented region; and (f) using the maximum value of the staining intensity of the plurality of small regions as a representative value of the segmented region for image reduction.

[0013] In the region selection auxiliary method of the present invention, in step (b), the aggregation of stained cells in the segmented region is determined based on the number of positive cells in the segmented region in step (b).

[0014] In the region selection assistance method of any one of methods 1 to 4, in step (d), the stained image, or other selected assistance images generated from other stained images of the specimen obtained by immunostaining using the same method as the selected assistance image, are arranged and displayed on the screen in a state of alignment with the selected assistance image.

[0015] In the region selection assistance method of Method 5, Method 6 of the present invention allows the display magnification of the selected auxiliary image to be changed in conjunction with the display magnification of the stained image or the other selected auxiliary image.

[0016] In the region selection assistance method of Method 5 (or Method 5 or 6) of the present invention, the display range of the selected auxiliary image on the specimen can be changed in conjunction with the display range of the stained image or the other selected auxiliary image on the specimen.

[0017] In the region selection assistance method of the present invention, in aspect 8, a position indicator displayed on the selected auxiliary image and indicating the position on the image is capable of moving in conjunction with a position indicator displayed on the stained image or the other selected auxiliary image and indicating the position on the image.

[0018] In the region selection auxiliary method of Method 5 (or any one of Methods 5 to 8) of the present invention, in step (d), the scale of the selected auxiliary image relative to the specimen is the same as the scale of the stained image or the other selected auxiliary image relative to the specimen.

[0019] In any one of methods 1 to 4 (or any one of methods 1 to 9) of the present invention, the region selection assistance method further comprises the following steps: (g) prior to step (b), dividing other stained images of the specimen obtained by immunostaining into multiple segmented regions in the same manner as the segmentation method in step (a), and assigning the display color in step (b) according to the aggregation of stained cells in the stained images and other stained images in the segmented regions.

[0020] In any of the methods 1 to 4 (or any of the methods 1 to 10) of the present invention, other stained images of the specimen obtained by immunostaining are thresholded to generate a binary image representing positive and negative regions. In step (d), the selected auxiliary image is masked by the positive or negative regions.

[0021] In the region selection auxiliary method of any one of methods 1 to 4 (or any one of methods 1 to 11) of the present invention, thresholding is performed on other stained images of the specimen obtained by immunostaining to obtain the boundary line of the positive or negative region, and in step (d), the boundary line is displayed overlappingly in the selected auxiliary image.

[0022] Embodiment 13 of the present invention is a region selection aid device that assists in selecting a region in a stained image obtained by immunostaining. The region selection aid device comprises: an image segmentation unit that segments the stained image of a specimen obtained by immunostaining into multiple segmented regions; a display color allocation unit that allocates a display color from a set of multiple colors containing different saliences for each of the multiple segmented regions, such that salience increases as the aggregation of stained cells in the segmented regions increases; a selection aid image generation unit that generates a selection aid image, which is an image obtained by filling each of the multiple segmented regions with the display color allocated by the display color allocation unit; and a screen that displays the selection aid image.

[0023] Method 14 of the present invention is a program that assists in the selection of a resolution region in a stained image obtained by immunostaining, wherein the program is executed by a computer to perform the following steps: (a) dividing the stained image of the specimen obtained by immunostaining into a plurality of segmented regions; (b) assigning a display color from a set of multiple colors containing different saliences to each of the plurality of segmented regions, such that salience increases as the aggregation of stained cells in the segmented region increases; (c) generating a selected auxiliary image, which is an image obtained by filling each of the plurality of segmented regions with the display color assigned in step (b); and (d) displaying the selected auxiliary image on a screen.

[0024] The above-mentioned objectives, as well as other objectives, features, forms, and advantages, will become clear from the following detailed description of the invention with reference to the accompanying drawings. Attached Figure Description

[0025] Figure 1 This is a diagram illustrating the structure of an image display device according to one embodiment.

[0026] Figure 2 It is a block diagram representing the functional structure of an image display device.

[0027] Figure 3 It is a diagram representing a stained image.

[0028] Figure 4 This represents a selected auxiliary image.

[0029] Figure 5 This is a diagram illustrating the display process of the selected auxiliary image.

[0030] Figure 6 This is a diagram representing part of the display process of the selected auxiliary image.

[0031] Figure 7 This is a diagram representing part of the display process of the selected auxiliary image.

[0032] Figure 8 This is an example of what is displayed on the screen.

[0033] Figure 9 This is an example of what is displayed on the screen.

[0034] Figure 10 This is an example of what is displayed on the screen.

[0035] Figure 11 This is an example of what is displayed on the screen.

[0036] Figure 12 This is an example of what is displayed on the screen.

[0037] Figure 13 This is an example of what is displayed on the screen.

[0038] Figure 14 This is an example of what is displayed on the screen.

[0039] Figure 15 This is an example of what is displayed on the screen.

[0040] Figure 16 This is an example of what is displayed on the screen.

[0041] Figure 17 This is an example of what is displayed on the screen.

[0042] Figure 18 This is an example of what is displayed on the screen.

[0043] Figure 19 This is an example of what is displayed on the screen.

[0044] Figure 20 This is a diagram representing part of the display process of the selected auxiliary image. Detailed Implementation

[0045] Figure 1 This diagram illustrates the structure of a computer functioning as a region selection assisting device 1 according to one embodiment of the present invention. The region selection assisting device 1 is used to assist in selecting a region suitable for analysis from a stained image of a biological specimen obtained by immunostaining.

[0046] Figure 1This diagram illustrates the structure of a computer that functions as a region selection auxiliary device 1. The computer has the structure of a general computer system, including a CPU 51, ROM 52, RAM 53, a fixed disk 54, a display 55, an input unit 56, a reading device 57, a communication unit 58, a GPU 59, and a bus 50. The CPU 51 performs various arithmetic operations. The GPU 59 performs various arithmetic operations related to image processing. The ROM 52 stores the basic program. The RAM 53 stores various information. The fixed disk 54 stores information. The display 55 is a display unit that displays images and other information.

[0047] The input unit 56 includes a keyboard 56a and a mouse 56b for receiving input from the operator. The reading device 57 reads information from a computer-readable recording medium 571, such as an optical disc, magnetic disk, optical disc drive, or memory card. The display 55, keyboard 56a, mouse 56b, and reading device 57 are connected to the bus 50 via an interface I / F. The communication unit 58 transmits and receives signals between the display 56 and external devices such as the area selection auxiliary device 1. The bus 50 is a signal circuit that connects the CPU 51, GPU 59, ROM 52, RAM 53, fixed disk 54, display 55, input unit 56, reading device 57, and communication unit 58.

[0048] In the region selection assist device 1, the program 572 is read from the recording medium 571 in advance via the reading device 57 and stored in the fixed disk 54. The program 572 can also be stored in the fixed disk 54 via a network. The CPU 51 and GPU 59 perform arithmetic processing simultaneously using the RAM 53 and the fixed disk 54 according to the program 572. The CPU 51 and GPU 59 function as a computing unit in the region selection assist device 1. In addition to the CPU 51 and GPU 59, other structures that function as computing units can also be used.

[0049] Figure 2 This is a block diagram illustrating the functional structure of the region selection auxiliary device 1, which is implemented by the computer performing calculations and processing according to program 572. These functional structures include: a storage unit 501, an image segmentation unit 502, a display color allocation unit 503, a selection auxiliary image generation unit 504, a display control unit 505, a resolution image acquisition unit 506, a mask generation unit 507, and a boundary line acquisition unit 508. All or part of these functions can be implemented using dedicated circuitry. Alternatively, these functions can be implemented using multiple computers.

[0050] Figure 2The storage unit 501 in the shown functional structure is mainly implemented by RAM 53 and fixed disk 54. The image segmentation unit 502, display color allocation unit 503, selected auxiliary image generation unit 504, display control unit 505, image acquisition unit 506 for analysis, mask generation unit 507, and boundary line acquisition unit 508 are implemented by CPU 51, GPU 59, ROM 52, RAM 53, fixed disk 54, and their peripheral structures.

[0051] Storage unit 501 pre-stores stained images derived from biological specimens (hereinafter also referred to as "specimens"). Figure 3 This diagram illustrates an example of a stained image 91 of a specimen 90 stored in storage unit 501. The stained image 91 is an image obtained by photographing the specimen 90 after staining using an immunostaining method. For example, the stained image 91 is a single-stain image captured by staining the specimen 90 using a specific staining method. This staining method may be, for example, a staining method using a CD45 antibody that detects the CD45 protein, one of the CD categories. In this case, in the stained image 91, cells 96 detected by the CD45 antibody, among the numerous cells contained in the specimen 90, are displayed stained in a single color. Furthermore, the staining of the specimen 90 can be based on various staining methods using antibodies, or on various staining methods without using antibodies.

[0052] Figure 2 The storage unit 501 shown also pre-stores the display color group used in the selected auxiliary image, described later. This display color group includes multiple colors (i.e., display colors) with varying degrees of saturation. "Saturation" refers to the degree to which it attracts the attention of an operator or similar visually oriented person. For example, warm colors are more saturated than cool colors, and saturation increases with brightness. Furthermore, the aforementioned "multiple colors" do not consist solely of the same color with varying shades (i.e., monochrome), but rather include multiple colors with different hues. For example, the display color group includes "yellow," "green," "light blue," and "dark blue" as display colors, with saturation ranging from high to low. In this embodiment, as described above, the stained image 91 is a monochromatic image (i.e., an image displayed in monochrome), therefore, the recognizability of the display color group containing multiple colors is higher than that of the color group (i.e., monochrome) constituting the stained image 91.

[0053] The image segmentation unit 502 segments the stained image 91 into multiple segmentation regions 95. Figure 3In the example shown, the stained image 91 is divided into multiple segmented regions 95 arranged in a matrix along the vertical and horizontal axes of the image. Each segmented region 95 is, for example, square. Each segmented region 95 is, for example, larger than each pixel constituting the stained image 91. In other words, each segmented region 95 contains multiple pixels. Furthermore, each segmented region 95 is, for example, larger than each cell 96 contained in the stained image 91. In addition, each segmented region 95 may also have various shapes other than squares (e.g., rectangles). Furthermore, in the stained image 91, segmented regions 95 may not be generated outside the area of ​​the specimen 90. In the actual stained image 91, no lines indicating the boundaries of the multiple segmented regions 95 are included; therefore, on the display 55 (refer to...), Figure 1 If the stained image 91 is displayed on the screen, the boundary is not shown.

[0054] The color distribution unit 503 determines the aggregation of stained cells 96 (hereinafter also referred to as "stained cells 96") in each of the plurality of segmented regions 95. Furthermore, based on the aggregation of stained cells 96 in each segmented region 95, the color distribution unit 503 assigns a color from the aforementioned color group to each segmented region 95. The salience of the color assigned to the segmented region 95 increases with the aggregation of stained cells 96 in the segmented region 95. Additionally, in the color distribution unit 503, areas outside the specimen 90 are assigned "black" as a background color.

[0055] The auxiliary image generation unit 504 generates a selected auxiliary image corresponding to the stained image 91. Figure 4 This diagram illustrates an example of a selected auxiliary image 81. The selected auxiliary image 81 is an image obtained by filling each segmented region 95 of the stained image 91 with a display color allocated by the display color allocation unit 503.

[0056] For example, stained cells 96 (reference) Figure 3 The segmented region 95, containing the highest aggregation range, is filled with "yellow," forming the first region 811 in the selected auxiliary image 81. Figure 4 In the illustration, for ease of understanding, the two first regions 811 are not marked with parallel diagonal lines, but are instead represented as white areas. The segmented region 95, contained within the area of ​​the second highest aggregation of stained cells 96, is filled with "green," constituting the second region 812 in the selected auxiliary image 81. Figure 4 In the image, the second region 812 is marked with relatively large parallel diagonal lines. The segmented region 95, containing the area with the third highest aggregation of stained cells 96, is filled with light blue, constituting the third region 813 in the selected auxiliary image 81. Figure 4In the image, the third region 813 is marked with parallel diagonal lines with smaller intervals than the second region 812. The segmented region 95, containing the area with the lowest aggregation of stained cells 96, is filled with "dark blue," constituting the fourth region 814 in the selected auxiliary image 81. Figure 4 In the middle, the parallel oblique lines with the smallest intervals are marked in the fourth region 814. Figure 4 In the example shown, the spacing of the parallel diagonal lines is reduced as the salience of the displayed color decreases.

[0057] Figure 2 The display control unit 505 shown causes the selected auxiliary image 81 to be displayed on the display 55 (see reference). Figure 1 The display control unit 505 can also change the magnification of the selected auxiliary image 81 displayed on the screen. In addition, it can also change the display range of the selected auxiliary image 81 on the specimen 90.

[0058] The image acquisition unit 506 for analysis cuts out a region on the stained image 91 that is set using the selected auxiliary image 81 displayed on the display 55 from the stained image 91, and stores it in the storage unit 501 as an image for detailed analysis.

[0059] Next, refer to Figures 5 to 7 The display process of the selected auxiliary image 81 based on the region selection auxiliary device 1 will be explained. First, as... Figure 3 As shown, the image segmentation unit 502 (refer to...) Figure 2 The stained image 91 is divided into multiple segmented regions 95 (step S11). Next, the color distribution unit 503 (see reference) displays the color. Figure 2 For each of the multiple segmented regions 95, the aggregation of stained cells 96 is determined. Then, a display color is assigned to each segmented region 95, such that as the aggregation of stained cells 96 increases, the visibility increases (step S12).

[0060] In step S12, the aggregation of stained cells 96 in the segmented region 95 can be determined using various methods. For example, the aggregation of stained cells 96 in the segmented region 95 can be determined based on the number of positive cells in the segmented region 95. Specifically, firstly, the location of stained cells 96 in the stained image 91 is determined ( Figure 6 (Step S21) To determine the location of the stained cells 96, known techniques such as image segmentation based on deep learning can be used. Next, for each identified stained cell 96, feature quantities (e.g., average brightness within the cell region) related to the prescribed analytical method are calculated (Step S22).

[0061] Next, the presence or absence of a positive staining cell 96 is determined by comparing the aforementioned characteristic values ​​of each stained cell 96 with a predetermined threshold (step S23). Then, the number of stained cells 96 (i.e., positive cells) determined to be positive in each segmented region 95 is calculated (step S24). Thus, the number of positive cells consistent with the above-described analysis method can be obtained. Afterward, for each segmented region 95, a display color is assigned from the aforementioned display color group, such that the conspicuousness increases as the number of positive cells increases (i.e., the aggregation increases) (step S25).

[0062] Alternatively, the aggregation of stained cells 96 within segmented region 95 can be determined based on the staining intensity within that segment. When segmented region 95 contains multiple stained cells 96, the average staining intensity of segmented region 95 depends on the density of stained cells 96 within it. Therefore, a higher staining intensity in segmented region 95 indicates a higher aggregation of stained cells 96, and vice versa.

[0063] Figure 7 This is a graph illustrating an example of calculating the aggregation of 96 stained cells based on staining intensity. Figure 7 In the example shown, firstly, after performing noise removal processing (e.g., blurring) on ​​the stained image 91, color decomposition processing is performed to extract a color corresponding to the staining of the stained cells 96. Next, the segmentation region 95 is divided into multiple small regions arranged in a matrix (e.g., four small regions of 2×2), and the staining intensity of each small region (the so-called kernel) is calculated for the extracted color (step S31). Next, max pooling is performed, and the maximum value of the staining intensity among the multiple small regions (i.e., the largest staining intensity) is used as the representative value of the staining intensity of the segmentation region 95 for image reduction (step S32). Then, for each segmentation region 95, a display color is assigned from the display color group, such that the salience increases as the staining intensity increases (i.e., the aggregation increases) (step S33).

[0064] In the methods shown in steps S31 to S33, it is not necessary to determine each individual stained cell 96 in the stained image 91. Therefore, compared with the methods shown in steps S21 to S25, the amount of processing required to assign display colors to each segmented region 95 can be reduced, and processing can be performed using the stained image 91 obtained at low magnification. For example, according to the methods shown in steps S31 to S33, even when using a stained image 91 with a resolution (e.g., a pixel pitch of 10 μm) where the size of one pixel is the same as the size of the cell, the display colors can be appropriately assigned to improve the visibility of areas with high aggregation of stained cells (i.e., areas suitable for resolution). Furthermore, in the methods shown in steps S31 to S33, even when the number of stained cells 96 contained in the stained image 91 is small, the display colors can be appropriately assigned.

[0065] When step S12 ends, the auxiliary image generation unit 504 (see reference) is selected. Figure 2 Generate selected auxiliary image 81 (refer to) Figure 4 The selected auxiliary image 81 is an image in which each of the plurality of segmented regions 95 is filled with the display color assigned in step S12 (step S13). Then, the display control unit 505 (see reference) displays the image. Figure 2 The selected auxiliary image 81 is displayed on the screen of the display 55 (step S14).

[0066] In the area selection assist device 1, the display magnification of the selected assist image 81 on the display 55 and the display range of the selected assist image 81 on the specimen 90 can be changed (i.e., the range of the specimen 90 displayed on the screen as the selected assist image 81). The operator can make various changes to the display mode of the selected assist image 81 as needed, and specify the area for analysis (hereinafter also referred to as the "analysis area") on the selected assist image 81 (step S15). The analysis image acquisition unit 506 cuts out the area on the stained image 91 corresponding to the analysis area and transmits it to the storage unit 501 as an image for detailed analysis (step S16).

[0067] The display control unit 505 can display only the selected auxiliary image 81 on the screen of the display 55, or it can display various other images together with the selected auxiliary image 81 on the screen of the display 55. For example, in step S14, if... Figure 8 As shown, the selected auxiliary image 81 and the tinted image 91 can be arranged and displayed on the screen 551 of the display 55 in a non-overlapping manner. Figure 8In the example shown, the selected auxiliary image 81 and the stained image 91 are displayed at the same scale in a mutually aligned state. That is, the selected auxiliary image 81 and the stained image 91 are corrected and displayed as needed, such that the size and position of the specimen 90 in the selected auxiliary image 81 are approximately the same as the size and position of the specimen 90 in the stained image 91. Thus, the operator can easily compare the selected auxiliary image 81 with the stained image 91.

[0068] Preferably, in the region selection assist device 1, the display magnification of the selection assist image 81 and the display magnification of the staining image 91 on the screen 551 can be changed in conjunction with each other. Furthermore, it is preferable that the display range of the selection assist image 81 on the specimen 90 and the display range of the staining image 91 on the specimen 90 (i.e., the range of the specimen 90 displayed as the staining image 91 on the screen 551) can be changed in conjunction with each other.

[0069] For example, when an operator places the mouse pointer (without the illustration) over the selected auxiliary image 81 and manipulates the mouse wheel to change the display magnification of the selected auxiliary image 81, such as... Figure 9 As shown, the display magnification of the tinted image 91 on screen 551 is also changed to be the same as the display magnification of the selected auxiliary image 81. When the display magnification of the selected auxiliary image 81 is greater than the initial state, only a portion of the selected auxiliary image 81 and a portion of the tinted image 91 are displayed on screen 551. The same applies when the operator changes the display magnification of the tinted image 91; the display magnification of the selected auxiliary image 81 is changed accordingly.

[0070] Therefore, the operator can compare the selected auxiliary image 81 and the stained image 91 at the same display magnification without having to adjust the display magnification of the selected auxiliary image 81 and the stained image 91 separately.

[0071] In addition, such as Figure 9 As shown, the operator selects auxiliary image 81 and stained image 91 compared to the initial state (refer to...). Figure 8 In the zoomed-in state, place the mouse pointer on the selected auxiliary image 81, and operate the mouse buttons, etc., to change the specimen 90 of the selected auxiliary image 81 (refer to...). Figure 8 The display range on the specimen 90 is changed (i.e., the position of the display range of the selected auxiliary image 81 on the specimen 90 is altered). Accompanying this, such as... Figure 10 As shown, the display area of ​​the stained image 91 on the specimen 90 on screen 551 is also changed accordingly to be the same as the display area of ​​the selected auxiliary image 81. The same applies when the operator changes the display area of ​​the stained image 91 on the specimen 90; the display area of ​​the selected auxiliary image 81 on the specimen 90 is also changed accordingly.

[0072] Thus, the operator can immediately observe the area of ​​interest on one of the selected auxiliary images 81 and the stained image 91 without having to search for the area again on the other image.

[0073] In the area selection auxiliary device 1, such as Figures 8 to 10 As shown, position indicators 553, indicating positions on each of the selected auxiliary image 81 and the tinted image 91, can be displayed respectively. The position indicators 553 can be, for example, crosshairs or rectangles that can be moved by a mouse 56b, etc. Preferably, the two position indicators 553 displayed on the selected auxiliary image 81 and the tinted image 91 can move in tandem with the movement of the mouse 56b, etc. This allows the operator to easily focus on approximately the same positions indicated by the position indicators 553 in the selected auxiliary image 81 and the tinted image 91.

[0074] The operator can make various changes to the display mode of the selected auxiliary image 81 and the stained image 91 as needed, and specify the resolution area on the selected auxiliary image 81. Specifically, the operator uses a mouse 56b, etc. Figure 11 The illustrated analysis window 87 is set at a desired position on the selected auxiliary image 81. In the region selection auxiliary device 1, in conjunction with the setting of the analysis window 87 in the selected auxiliary image 81, the analysis window 97 is also displayed on the stained image 91. The size of the analysis window 97 and its relative position on the stained image 91 relative to the specimen 90 are the same as the size of the analysis window 87 and its relative position on the selected auxiliary image 81 relative to the specimen 90. Figure 11 In the example shown, the parsing window 87 is rectangular in shape and its size can be changed. When the size or position of the parsing window 87 is changed, the size or position of the parsing window 97 is also changed accordingly.

[0075] If the settings for the parsing windows 87 and 97 (i.e., the specification of the parsing region) are completed, the image acquisition unit 506 for parsing (see reference) Figure 2 The region contained in the analysis window 97 of the stained image 91 is cut out and transported to the storage unit 501 as an image for detailed analysis. Thus, the operator can easily identify the distribution of the aggregation of stained cells 96 in the selected auxiliary image 81 and select a region in the stained image 91 suitable for analysis based on this distribution. For example, the operator can identify a region with high aggregation of stained cells 96 in the selected auxiliary image 81 and select a region suitable for analysis from the regions on the stained image 91 corresponding to that region.

[0076] Alternatively, the operator can directly set the analysis window 97 on the stained image 91, instead of setting the analysis window 87 on the selected auxiliary image 81. In this case, the analysis window 87, which is linked to the analysis window 97, can be displayed on the selected auxiliary image 81. Furthermore, as... Figure 9 and Figure 10 As shown, the resolution windows 87 and 97 can also be set in the screen 551 after the display magnification or display range of the selected auxiliary image 81 and the stained image 91 has been changed from the initial state.

[0077] In step S14, as Figure 12 As shown, other stained images 92 that are different from stained image 91, and other selected auxiliary images 82 generated from these other stained images 92 using the same method as the selected auxiliary image 81, can be arranged and displayed on screen 551 in a manner that does not overlap with stained image 91 and selected auxiliary image 81.

[0078] Staining image 92 is a single-staining image obtained by staining the same specimen 90 as staining image 91 using a different staining method (e.g., immunostaining using CD20 antibody). Staining images 91 and 92 can be obtained, for example, by sequentially staining and photographing a specimen 90 using a first antibody, destaining the specimen 90, and then staining and photographing using a second antibody. Alternatively, multiple single-staining images can be obtained by staining a specimen 90 with multiple antibodies, followed by color decomposition of the captured multicolor images according to the colors corresponding to each antibody.

[0079] In the selected auxiliary image 82, similar to the selected auxiliary image 81, the segmented regions containing the highest aggregation of stained cells 96 are filled with "yellow," constituting the first region 821 in the selected auxiliary image 82. Figure 12 In the illustration, for ease of understanding, the first region 821 is not marked with parallel diagonal lines and is represented as a white area. The segmented region contained within the area of ​​the second highest aggregation of stained cells 96 is filled with "green," constituting the second region 822 in the selected auxiliary image 82. Figure 12 In the second region 822, parallel diagonal lines with relatively large intervals are marked. The segmented region containing the area with the third highest aggregation of stained cells 96 is filled with "light blue," constituting the third region 823 in the selected auxiliary image 82. Figure 12 In the image, the third region 823 is marked with parallel diagonal lines at smaller intervals than the second region 822. The segmented regions containing the lowest aggregation of stained cells 96 are filled with "dark blue," forming the fourth region 824 in the selected auxiliary image 82. Figure 12 In the middle, the parallel oblique lines with the smallest intervals are marked for the fourth region 824. Figure 12 In the example shown, the spacing of the parallel diagonal lines is reduced as the salience of the displayed color decreases.

[0080] exist Figure 12 In the example shown, the selected auxiliary image 81, staining image 91, selected auxiliary image 82, and staining image 92 are displayed in a mutually aligned state and at the same scale relative to specimen 90. That is, the selected auxiliary images 81, 82, and staining images 91, 92 are corrected and displayed as needed so that the size and position of specimen 90 in selected auxiliary image 81, staining image 91, selected auxiliary image 82, and staining image 92 are approximately the same. This allows the operator to easily compare the selected auxiliary images 81, 82, and staining images 91, 92.

[0081] The alignment of the selected auxiliary images 81 and 82 and the stained images 91 and 92 can be performed automatically, for example, using known image registration techniques, or manually by the operator. When performing manual alignment, the operator first operates mouse 56b (see...). Figure 1 For each of the selected auxiliary images 81 and 82 and the stained images 91 and 92, a reference point is set. This reference point is set in each of the selected auxiliary images 81 and 82 and the stained images 91 and 92 as a point considered to be at the same location on the specimen 90. Then, in each of the selected auxiliary images 81 and 82 and the stained images 91 and 92, the display range of the image is adjusted so that the reference point is located at a predetermined position (e.g., the center of each image).

[0082] Preferably, in the region selection assist device 1, the display magnification of the selection assist image 81 and the staining image 91 on the screen 551 can be changed in conjunction with the display magnification of the selection assist image 82 and the staining image 92. Furthermore, preferably, the display range of the selection assist image 81 and the staining image 91 on the specimen 90 can be changed in conjunction with the display range of the selection assist image 82 and the staining image 92 on the specimen 90 (i.e., the range of the specimen 90 displayed on the screen 551 as the selection assist image 82 and the staining image 92).

[0083] For example, when an operator places the mouse pointer (without the illustration) over the selected auxiliary image 81 and manipulates the mouse wheel to change the display magnification of the selected auxiliary image 81, such as... Figure 13As shown, the display magnification of the selected auxiliary image 82 and the tinted images 91 and 92 on screen 551 is also changed to be the same as the display magnification of the selected auxiliary image 81. When the display magnification of the selected auxiliary image 81 is greater than the initial state, only a portion of the selected auxiliary images 81 and 82 and a portion of the tinted images 91 and 92 are displayed on screen 551. Similarly, if the operator changes the display magnification of the selected auxiliary image 82, tinted image 91, or tinted image 92, the display magnification of the other three images is also changed accordingly.

[0084] Therefore, the operator can compare the selected auxiliary images 81 and 82 and the dyed images 91 and 92 at the same display magnification without having to adjust the display magnification separately in the selected auxiliary images 81 and 82 and the dyed images 91 and 92.

[0085] In addition, operators in such Figure 13 As shown, auxiliary images 81 and 82 and stained images 91 and 92 were selected compared to the initial state (refer to...). Figure 12 In the magnified state, place the mouse pointer on the selected auxiliary image 81, and operate the mouse buttons, etc., to change the specimen 90 of the selected auxiliary image 81 (refer to...). Figure 12 The display range on ) . Along with this, such as Figure 14 As shown, the display range of the selected auxiliary image 82 and staining images 91 and 92 on the specimen 90 on screen 551 is also changed in conjunction with the display range of the selected auxiliary image 81. Similar to the case where the operator changes the display range of the selected auxiliary image 82, staining image 91, or staining image 92 on the specimen 90, the display range of the three other images on the specimen 90 is also changed in conjunction with the display range of these images.

[0086] Therefore, the operator can immediately observe the region of interest on one of the selected auxiliary images 81, 82 and the stained images 91, 92 without having to search for the region again on the other three images.

[0087] In the area selection auxiliary device 1, such as Figures 12 to 14 As shown, position indicators 553, representing the positions on each of the selected auxiliary images 81 and 82 and the stained images 91 and 92, can be displayed. The position indicators 553 can be, for example, crosshairs or rectangles that can be moved using a mouse 56b or similar device. Preferably, the four position indicators 553 displayed on the selected auxiliary images 81 and 82 and the stained images 91 and 92 can move in tandem with the movement of the mouse 56b or similar device. This allows the operator to easily focus on approximately the same positions indicated by the position indicators 553 in the selected auxiliary images 81 and 82 and the stained images 91 and 92.

[0088] The operator can make various changes to the display mode of the selected auxiliary images 81 and 82 and the stained images 91 and 92 as needed, and specify the resolution area on the selected auxiliary image 81 or the selected auxiliary image 82. For example, the operator can use a mouse 56b, etc. Figure 15 The illustrated analysis window 87 is set at a desired position on the selected auxiliary image 81, thereby specifying the analysis area (step S15). In the area selection auxiliary device 1, in conjunction with the setting of the analysis window 87 in the selected auxiliary image 81, the analysis window 88 is displayed on the selected auxiliary image 82, and the analysis windows 97 and 98 are displayed on the stained images 91 and 92.

[0089] The size of the analysis window 88, its relative position on the selected auxiliary image 82 relative to the specimen 90, the size of the analysis window 97, its relative position on the stained image 91 relative to the specimen 90, the size of the analysis window 98, and its relative position on the stained image 92 relative to the specimen 90 are the same as the size of the analysis window 87 and its relative position on the selected auxiliary image 81 relative to the specimen 90. Figure 15 In the example shown, the parsing window 87 is rectangular in shape and its size can be changed. When the size or position of the parsing window 87 is changed, the size and position of the parsing windows 88, 97, and 98 are also changed accordingly.

[0090] When the settings for the parsing windows 87, 88, 97, and 98 (i.e., the specification of the parsing region) are completed, the image acquisition unit 506 for parsing (see reference) Figure 2 The regions contained in the analysis window 97 of the stained image 91 and / or the analysis window 98 of the stained image 92 are cut out and transported to the storage unit 501 as images for detailed analysis (step S16). Thus, the operator can easily identify the distribution of the aggregation of stained cells 96 in the selected auxiliary images 81 and 82, and select regions suitable for analysis in the stained images 91 and 92 based on this distribution. For example, the operator can identify on the screen 551 the region with the highest aggregation of stained cells 96 in both the selected auxiliary images 81 and 82 (in the above example, the region shown in "yellow"), and select a region suitable for analysis from the regions on the stained images 91 and 92 corresponding to that region.

[0091] Alternatively, the operator can set the analysis window 87 on the selected auxiliary image 81 and the analysis window 88 on the selected auxiliary image 82. In this case, the analysis windows 87, 97, and 98, which are linked to the analysis window 88, are also displayed on the selected auxiliary image 81, the stained image 91, and the stained image 92, respectively.

[0092] Alternatively, the operator can set the analysis window 87 by performing operations on the selected auxiliary image 81, or directly set the analysis window 97 on the stained image 91, or directly set the analysis window 98 on the stained image 92. In these cases, the analysis windows 87 and 88, which are linked to the analysis window 97 or analysis window 98, can be displayed on the selected auxiliary images 81 and 82. Furthermore, as... Figure 13 and Figure 14 As shown, the resolution windows 87, 88, 97, and 98 can be set in the screen 551 after the initial state is changed, where the display magnification or display range of the selected auxiliary images 81, 82 and the stained images 91, 92 is selected.

[0093] As described above, when the resolution region is specified based on both stained images 91 and 92, in step S14, the selected auxiliary image 81 and the selected auxiliary image 82 do not necessarily need to be displayed side-by-side on screen 551. For example, as Figure 16 and Figure 17 As shown, the mask generation unit 507 (refer to...) Figure 2 According to the stained image 92 (reference) Figure 12 The generated masks 85a and 85b are displayed overlappingly on the selected auxiliary image 81, thereby selecting only a portion of the auxiliary image 81 for display in a visually confirmable state. The mask generation unit 507 performs thresholding processing on the stained image 92, which is aligned with the stained image 91, thereby generating a binary image representing positive and negative regions. Furthermore, one of the positive and negative regions is designated as a transparent region, and the other as a mask region (i.e., an opaque region), thereby generating masks 85a and 85b.

[0094] exist Figure 16 In the illustrated mask 85a, only the region containing the highest aggregation of stained cells 96 in the stained image 92 (i.e., the region with...) Figure 12 The area corresponding to the first region 821 shown is designated as a positive region, which is set as an opening 851a for the transmission region. In addition, the area outside the opening 851a (i.e., the negative region) is filled with "black" as the background color to become a mask area.

[0095] exist Figure 16In the example shown, a mask 85a generated from staining image 92 is overlaid on a selected auxiliary image 81 generated from staining image 91. In the selected auxiliary image 81, only regions overlapping with the positive regions of staining image 92 are visually identifiable. Therefore, by specifying a resolution region within this visually identifiable region, the operator can select a resolution region based on the aggregation of stained cells 96 in staining image 91, taking into account the positive regions defined in staining image 92. For example, by specifying a resolution region within the opening 851a on the first region 811 of the selected auxiliary image 81, a region with high aggregation of stained cells 96 in staining image 91 can be selected as the resolution region based on the positive regions defined in staining image 92.

[0096] exist Figure 17 In the illustrated mask 85b, the regions contained in the area with the highest aggregation of stained cells 96 in the stained image 92 and the regions contained in the area with the second highest aggregation (i.e., with...) Figure 12 The area corresponding to the first region 821 and the second region 822 shown is designated as a positive region, and this positive region is designated as an opening 851b, which is a transparent area. In addition, the area outside the opening 851b (i.e., the negative region) is filled with "black" as the background color to become a mask area.

[0097] exist Figure 17 In the example shown, the mask 85b generated from the stained image 92 is overlaid on the selected auxiliary image 81 generated from the stained image 91. In this case, the operator can also specify the resolution region in a visually identifiable area on the selected auxiliary image 81 (i.e., the area overlapping with the opening 851b of the mask 85b), thereby selecting the resolution region based on the aggregation of stained cells 96 in the stained image 91, while limiting it to the positive region in the stained image 92.

[0098] In addition, Figure 16 and Figure 17 In the example shown, the auxiliary image 81 is masked in the stained image 92 by a region designated as a negative region. However, it is also possible to mask in the stained image 92 by a region designated as a positive region, as described above. In this case, the operator can select the resolution region based on the negative region in the stained image 92, taking into account the aggregation of stained cells 96 in the stained image 91.

[0099] In step S14, masks 85a and 85b described above can be replaced by, for example... Figure 18 As shown, the boundary line acquisition part 508 (refer to) Figure 2 According to the stained image 92 (reference) Figure 12The obtained boundary lines 86a are displayed overlappingly on the selected auxiliary image 81. The boundary line acquisition unit 508, for example, generates a binary image representing positive and negative regions by thresholding the stained image 92, and obtains the boundary lines 86a of the positive and negative regions. Figure 18 In the example shown, only the region containing the highest aggregation of stained cells 96 in staining image 92 (i.e., with) Figure 12 The area corresponding to the first region 821 shown is designated as a positive region, and the other regions are designated as negative regions. Furthermore, the outer perimeter of the positive region is designated as a boundary line 86a.

[0100] By specifying the resolution region inside the boundary line 86a, the operator can select a resolution region that, based on the positive region defined in the stained image 92, takes into account the aggregation of stained cells 96 in the stained image 91. Furthermore, by specifying the resolution region outside the boundary line 86a, the operator can select a resolution region that, based on the negative region defined in the stained image 92, takes into account the aggregation of stained cells 96 in the stained image 91. Moreover, by specifying the resolution region at a location overlapping the boundary line 86a, the operator can select a resolution region that, based on the boundary between the positive and negative regions in the stained image 92, takes into account the aggregation of stained cells 96 in the stained image 91.

[0101] Additionally, in step S14, as Figure 19 As shown, the boundary line acquisition part 508 (refer to) Figure 2 The boundary lines 86b and 86c obtained from the stained image 92 can also be displayed overlappingly on the selected auxiliary image 81. For example, the boundary line acquisition unit 508 performs thresholding processing on the stained image 92 to generate an image representing positive regions, negative regions, and the regions where the positive and negative regions are separated (hereinafter also referred to as "boundary regions"), and obtains the boundary line 86b between the positive region and the boundary region, and the boundary line 86c between the boundary region and the negative region.

[0102] exist Figure 19 In the example shown, the region containing the highest aggregation of stained cells 96 in stained image 92 (i.e., with...) Figure 12 The region corresponding to the first region 821 shown is designated as the positive region, and the region contained in the range with the second highest aggregation of stained cells 96 (i.e., the region corresponding to the first region 821 shown) is designated as the positive region. Figure 12 The region corresponding to the second region 822 shown is designated as the boundary region, and the other regions are designated as negative regions. Furthermore, the outer periphery of the positive region is designated as boundary line 86b, and the inner periphery of the negative region is designated as boundary line 86c.

[0103] By specifying a resolution region inside boundary line 86b, the operator can select a resolution region based on the positive region defined in staining image 92, taking into account the aggregation of stained cells 96 in staining image 91. Furthermore, by specifying a resolution region between boundary lines 86b and 86c, the operator can select a resolution region based on the boundary region defined in staining image 92, taking into account the aggregation of stained cells 96 in staining image 91. And, by specifying a resolution region outside boundary line 86b, the operator can select a resolution region based on the negative region defined in staining image 92, taking into account the aggregation of stained cells 96 in staining image 91.

[0104] The assignment of display color to each segmented region 95 in step S12 is not necessarily based solely on the aggregation of stained cells 96 in a single stained image 91, but can also be based on the aggregation of stained cells 96 in each of multiple stained images obtained by staining the same specimen 90 using multiple different staining methods.

[0105] For example, in the case where the stained images 91 and 92 are used to display color allocation, the stained images 91 and 92 are first aligned before step S11. As described above, this alignment can be performed automatically using known image registration techniques or manually by the operator. Furthermore, if the scales of the stained images 91 and 92 relative to the specimen 90 are different, image processing is performed to make the scales the same.

[0106] Next, as described above, the stained image 91 is segmented into Figure 3 The multiple segmented regions shown are 95 ( Figure 5 Step S11). Furthermore, the stained image 92 is segmented into multiple segmented regions 95 in the same manner as the stained image 91. Figure 20 (Step S41). That is, multiple segmentation regions 95 with the same size and configuration as those set in the multiple segmentation regions 95 of the stained image 91 are set in the stained image 92. Furthermore, step S41 can be performed in parallel with step S11, or it can be performed before or after step S11.

[0107] When steps S11 and S41 are completed, the color distribution unit 503 is displayed (see reference). Figure 2 For each segmented region 95 of stained image 91 and each segmented region 95 of stained image 92, the aggregation of stained cells 96 is determined. Then, a display color is assigned to each segmented region 95, such that as the aggregation of stained cells 96 increases, the visibility increases (step S12).

[0108] The display color allocation in step S12 is performed based on the aggregation of stained cells 96 in each of the stained images 91 and 92 within the segmented region 95. For example, in each segmented region 95, the display color can be allocated based on the lower aggregation of stained cells 96 in stained image 91 and stained image 92. Alternatively, the display color can be allocated based on the higher aggregation of stained cells 96 in stained image 91 and stained image 92 within each segmented region 95. In either case, in step S14, a selected auxiliary image that takes into account the aggregation of stained cells 96 in multiple stained images can be displayed.

[0109] Alternatively, in step S12, a composite staining image is prepared by pixel-wise max-combining of multiple staining images (i.e., the image obtained by setting the largest pixel value among the multiple staining images to the pixel value of that pixel). Based on dividing this composite staining image into multiple segmented regions 95, the display color is assigned according to the aggregation of the stained cells 96 in each segmented region 95. In this case, by dividing the composite staining image into multiple segmented regions 95, for example, steps S11 and S41 described above are performed simultaneously. According to this method, in the selected auxiliary image displayed in step S14, the region with high aggregation of stained cells 96 in any of the multiple staining images can be displayed with high visibility.

[0110] As explained above, the method for selecting a region to be analyzed in a stained image obtained by immunostaining includes the following steps: dividing the stained image 91 of the specimen 90 obtained by immunostaining into multiple segmented regions 95 (step S11); assigning a display color from a set of multiple colors containing different colors of high saturation to each of the multiple segmented regions 95, such that the saturation increases as the aggregation of stained cells 96 in the segmented region 95 increases (step S12); generating a selection auxiliary image 81 (step S13), which is an image obtained by filling each of the multiple segmented regions 95 with the display color assigned in step S12; and displaying the selection auxiliary image 81 on the screen 551 (step S14). Thus, the degree of aggregation of stained cells 96 at various locations in the stained image 91 can be easily visually identified. As a result, the operator can easily select a region in the stained image 91 suitable for analysis as the analysis region based on the aggregation of stained cells 96.

[0111] As described above, in step S12, it is preferable to determine the aggregation of stained cells 96 in segmented region 95 based on the staining intensity in segmented region 95. In this case, it is not necessary to determine individual cells in stained image 91; therefore, the amount of processing required for color distribution in step S12 is reduced, and processing using stained image 91 obtained at low magnification can be performed. As a result, the time required for step S12 can be shortened.

[0112] As described above, preferably, the calculation of the aggregation of stained cells 96 in the segmented region 95 includes the following steps: obtaining the staining intensity of each of the multiple small regions constituting the segmented region 95 (step S31); and performing image reduction by using the maximum value of the staining intensity of the multiple small regions as the representative value of the segmented region 95 (step S32). Therefore, even when the number of stained cells 96 contained in the segmented region 95 is small, the prominence of the segmented region 95 in the selected auxiliary image 81 can be improved.

[0113] As described above, preferably, in step S12, the aggregation of stained cells 96 in segmented region 95 is determined based on the number of positive cells in segmented region 95. This allows for highly precise improvement in the visibility of areas with a high number of positive cells.

[0114] As described above, preferably, in step S14, the stained image 91, or other selected auxiliary images 82 generated from other stained images 92 of the specimen 90 obtained by immunostaining using the same method as the selected auxiliary image 81, are arranged and displayed on the screen 551 in a state aligned with the selected auxiliary image 81. When the selected auxiliary image 81 and the stained image 91 are displayed side by side, generally speaking, the stained image 91 is easier to understand regarding the tissue structure in the specimen 90. Therefore, the operator can select the analytical region while observing the actual dispersion morphology of the stained cells 96 in the stained image 91. In addition, regarding artifacts (e.g., cases where staining is non-specifically enhanced due to tissue folds, etc.) that are sometimes difficult to judge only in the selected auxiliary image 81, the operator can make a high-precision judgment by observing the selected auxiliary image 81 and the stained image 91. When the selected auxiliary image 81 and other selected auxiliary images 82 are displayed in an arrangement, the operator can easily select the analytical region that takes into account the aggregation of stained cells 96 in both the stained image 91 and the other stained image 92 by comparing the selected auxiliary image 81 and the other selected auxiliary image 82.

[0115] As described above, preferably, the display magnification of the selected auxiliary image 81 can be changed in conjunction with the display magnification of the stained image 91 or other selected auxiliary images 82. Thus, the operator can easily compare multiple images at the same display magnification without having to adjust the display magnification of each of the selected auxiliary image 81 and the stained image 91 or other selected auxiliary images 82.

[0116] As described above, preferably, the display range of the selected auxiliary image 81 on the specimen 90 can be changed in conjunction with the display range of the stained image 91 or other selected auxiliary images 82 on the specimen 90. Thus, the operator can immediately observe the area of ​​interest in one of the selected auxiliary images 81 and stained image 91 or other selected auxiliary images 82 without having to search for that area again in other images.

[0117] As described above, preferably, the position indicator 553 displayed on the selected auxiliary image 81 and indicating the position on the image is movable in conjunction with the position indicator 553 displayed on the stained image 91 or other selected auxiliary image 82 and indicating the position on the image. This allows the operator to easily focus on approximately the same position indicated by the position indicator 553 in both the selected auxiliary image 81 and the stained image 91 or other selected auxiliary image 82.

[0118] As described above, in step S14, it is preferable that the scale of the selected auxiliary image 81 relative to the specimen 90 is the same as the scale of the stained image 91 or other selected auxiliary image 82 relative to the specimen 90. This allows the operator to easily compare the selected auxiliary image 81 with the stained image 91 or other selected auxiliary image 82.

[0119] Preferably, the above-described region selection assistance method further includes the following step: before step S12, dividing other stained images 92 of the specimen 90 obtained by immunostaining into multiple segmented regions 95 in the same manner as the segmentation pattern in step S11 (step S41). Furthermore, it is preferable that the color assignment in step S12 is performed based on the aggregation of stained cells 96 in each of the stained images 91 and 92 within the segmented regions 95. This allows the operator to easily select analytical regions that take into account the aggregation of stained cells 96 in both stained images 91 and 92.

[0120] As described above, preferably, thresholding is performed on other stained images 92 of the specimen 90 obtained by immunostaining to generate binary images representing positive and negative regions. In step S14, the selected auxiliary image 81 is masked by the positive or negative region. Thus, the operator can select the resolvable region based on the aggregation of stained cells 96 in the stained image 91, while considering the positive or negative regions defined in the stained image 92.

[0121] As described above, preferably, thresholding is performed on other stained images 92 of the specimen 90 obtained by immunostaining to obtain boundary lines 86a, 86b, and 86c of positive or negative regions. In step S14, the boundary lines 86a, 86b, and 86c are displayed overlapping on the selected auxiliary image 81. Thus, the operator can select the resolution region by considering the aggregation of stained cells 96 in the stained image 91, taking into account both positive and negative regions in the stained image 92.

[0122] The aforementioned region selection assist device 1 includes: an image segmentation unit 502, a display color allocation unit 503, a selection assist image generation unit 504, and a screen 551. The image segmentation unit 502 segments the stained image 91 of the specimen 90 obtained by immunostaining into multiple segmented regions 95. The display color allocation unit 503 allocates a display color from a group of multiple colors containing different colors for each of the multiple segmented regions 95, such that the stigma increases as the aggregation of stained cells 96 in the segmented region 95 increases. The selection assist image generation unit 504 generates a selection assist image 81, which is an image obtained by filling each of the multiple segmented regions 95 with the display color allocated by the display color allocation unit 503. The screen 551 displays the selection assist image 81. Thus, as described above, the operator can easily select a region in the stained image 91 suitable for resolution as the resolution region based on the aggregation of stained cells 96.

[0123] Furthermore, by executing the aforementioned program 572 by a computer, the following steps are performed: dividing the stained image 91 of the specimen 90 obtained by immunostaining into multiple segmented regions 95 (step S11); assigning a display color from a set of multiple display colors containing different colors of varying salience to each of the multiple segmented regions 95, such that the salience increases as the aggregation of stained cells 96 in the segmented region 95 increases (step S12); generating a selected auxiliary image 81 (step S13), wherein the selected auxiliary image 81 is an image obtained by filling each of the multiple segmented regions 95 with the display color assigned in step S12; and displaying the selected auxiliary image 81 on the screen 551 (step S14). Thus, as described above, the operator can easily select a region in the stained image 91 suitable for resolution as the resolution region based on the aggregation of stained cells 96.

[0124] Various modifications can be made to the aforementioned area selection assistance method, area selection assistance device 1, and procedure 572.

[0125] For example, the number of images displayed on screen 551 of display 55 is not limited to the examples above, and various changes can be made. In addition, the image configuration on screen 551 can also be changed in various ways.

[0126] In the area selection auxiliary device 1, when multiple displays 55 are arranged longitudinally and / or laterally as a single display unit, the aforementioned screen 551 is obtained by summing the screens of the multiple displays 55.

[0127] The aforementioned boundary lines 86a, 86b, and 86c can also be displayed overlapping with mask 85a or mask 85b on the selected auxiliary image 81.

[0128] In the selected auxiliary images 81 and 82, contour lines corresponding to the aggregation can be displayed, for example, by assigning a display color whose conspicuousness increases as the aggregation of stained cells 96 in each segmented region 95 increases. In this case, the outer periphery of a set of multiple segmented regions 95 that are assigned the same display color and are arranged adjacently is displayed using that display color. In addition, the outer periphery of a segmented region 95 that is assigned a display color different from all the adjacent segmented regions 95 is displayed using the display color assigned to that segmented region 95. Furthermore, when the above-mentioned contour lines are displayed in the selected auxiliary images 81 and 82, the coating of each segmented region 95 can be omitted. In this case, it is also possible to easily visually identify the degree of aggregation of stained cells 96 at each position in the stained images 91 and 92 in a manner substantially similar to that described above.

[0129] As described above, when multiple images (e.g., selected auxiliary images 81 and 82, and stained images 91 and 92) are displayed on screen 551, the magnification of these multiple images can be changed independently without being linked. Furthermore, the display area of ​​these multiple images on specimen 90 can be changed independently without being linked. Also, the position indicators 553 displayed on each of the multiple images can be moved independently without being linked. Moreover, the display of the position indicators 553 can be omitted.

[0130] The staining image stored in the storage unit 501 of the region selection auxiliary device 1 may be, for example, a multicolor image obtained by using multiple antibodies on a single specimen 90. In this case, for example, a single staining image corresponding to a specific antibody may be obtained by performing color decomposition on the multicolor image, and this single staining image may be designated as the aforementioned staining image 91. Alternatively, the aforementioned multicolor image may be designated as staining image 91, and the aggregation of stained cells 96 corresponding to a specific antibody may be determined from the multicolor image, and a selection auxiliary image 81 may be generated based on this aggregation.

[0131] The structures in the above-described embodiments and their variations can be appropriately combined as long as they do not contradict each other.

[0132] Although the present invention has been described and illustrated in detail, the above description is merely illustrative and not limiting. Therefore, it can be considered that various modifications or methods can be adopted without departing from the scope of the present invention.

[0133] Symbol Explanation

[0134] 1. Area Selection Auxiliary Device

[0135] 81, 82 Select auxiliary images

[0136] Boundary lines of 86a, 86b, and 86c

[0137] 90 specimens

[0138] Images 91 and 92 (stained images)

[0139] 95. Segmentation

[0140] 96 stained cells

[0141] 502 Image Segmentation Unit

[0142] 503 Display Color Distribution Section

[0143] 504 Selected Auxiliary Image Generation Unit

[0144] 551 screens

[0145] 553 Position Indicator

[0146] 572 program

[0147] Steps S11~S16, S21~S25, S31~S33, and S41.

Claims

1. A region selection assistance method for assisting in the selection of a resolvable region in a stained image obtained by immunostaining, characterized in that, The region selection assistance method has the following steps: (a) A process of segmenting a stained image of a specimen obtained by immunostaining into multiple segmented regions; (b) For each of the plurality of segmented regions, a display color is assigned from a display color group containing a plurality of colors with different salience, such that the salience increases as the aggregation of stained cells in the segmented region increases. (c) The step of generating a selected auxiliary image, which is an image obtained by filling each region of the plurality of segmented regions with the display color assigned in step (b); and (d) The process of displaying the selected auxiliary image on the screen.

2. The region selection auxiliary method according to claim 1, characterized in that, In step (b), the aggregation of stained cells in the segmented region is determined based on the staining intensity in the segmented region.

3. The region selection auxiliary method according to claim 2, characterized in that, Calculating the aggregation of stained cells in the segmented region involves the following steps: (e) A process of obtaining the staining intensity of each of the multiple small regions arranged in a matrix constituting the segmented region; and (f) The process of using the maximum value of the staining intensity of the plurality of small regions as the representative value of the segmented region for image reduction.

4. The region selection auxiliary method according to claim 1, characterized in that, In step (b), the aggregation of stained cells in the segmented region is determined based on the number of positive cells in the segmented region.

5. The region selection auxiliary method according to any one of claims 1 to 4, characterized in that, In step (d), the stained image, or other selected auxiliary images generated from other stained images of the specimen obtained by immunostaining using the same method as the selected auxiliary image, are arranged and displayed on the screen in a state of alignment with the selected auxiliary image.

6. The region selection auxiliary method according to claim 5, characterized in that, The display magnification of the selected auxiliary image can be changed in conjunction with the display magnification of the stained image or the other selected auxiliary image.

7. The region selection auxiliary method according to claim 5, characterized in that, The display range of the selected auxiliary image on the specimen can be changed in conjunction with the display range of the stained image or the other selected auxiliary image on the specimen.

8. The region selection auxiliary method according to claim 5, characterized in that, The position indicator displayed on the selected auxiliary image and indicating the position on the image, and the position indicator displayed on the stained image or the other selected auxiliary image and indicating the position on the image, can move in conjunction with each other.

9. The region selection auxiliary method according to claim 5, characterized in that, In step (d), the scale of the selected auxiliary image relative to the specimen is the same as the scale of the stained image or the other selected auxiliary image relative to the specimen.

10. The region selection auxiliary method according to any one of claims 1 to 4, characterized in that, The area selection assistance method also includes the following steps: (g) A step prior to step (b) that divides other stained images of the specimen obtained by immunostaining into multiple segmented regions in the same manner as in step (a). The distribution of display color in step (b) is performed based on the aggregation of stained cells in the stained image and the other stained images in the segmented region.

11. The region selection auxiliary method according to any one of claims 1 to 4, characterized in that, Other stained images of the specimen obtained by immunostaining are thresholded to generate binary images representing positive and negative regions, and in step (d), the selected auxiliary image is masked by the positive or negative regions.

12. The region selection auxiliary method according to any one of claims 1 to 4, characterized in that, Thresholding is performed on other stained images of the specimen obtained by immunostaining to obtain the boundary lines of positive or negative regions, and in step (d), the boundary lines are displayed overlapping on the selected auxiliary images.

13. A region selection aid device, which assists in the selection of a region to be resolved in a stained image obtained by immunostaining, characterized in that, The area selection auxiliary device has: The image segmentation unit divides the stained image of the specimen obtained by immunostaining into multiple segmentation regions; The color distribution unit assigns a color from a color group containing multiple colors with different salience to each of the plurality of segmented regions, such that the salience increases as the aggregation of stained cells in the segmented regions increases. A selected auxiliary image generation unit generates a selected auxiliary image, which is an image obtained by filling each region of the plurality of segmented regions with a display color allocated by the display color allocation unit; and The screen displays the selected auxiliary image.

14. A procedure that assists in the selection of a resolvable region in a stained image obtained by immunostaining, characterized in that, The following steps are performed by executing the program via a computer: (a) A process of segmenting a stained image of a specimen obtained by immunostaining into multiple segmented regions; (b) For each of the plurality of segmented regions, a display color is assigned from a display color group containing a plurality of colors with different salience, such that the salience increases as the aggregation of stained cells in the segmented region increases. (c) The step of generating a selected auxiliary image, which is an image obtained by filling each region of the plurality of segmented regions with the display color assigned in step (b); and (d) The process of displaying the selected auxiliary image on the screen.

Images