Information processing device, information processing method, and program

Abstract

To manage multiple tissue areas included in a tissue image, by segment.SOLUTION: An information processing apparatus includes an acquisition unit, a tissue area recognition unit, and a segment area recognition unit. The acquisition unit acquires a tissue image in which a tissue sample including at least one segment is drawn. The tissue area recognition unit recognizes multiple tissue areas in which the tissue sample is drawn, from the tissue image. The segment area recognition unit recognizes one or more tissue areas corresponding to the one segment, out of the multiple tissue areas, as one-segment areas.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The embodiments disclosed in this specification and the drawings relate to an information processing device, an information processing method, and a program. [Background technology]

[0002] 2. Description of the Related Art In the field of pathology and the like, there is known a technique for setting information such as annotations on tissue images obtained by capturing images of multiple slices cut out from a specimen, based on user operations or image processing results.

[0003] Since the condition of each section of a single specimen is different, it is desirable to associate meta-information such as annotations with each section. However, when a tissue image contains multiple tissue regions, it can be difficult to identify the tissue regions that make up a single section. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2019-095853 A Summary of the Invention [Problem to be solved by the invention]

[0005] One of the problems to be solved by the embodiments disclosed in this specification and the drawings is to manage a plurality of tissue regions included in a tissue image on a slice-by-slice basis. However, the problems to be solved by the embodiments disclosed in this specification and the drawings are not limited to the above problem. Problems corresponding to the effects of each configuration shown in the embodiments described later can also be positioned as other problems. [Means for solving the problem]

[0006] An information processing device according to an embodiment includes an acquisition unit, a tissue region recognition unit, and a slice region recognition unit. The acquisition unit acquires a tissue image in which a tissue specimen including at least one slice is depicted. The tissue region recognition unit recognizes a plurality of tissue regions in which the tissue specimen is depicted from the tissue image. The slice region recognition unit recognizes one or more tissue regions corresponding to one slice among the plurality of tissue regions as one slice region. The slice area recognition unit recognizes that multiple tissue areas are included in the same slice area when the shortest distance between the contours of multiple tissue areas is equal to or less than a threshold value, or when multiple tissue areas that are relatively close to each other are identified based on the average or standard deviation of the shortest distances between all tissue areas included in the tissue image, based on a specified slice arrangement rule that states that when a tissue specimen depicted in a single tissue image contains multiple slices, the distance between multiple tissue areas that make up the same slice is closer than the distance between multiple tissue areas that make up other slices. [Brief description of the drawings]

[0007] [Figure 1] FIG. 1 is a diagram illustrating an example of an overall configuration of an information processing system according to the first embodiment. [Diagram 2] FIG. 2 is a diagram for explaining a flow until a tissue image is captured according to the first embodiment. [Diagram 3] FIG. 3 is a diagram showing an example of a tissue image according to the first embodiment. [Figure 4] FIG. 4 is a diagram showing an example of a state in which a plurality of tissue regions are recognized from the tissue image shown in FIG. [Diagram 5] FIG. 5 is a diagram showing an example of a recognition result of the slice region according to the first embodiment. [Figure 6] FIG. 6 is a diagram showing an example of a tissue image based on the slice arrangement rule (1) according to the first embodiment. [Figure 7] FIG. 7 is a diagram showing an example of a tissue image based on the slice arrangement rule (2) according to the first embodiment. [Figure 8] FIG. 8 is a diagram showing an example of a tissue image based on the slice arrangement rule (3) according to the first embodiment. [Figure 9] FIG. 9 is a diagram showing an example of a method for recognizing a segment region based on the segment arrangement rule (3) according to the first embodiment. [Figure 10] FIG. 10 is a diagram showing an example of a segment arrangement that is not permitted by segment arrangement rule (3) in the first embodiment. [Figure 11] FIG. 11 is a diagram showing an example of the image viewer according to the first embodiment. [Figure 12]FIG. 12 is a diagram showing an example of a method for selecting a slice region using a list box according to the first embodiment. [Figure 13] FIG. 13 is a diagram showing an example in which the selected slice region has been changed from that shown in FIG. [Figure 14] FIG. 14 is a diagram showing an example of annotation settings according to the first embodiment. [Figure 15] FIG. 15 is a diagram showing an example in which the selected slice region has been changed from that shown in FIG. [Figure 16] FIG. 16 is a diagram showing an example of a state in which the selection of a slice region is released in the first embodiment. [Figure 17] FIG. 17 is a flowchart showing an example of the flow of the segment region recognition processing according to the first embodiment. [Figure 18] FIG. 18 is a flowchart showing an example of the flow of the segment region recognition processing according to the first embodiment. [Figure 19] FIG. 19 is a diagram showing an example of a recognition result of the slice region according to the second embodiment. [Figure 20] FIG. 20 is a diagram showing an example of an analysis result of an image feature amount according to the second embodiment. [Figure 21] FIG. 21 is a diagram illustrating an example of evaluation of periodicity of image feature amounts according to the second embodiment. [Figure 22] FIG. 22 is a diagram illustrating an example of input and output of a trained model according to the third embodiment. [Figure 23] FIG. 23 is a diagram illustrating an example of a bounding box according to the third embodiment. [Figure 24] FIG. 24 is a diagram illustrating an example of a mask image according to the third embodiment. [Diagram 25] FIG. 25 is a diagram showing an example of a display of an image viewer according to the first modification. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0008] Hereinafter, embodiments of an information processing device, an information processing method, and a program will be described in detail with reference to the drawings.

[0009] (First embodiment) Fig. 1 is a diagram showing an example of the overall configuration of an information processing system S according to the first embodiment. As shown in Fig. 1, the information processing system S includes an information processing device 100, a specimen image storage device 201, and a tissue image storage device 202. The information processing device 100 is communicatively connected to the specimen image storage device 201 and the tissue image storage device 202 via a network 300 such as an in-hospital LAN (Local Area Network).

[0010] The information processing system S may further include a specimen management system, a hospital information system (HIS), a laboratory information system (LIS), a radiology information system (RIS), etc. Alternatively, the information processing system S may be a part of a hospital information system. The information processing system S may further include a terminal device such as a PC (Personal Computer) or a tablet terminal.

[0011] The information processing system S is installed in, for example, a medical institution such as a hospital, a research institute such as a university, an examination center, etc. Furthermore, some or all of the devices constituting the information processing system S may be installed in a cloud environment.

[0012] The specimen image storage device 201 is a device that stores specimen images. The specimen image is an image of the entire tissue piece, which is a specimen. The specimen image is a digital image captured by an imaging device such as a digital camera. The specimen image is also called a macro image.

[0013] In this embodiment, the specimen is a portion of tissue collected from a patient's body, an animal, etc. For example, the specimen is the mucosa and submucosa of the digestive tract excised by endoscopic submucosal dissection (ESD), but is not limited thereto, and may be a piece of tissue excised by laparotomy or the like.

[0014] The tissue image storage device 202 is a device that stores tissue images. A tissue image is an image of at least one slice cut out from a tissue piece that is a specimen, captured at a higher magnification than a specimen image. For example, a tissue image is a WSI (Whole Slide Imaging) image in which the entire slice or a part of the slice placed on a glass slide is digitalized with high accuracy. A tissue image may be a micrograph captured by a WSS (Whole Slide Scanner) or a digital microscope, or an image captured by other methods. A tissue image is also called a micro image.

[0015] The sample image storage device 201 and the tissue image storage device 202 are, for example, a server device or a PC. The sample image storage device 201 and the tissue image storage device 202 may be collectively referred to as an image storage device. Although the sample image storage device 201 and the tissue image storage device 202 are illustrated as separate devices in Fig. 1, the sample image storage device 201 and the tissue image storage device 202 may be configured as an integrated device. The sample image storage device 201, the tissue image storage device 202, and the information processing device 100 may be configured as an integrated device.

[0016] FIG. 2 is a diagram for explaining a flow until a tissue image according to the first embodiment is captured. For example, a specimen 5 collected from a patient P by a diagnostician is transferred to a pathologist or a laboratory technician for pathological diagnosis. At this time, a pathology number is assigned for each request (order) for pathological diagnosis. Also, a different specimen number is assigned for each specimen to be the subject of pathological diagnosis. Note that, when there are multiple specimens to be diagnosed in one request for pathological diagnosis, multiple specimen numbers are associated with one pathology number. Hereinafter, in this embodiment, a pathologist or a laboratory technician is referred to as a pathologist, etc.

[0017] The processing performed by the information processing device 100 in this embodiment does not necessarily have to be for the purpose of pathological diagnosis. For example, the processing may be for the purpose of research by a research institute or for the purpose of creating a report at an examination center that performs pathological examinations on behalf of a medical institution.

[0018] A specimen image ID is assigned as identification information of the specimen image 51 obtained by capturing an image of the specimen 5 using a digital camera or the like. The specimen image ID may be registered as additional information of the specimen image 51. Information related to the specimen image 51, such as the pathology number of the specimen 5 that is the subject of the image of the specimen image 51, the specimen number of the specimen 5, the order number of the pathological diagnosis related to the specimen 5, the patient ID of the patient P from whom the specimen 5 was obtained, and the image date and time of the specimen image 51, may be registered as additional information of the specimen image 51. The specimen image ID and other information related to the specimen image 51 may be displayed as text information on the specimen image 51.

[0019] A pathologist or the like cuts out slices 60a-60e from specimen 5, and after performing processing such as staining, for example, slices the cross sections of slices 60a-60e and places them on a glass slide 7. Hereinafter, when there is no need to distinguish between individual slices 60a-60e, they will simply be referred to as slices 60.

[0020] In this embodiment, the group of sections placed on the slide glass 7 is referred to as a tissue specimen 6. The tissue specimen 6 includes at least one section 60.

[0021] In addition, in FIG. 2, each section 60 is placed directly on the glass slide 7, but depending on the length of the section 60 or the shape of the cutting position, one section 60 may be divided into multiple pieces and placed on the glass slide 7.

[0022] In FIG. 2, as an example, the slices 60a to 60e are imaged with their cross sections facing the imaging device, but the orientation of the slices is not limited to this. For example, the slice 60 may be imaged in the same orientation as the specimen 5 when the specimen image 51 is imaged. In FIG. 2, the slice 60 is arranged on the slide glass 7 so that the horizontal direction of the tissue image 61 is the longitudinal direction of the tissue region 62, but the arrangement direction is not limited to this. For example, the slice 60 may be arranged on the slide glass 7 so that the vertical direction of the tissue image 61 is the longitudinal direction of the tissue region 62. Alternatively, the slice 60 may be arranged obliquely on the slide glass 7. In this embodiment, the arrangement of the slice 60 is determined by slice arrangement rules described later.

[0023] An image such as a WSI image obtained by capturing an image of the tissue specimen 6 placed on the slide glass 7 is a tissue image 61. In the tissue image 61, the tissue specimen 6 including slices 60a to 60e is depicted.

[0024] A tissue image ID is assigned as identification information of the tissue image 61. The tissue image ID may be registered as additional information of the tissue image 61. Information related to the tissue image 61, such as the pathology number of the specimen 5 from which the slice that is the subject of imaging of the tissue image 61 was obtained, the specimen number of the specimen 5, the order number of the pathological diagnosis related to the specimen 5, the patient ID of the patient P from which the specimen 5 was obtained, and the imaging date and time of the tissue image 61, may be registered as additional information of the tissue image 61. The tissue image ID and other information related to the tissue image 61 may be depicted as text information on the tissue image 61. The additional information of the tissue image 61 may include identification information of a facility such as a medical institution that captured the tissue image 61. Identification information of a facility such as a medical institution that captured the imaging process of the tissue image 61 may be acquired from, for example, a clinical examination system or a specimen management system.

[0025] The tissue image 61 includes a background region 70 depicting the glass slide 7, and a plurality of tissue regions 62a-62e depicting the tissue specimen 6. Hereinafter, the tissue regions 62a-62e will be simply referred to as tissue regions 62 when they are not to be distinguished from one another.

[0026] Returning to FIG. 1, the information processing device 100 is, for example, a server device or a PC, and includes a NW interface 110, a storage circuit 120, an input interface 130, a display 140, and a processing circuit 150.

[0027] The NW interface 110 is connected to the processing circuit 150, and controls the transmission and communication of various data between the information processing device 100 and the sample image storage device 201 or the tissue image storage device 202. The NW interface 110 is realized by a network card, a network adapter, a NIC (Network Interface Controller), or the like.

[0028] The memory circuitry 120 pre-stores various types of information used by the processing circuitry 150. The memory circuitry 120 also stores various programs.

[0029] The input interface 130 is realized by a trackball, a switch button, a mouse, a keyboard, a touch pad that performs an input operation by touching the operation surface, a touch screen in which a display screen and a touch pad are integrated, a non-contact input circuit using an optical sensor, a voice input circuit, etc. The input interface 130 is connected to the processing circuit 150, and converts an input operation received from an operator into an electric signal and outputs it to the processing circuit 150. Note that in this specification, the input interface is not limited to an input interface having physical operation parts such as a mouse and a keyboard. For example, an electric signal processing circuit that receives an electric signal corresponding to an input operation from an external input device provided separately from the device and outputs the electric signal to the processing circuit 150 is also included as an example of the input interface 130.

[0030] The display 140 is a liquid crystal display, an organic electro-luminescence (OEL) display, or the like. The input interface 130 and the display 140 may be integrated. For example, the input interface 130 and the display 140 may be realized by a touch panel. The display 140 is an example of a display unit.

[0031] The processing circuit 150 is a processor that reads out a program from the storage circuit 120 and executes it to realize a function corresponding to each program. The processing circuit 150 of this embodiment includes an acquisition function 151, a tissue area recognition function 152, a slice area recognition function 153, a display image generation function 154, a display control function 155, a reception function 156, an image processing function 157, a meta information setting function 158, and a correction function 159. The acquisition function 151 is an example of an acquisition unit. The tissue area recognition function 152 is an example of a tissue area recognition unit. The slice area recognition function 153 is an example of a slice area recognition unit. The display image generation function 154 is an example of a display image generation unit. The display control function 155 is an example of a display control unit. The reception function 156 is an example of a reception unit. The image processing function 157 is an example of an image processing unit. The meta information setting function 158 is an example of a meta information setting unit. Correction function 159 is an example of a corrector.

[0032] Here, for example, each processing function of the components of the processing circuitry 150, that is, the acquisition function 151, the tissue area recognition function 152, the slice area recognition function 153, the display image generation function 154, the display control function 155, the reception function 156, the image processing function 157, the meta information setting function 158, and the correction function 159, is stored in the storage circuitry 120 in the form of a program executable by a computer. The processing circuitry 150 is a processor. For example, the processing circuitry 150 realizes a function corresponding to each program by reading the program from the storage circuitry 120 and executing it. In other words, the processing circuitry 150 in a state in which each program has been read has each function shown in the processing circuitry 150 of FIG. 1. 1, the processing functions performed by the acquisition function 151, tissue area recognition function 152, slice area recognition function 153, display image generation function 154, display control function 155, reception function 156, image processing function 157, meta information setting function 158, and correction function 159 are realized by a single processor, but a processing circuit 150 may be configured by combining multiple independent processors and each processor may execute a program to realize a function. Also, in FIG. 1, the single storage circuit 120 is described as storing a program corresponding to each processing function, but a configuration may be adopted in which multiple storage circuits are distributed and the processing circuit 150 reads out the corresponding program from each storage circuit.

[0033] In the above description, an example has been described in which the "processor" reads out and executes a program corresponding to each function from a storage circuit, but the embodiment is not limited to this. The term "processor" refers to circuits such as a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), an Application Specific Integrated Circuit (ASIC), a programmable logic device (e.g., a Simple Programmable Logic Device (SPLD), a Complex Programmable Logic Device (CPLD), and a Field Programmable Gate Array (FPGA)). When the processor is, for example, a CPU, the processor realizes a function by reading out and executing a program stored in a storage circuit. On the other hand, when the processor is an ASIC, instead of storing a program in the storage circuit 120, the function is directly incorporated as a logic circuit in the circuit of the processor. Note that each processor in this embodiment is not limited to being configured as a single circuit for each processor, and may be configured as one processor by combining multiple independent circuits to realize the function. Furthermore, multiple components in FIG. 1 may be integrated into one processor to realize the functions thereof.

[0034] The acquiring function 151 acquires the tissue image 61 from the tissue image storage device 202. Also, the acquiring function 151 acquires the sample image 51 from the sample image storage device 201. Note that the source of acquisition of the sample image 51 and the tissue image 61 is not limited to the sample image storage device 201 and the tissue image storage device 202. For example, the acquiring function 151 may acquire the tissue image 61 from an imaging device that captured the tissue image 61.

[0035] The specimen image 51 and tissue image 61 to be acquired are designated, for example, by a user operation accepted by the acceptance function 156. The user of the information processing device 100 is a pathologist or the like.

[0036] The user specifies the specimen image 51 or tissue image 61 to be acquired by inputting, for example, a pathology number, a patient ID, a specimen number, an order number, or an imaging date, or a combination of these. Furthermore, one pathology number corresponds to one order number, one pathology number is associated with one or more specimen numbers and specimen image IDs, and one specimen number is associated with one or more tissue image IDs. Therefore, when a user inputs one pathology number to process a request for a pathological diagnosis, one or more specimen images 51 and one or more tissue images 61 are acquired.

[0037] The user may also select the image to be acquired from a list of the sample images 51 and tissue images 61 stored in the sample image storage device 201.

[0038] The sample image 51 and the tissue image 61 may be stored in advance in the memory circuitry 120 and read out by a user operation.

[0039] The tissue region recognition function 152 recognizes, by image processing, a plurality of tissue regions 62 in which the tissue specimen 6 is depicted from the tissue image 61 .

[0040] Fig. 3 is a diagram showing an example of a tissue image 61 according to the first embodiment. Fig. 4 is a diagram showing an example of a state in which a plurality of tissue regions 62f-62g are recognized from the tissue image 61 shown in Fig. 3. In the tissue image 61, the image region other than the plurality of tissue regions 62f-62g is a background region 70 in which the glass slide 7 is depicted.

[0041] Although the method for recognizing the tissue region 62 from the tissue image 61 is not particularly limited, for example, the tissue region recognition function 152 may recognize the boundary between the background region 70 and the tissue region 62 by executing an image segmentation process on the tissue image 61. Also, for example, the tissue region recognition function 152 may employ rule-based image processing such as selective search that selects candidate regions by grouping similar regions at the pixel level.

[0042] Alternatively, image processing using machine learning such as deep learning may be adopted as a method for recognizing the tissue region 62 from the tissue image 61. When deep learning is adopted, the tissue region recognition function 152 inputs the tissue image 61 to a trained model stored in the storage circuitry 120, and acquires a recognition result of the tissue region 62 output from the trained model. The trained model used in the recognition process of the tissue region 62 is a model in which the correspondence between the multiple tissue images 61 and the tissue regions 62 in the multiple tissue images 61 is learned by a method such as deep learning. The trained model may be incorporated into the tissue region recognition function 152.

[0043] Next, the section area recognition function 153 recognizes one or more tissue areas 62 corresponding to one section 60 from among the multiple tissue areas recognized by the tissue area recognition function 152 as one section area. For example, one section 60 may be divided into multiple pieces and placed on the slide glass 7. In such a case, multiple tissue areas 62 correspond to one section 60. For this reason, the tissue area recognition function 152 groups the multiple tissue areas 62 on a section 60 basis. A group of one or more tissue areas 62 derived from one section 60 is called a section area.

[0044] For example, when one slice 60 is divided into a plurality of tissue regions 62 and depicted, the slice region recognition function 153 recognizes the plurality of tissue regions 62 corresponding to one slice 60 as one slice region.

[0045] A section region is an image region in which one section 60 is depicted in the tissue image 61. When one section 60 is divided into multiple pieces when placed on the slide glass 7, the single section 60 is depicted in the tissue image 61 as multiple tissue regions 62. In this case, the multiple tissue regions 62 become one section region. In other words, a section region is a group of tissue regions 62 derived from one section 60. Note that, since one section 60 may be depicted in the tissue image 61 as one tissue region 62 without being divided, the group of tissue regions 62 includes one or more tissue images 61.

[0046] Fig. 5 is a diagram showing an example of the recognition result of the slice regions 8a, 8b according to the first embodiment. In the example shown in Fig. 5, among tissue regions 62f-62i included in tissue image 61, tissue region 62f is slice region 8a, and tissue regions 62g-62i are slice region 8b. The three tissue regions 62g-62i are image regions in which fragments included in one slice 60 are captured. Furthermore, tissue region 62f is an image region in which a fragment included in a slice 60 different from the other tissue regions 62g-62i is captured. Hereinafter, when there is no need to distinguish between the individual slice regions 8a, 8b, they will simply be referred to as slice regions 8.

[0047] The slice area recognition function 153 recognizes the slice area 8 based on the prescribed slice arrangement rules.

[0048] The prescribed slice arrangement rule is, for example, an operational rule of a facility such as a medical institution or an examination center that captures the tissue image 61. A technician or doctor who captures the tissue image 61 arranges the slice 60 on the glass slide 7 in accordance with the operational rule.

[0049] In this embodiment, the following (1) to (3) are given as examples of the prescribed segment arrangement rules. (1) One tissue image 61 includes only one slice 60. (2) When the tissue specimen 6 depicted in one tissue image 61 includes multiple slices 60, the distance between the multiple tissue regions 62 constituting the same slice is closer than the distance between the multiple tissue regions 62 constituting other slices 60. (3) When the tissue specimen 6 depicted in one tissue image 61 includes multiple slices 60, the multiple tissue regions 62 constituting one slice 60 are arranged along the shape of the slice 60 when the slice 60 is cut out from the specimen 5.

[0050] The slice arrangement rule applied by the information processing device 100 may be determined in advance as being one of (1) to (3), or the user may be able to select one of the slice arrangement rules. For example, a system administrator or the like may set a default slice arrangement rule for a medical institution or the like where the information processing device 100 is used in the information processing device 100, and a general user may be able to change the slice arrangement rule by operating a GUI (Graphical User Interface) as necessary. Alternatively, a configuration may be adopted in which the user selects an applicable slice arrangement rule from a list of slice arrangement rules (1) to (3) each time the tissue image 61 is processed. The selection operation of the slice arrangement rule by the user is received by a reception function 156 described later. The slice area recognition function 153 may recognize the slice area 8 according to the slice arrangement rule selected by the user.

[0051] Alternatively, the information processing device 100 may be configured to be compatible with only one of the segment arrangement rules (1) to (3).

[0052] Alternatively, identification information of the facility where the tissue image 61 was captured may be registered in the supplementary information of the tissue image 61 or other metadata, and the slice area recognition function 153 may apply a slice arrangement rule corresponding to the facility. When this method is adopted, the memory circuitry 120 stores slice arrangement rule specification information in which identification information of the facility where the tissue image 61 is captured is associated with the slice arrangement rule adopted in the facility. Note that the facility where the tissue image 61 was captured may be the same as or different from the facility where the information processing device 100 is installed. The slice area recognition function 153 specifies the slice arrangement rule to be applied to the tissue image 61 based on the slice arrangement rule specification information.

[0053] The following describes a method for recognizing the segment region 8 according to each of the segment arrangement rules (1) to (3).

[0054] When the slice arrangement rule (1) is applied, the slice area recognition function 153 recognizes all tissue areas 62 included in the tissue image 61 as one slice area 8 .

[0055] FIG. 6 is a diagram showing an example of a tissue image 61 based on the slice arrangement rule (1) according to the first embodiment. The tissue image 61 shown in FIG. 6 includes three tissue regions 62g to 62i. According to the slice arrangement rule (1), one tissue image 61 includes only one slice 60. Therefore, the slice area recognition function 153 recognizes all the tissue regions 62 included in the tissue image 61 as one slice area 8. In the example shown in FIG. 6, the slice area recognition function 153 recognizes the three tissue regions 62g to 62i as slice area 8b corresponding to one slice 60.

[0056] 7 is a diagram showing an example of a tissue image 61 based on the slice arrangement rule (2) according to the first embodiment. According to the slice arrangement rule (2), when a tissue specimen 6 depicted in one tissue image 61 includes multiple slices 60, the distance between multiple tissue regions 62 constituting the same slice is closer than the distance between multiple tissue regions 62 constituting other slices 60. Therefore, the slice area recognition function 153 recognizes one or more tissue regions 62 corresponding to one slice 60 as one slice area 8 based on the distance between the multiple tissue regions 62.

[0057] For example, when the shortest distance between the contours of the tissue regions 62 is equal to or less than a threshold, the slice region recognition function 153 recognizes that the tissue regions 62 are included in the same slice region 8. The threshold is stored in advance in, for example, the storage circuitry 120. The shortest distance between the contours of the tissue regions 62 is the distance from the contour of one tissue region 62 to the contour of the other tissue region 62 at the point where the two tissue regions 62 are closest to each other.

[0058] In the example shown in Fig. 7, the shortest distance d1 between tissue region 62g and tissue region 62h and the shortest distance d2 between tissue region 62h and tissue region 62i are less than or equal to the threshold. In this case, the slice region recognition function 153 classifies the tissue regions 62g to 62i into one group. Also, if the shortest distance d6 between tissue region 62g and tissue region 62i is greater than the threshold, the tissue regions 62g and 62i are classified into the same group via tissue region 62h. The slice region recognition function 153 recognizes the tissue regions 62g to 62i included in one group as the tissue regions 62g to 62i that constitute one slice region 8b.

[0059] Also, the shortest distance d3 between tissue region 62f and tissue region 62g, the shortest distance d4 between tissue region 62f and tissue region 62h, and the shortest distance d5 between tissue region 62f and tissue region 62i are all greater than the threshold. In this case, the slice region recognition function 153 recognizes that tissue region 62f is a slice region 8 different from any of tissue regions 62g to 62i. Since the tissue image 61 does not include any tissue region 62 other than tissue regions 62f to 62i, the slice region recognition function 153 recognizes that tissue region 62f is a single slice region 8a by itself.

[0060] The threshold value may not be a fixed value, but may vary according to the relative difference in the shortest distance between the tissue regions 62f-62i included in the tissue image 61. For example, the slice region recognition function 153 may group tissue regions 62 that are relatively close to each other based on the average or standard deviation of the shortest distance between the tissue regions 62f-62i.

[0061] 8 is a diagram showing an example of a tissue image 61 based on the slice arrangement rule (3) according to the first embodiment. According to the slice arrangement rule (3), when a tissue specimen 6 depicted in one tissue image 61 includes multiple slices 60, a plurality of tissue regions 62 constituting one slice 60 are arranged according to the shape of the slice 60 when the slice 60 is cut out from the specimen 5.

[0062] 8, tissue image 61 includes tissue regions 62f-62l. Among these, tissue regions 62g-62i are imaged as multiple tissue regions 62g-62i because one slice 60 is interrupted. Based on slice arrangement rule (3), slice 60 is arranged along the shape of slice 60 when cut out from specimen 5, so tissue regions 62g-i are arranged in a row in the horizontal direction along the original cut-out position of specimen 5.

[0063] When the slice arrangement rule (3) is applied, the slice area recognition function 153 estimates the direction in which the slices 60 are arranged and the range in which each of the slices 60 may be arranged based on the areas of the tissue areas 62 and the positions of the centers of gravity 65a-65g, and recognizes one or more tissue areas 62 included in the estimated range as one slice area 8. Hereinafter, when there is no need to particularly distinguish between the individual centers of gravity 65a-65g, they will simply be referred to as centers of gravity 65.

[0064] The center of gravity 65 of the tissue region 62 is the center of the distribution of pixels included in the tissue region 62. For example, the coordinates of the center of gravity 65 are the average value of the coordinates of the pixels included in the tissue region 62. Note that the method of determining the center of gravity 65 can employ a known image processing method.

[0065] The slice region recognition function 153 obtains the positions of the centers of gravity 65a to 65g based on the pixels of each of the tissue regions 62g to 62l. The positions of the centers of gravity 65a to 65g are coordinates on the tissue image 61.

[0066] Moreover, the slice region recognition function 153 obtains the area of ​​each of the tissue regions 62g to 62l, and estimates the arrangement direction of the slice 60 based on the position of the center of gravity 65 of the tissue region 62 having an area equal to or larger than the reference area calculated based on the areas of the tissue regions 62g to 62l. The tissue region 62 having an area equal to or larger than the reference area calculated based on the areas of the tissue regions 62g to 62l is also referred to as a representative tissue region. For example, the reference area may be determined by using the average value of the area group of the tissue regions. In the example shown in FIG. 8, as an example of using the average value of the area group of the tissue regions as the reference area, the center of gravity 65e of the tissue region 62g and the center of gravity 65g of the tissue region 62i are excluded because the tissue regions 62g and 62i are smaller than the average area of ​​the tissue regions 62g to 62l. The tissue regions 62h, 62j to 62l are equal to or larger than the average area of ​​the tissue regions 62g to 62l, and therefore are the representative tissue regions.

[0067] The slice region recognition function 153 estimates the angle of the direction in which the slices 60 are arranged based on the centers of gravity 65a to 65f of the tissue regions 62h, 62j to 62l. The direction in which the slices 60 are arranged may be obtained by, for example, calculating the ratio of the variance of the X coordinate and the Y coordinate on the tissue image 61 in the coordinate group of the centers of gravity 65a to 65f, or by principal component analysis. The slice region recognition function 153 may perform principal component analysis on the coordinates of the centers of gravity 65a to 65f, and may set the first principal component direction as the direction in which the slices 60 are arranged. In the example shown in FIG. 8, the slice region recognition function 153 estimates the direction A as the direction in which the slices 60 are arranged. In addition, the slice region recognition function 153 estimates the direction perpendicular to the direction A in which the slices 60 are arranged as the longitudinal direction of the slices 60. When the supplementary information of the tissue image 61 includes information on the arrangement of the slices 60, the slice area recognition function 153 may estimate the direction A in which the slices 60 are arranged based on the information.

[0068] The slice area recognition function 153 estimates the range in which each of the multiple slices 60 may be located based on the centers of gravity 65a to 65f of the tissue areas 62h, 62j to 62l, which are representative tissue areas, and the angle of inclination of the direction A in which the slices 60 are arranged.

[0069] FIG. 9 is a diagram showing an example of a recognition method of the slice region 8b based on the slice arrangement rule (3) according to the first embodiment. As shown in FIG. 9, when the direction A is set as the height direction of the tissue region 62h, the length of the tissue region 62h in the direction A at the center of gravity 65f is set as the height h1 of the tissue region 62h. The slice region recognition function 153 estimates a range including the tissue image 61 derived from the same slice 60 as the tissue region 62h, based on the height h1. For example, the slice region recognition function 153 searches for the tissue region 62 at least partly included between two straight lines L1 and L2 that intersect perpendicularly with the direction A. The distance between the two straight lines L1 and L2 is equal to the height h1. In the example shown in FIG. 9, the tissue regions 62g and 62i are included between the two straight lines L1 and L2, so the slice region recognition function 153 recognizes that the tissue regions 62g to 62i are included in one slice region 8b.

[0070] 9, the distance between the two straight lines L1 and L2 is equal to the height h1, but is not limited to this. For example, the segment area recognition function 153 may set the distance between the two straight lines L1 and L2 to "n% of the height h1". The value of n may be determined in advance and stored in the storage circuitry 120.

[0071] Here, a supplementary explanation will be given regarding the slice arrangement rule (3). Fig. 10 is a diagram showing an example of an arrangement of slices 60 that is not permitted by the slice arrangement rule (3) in the first embodiment. When the slice arrangement rule (3) is applied, the shape of each slice 60 is maintained while it is arranged on the slide glass 7, so that one slice 60x is not cut into a plurality of pieces 60x1, 60x2 and arranged in the forward direction as shown in Fig. 10. Therefore, when the slice arrangement rule (3) is applied, it is possible to estimate the arrangement direction of the slice 60 based on the direction in which the centers of gravity 65 are arranged.

[0072] The section area recognition function 153 associates an identifier with each section area 8 generated by grouping the tissue areas 62 included in the section area 8, and stores the identifier in the storage circuitry 120. The identifier is, for example, a numeric number. The identifier may also be assigned in association with the order of the cut-out positions where the sections 60 are cut out from the specimen 5. The identifier may also be set by a meta information setting function 158 described below.

[0073] Returning to FIG. 1, the display image generating function 154 generates a display image based on the slice area 8 recognized by the slice area recognizing function 153. The display image is an image for highlighting the slice area 8, and is, for example, a mask image that covers a display slice area, which is an extension of one slice area 8, transparently and covers the other image areas with semi-transparent black. The display slice area is an image area that follows the contour of the slice area 8 and is slightly larger than the slice area 8. The reason that the display slice area is larger than the slice area 8 is to make it easier to visually recognize the boundary of the slice area 8 to be displayed when the tissue image 61 is covered with the mask image.

[0074] The display image generating function 154 generates a mask image for each individual slice region 8. Note that the display image is not limited to the mask image, and may be an image that depicts the outline of the display slice region with a dotted line, or the like.

[0075] The display control function 155 causes the tissue image 61 to be displayed on the display 140 .

[0076] 11 is a diagram showing an example of the image viewer 14 according to the first embodiment. The image viewer 14 is a GUI that allows the user to perform operations such as viewing, editing, and processing of the tissue image 61 and the specimen image 51. In the example shown in FIG. 11, the image viewer 14 displayed on the display 140 includes a micro image display area 141, a macro image display area 142, an operation tool display area 144, and an image list display area 143.

[0077] A tissue image 61 is displayed in the micro image display area 141. A specimen image 51 is displayed in the macro image display area 142. A variety of tools that can be operated by the user are displayed in the operation tool display area 144. A reduced version of the tissue image 61 and specimen image 51 that can be displayed in the micro image display area 141 or the macro image display area 142 are displayed in the image list display area 143.

[0078] In addition, in FIG. 11, for the sake of explanation, dashed lines indicating the slice regions 8n to 8r are illustrated, but in reality, the dashed lines do not have to be displayed on the image viewer .

[0079] Furthermore, the display control function 155 displays a tissue region 62 constituting one slice region 8, out of the multiple tissue regions 62n-62s included in the tissue image 61, in a manner different from the tissue regions 62 constituting the other slice regions 8. For example, when the user selects one of the multiple slice regions 8n-8r included in the tissue image 61 displayed in the micro image display area 141, the display control function 155 highlights the tissue region 62 included in the selected slice region 8.

[0080] When the user operates the micro image display area 141 of the display 140 by a later-described reception function 156, such as clicking a mouse, the display control function 155 determines that the slice region 8 corresponding to the clicked coordinates has been selected. In this case, the display control function 155 displays the tissue region 62 included in the slice region 8 selected by the user's operation in a manner different from the tissue regions 62 included in the other slice regions 8. For example, the display control function 155 displays a mask image generated by the display image generation function 154, in which only the display slice region including the selected slice region 8 is transparent and the other image regions are masked with semi-transparent black, on the tissue image 61 and on the display 140. With this display, only the selected slice region 8 is highlighted. Note that the manner of highlighting is not limited to this.

[0081] Furthermore, the means for selecting the slice region 8 by the user is not limited to clicking on the micro image display area 141. For example, Fig. 12 is a diagram showing an example of a selection method of the slice region 8 using the list box 91 according to the first embodiment.

[0082] 12, the user selects the identifier of the desired section region 8 from the drop-down menu of the list box 91. In Fig. 12, the display control function 155 displays a mask image 81p, in which the image region excluding the tissue region 62q and its surroundings is masked, superimposed on the tissue image 61 in order to highlight the tissue region 62q included in the section region 8 corresponding to the selected identifier. This causes the tissue region 62q included in the selected section region 8 to be highlighted.

[0083] The method of highlighting the tissue region 62q included in the selected section region 8 is not limited to highlighting. For example, the display control function 155 may surround the tissue region 62q included in the selected section region 8 with a frame such as a dashed line or a dotted line. The display control function 155 may also hide the tissue regions 62n-62p, 62r, and 62s included in the unselected section region 8 and display only the tissue region 62q included in the selected section region 8. The display control function 155 may also display the tissue regions 62n-62p, 62r, and 62s included in the unselected section region 8 in a lighter color than the tissue region 62q included in the selected section region 8.

[0084] The method of selecting the slice region 8 is not limited to a user's clicking operation with a mouse or specification using the list box 91. For example, a slice region 8 in which an abnormality such as a lesion is detected by an image processing function 157 described later may be selected.

[0085] 12, the identifier of each section region 8 is associated with the number of the cut-out position where the section 60 was cut out from the specimen 5. Since the sample image 51 displayed in the macro image display area 142 displays the number of the cut-out position, the user can select the identifier of the section region 8 by referring to the number displayed on the sample image 51. The display control function 155 may display the identifier of the section region 8 on the tissue image 61.

[0086] Furthermore, the display control function 155 may change the display mode of the cut-out position on the sample image 51 that corresponds to the selected slice region 8. For example, the display control function 155 may highlight the cut-out position on the sample image 51 that corresponds to the selected slice region 8.

[0087] Moreover, Fig. 13 is a diagram showing an example in which the selected slice region 8 has changed from that in Fig. 12. In Fig. 13, the selected identifier has changed from "3" to "4." When the selection of the slice region 8 has been changed, the display control function 155 cancels the highlighting of the slice region 8 before the change and highlights the newly selected slice region 8.

[0088] Furthermore, the display control function 155 may display, on the tissue image 61, various types of metadata set by a meta-information setting function 158 described later.

[0089] Returning to Fig. 1, the reception function 156 receives various operations by the user via the input interface 130. For example, when a plurality of section regions 8 are included in the tissue image 61, the reception function 156 receives a user operation to select one of the section regions 8. As described in Figs. 11 to 13, the selection method is a click operation using a mouse, a selection operation using a list box 91, or the like.

[0090] Furthermore, the reception function 156 receives input of information such as annotations by the user. Annotations are provided to the tissue image 61 in the form of diagrams or text. Since the state of lesions and the like differs for each section 60 even in a single specimen 5, it is preferable that information such as annotations be associated with each section region 8 unit rather than with each tissue image 61 unit. When the reception function 156 receives input of information such as annotations by the user, it sends the received content to the meta information setting function 158.

[0091] Furthermore, the reception function 156 receives an operation by the user to modify the slice region 8. When the reception function 156 receives an operation by the user to modify the slice region 8, the reception function 156 sends the received content to the modification function 159.

[0092] The image processing function 157 performs an analysis by image processing for each slice region 8. Specifically, the image processing function 157 analyzes the degree of abnormality, the presence or absence of abnormality, the range of the abnormal region, and the like for each slice region 8. An abnormality is, for example, a lesion depicted in the tissue region 62 included in the slice region 8. The site where the lesion is depicted is, for example, a tumor region such as an adenoma or adenocarcinoma. The method of image processing for analyzing the abnormality is not particularly limited, and a known method can be adopted. In addition, the image processing function 157 may analyze the degree of abnormality, the presence or absence of abnormality, the abnormal region, and the like using a trained model such as deep learning.

[0093] The image processing function 157 may also analyze the area, major axis, minor axis, etc. of each slice region 8. The image processing function 157 may also analyze the average, mode, median pixel value, etc. of the pixels of each slice region 8.

[0094] The function of the image processing function 157 may be executed by another information processing device connected to the information processing device 100. In this case, the acquisition function 151 acquires the analysis result from the other information processing device.

[0095] When a plurality of slice regions 8 are included in the tissue image 61, the meta-information setting function 158 sets meta-information for each of the slice regions 8.

[0096] In this embodiment, the meta-information is information related to the section region 8, and includes, for example, an identifier capable of identifying the position where the section 60 corresponding to the section region 8 was cut out from the specimen 5. As shown in Figures 12 and 13, the identifier associates the section region 8 in the tissue image 61 with the cut-out position in the specimen image 51. The meta-information setting function 158 associates, as an identifier, a number corresponding to the number of each cut-out position with each section region 8, based on, for example, the order of the cut-out positions included in the specimen image 51 and the order of the section regions 8 in the tissue image 61.

[0097] The meta information may also include an analysis result of image processing by the image processing function 157 on the section region 8 to which the meta information is to be added. The meta information may also include a doctor's diagnosis result on the section region 8 to which the meta information is to be added. The doctor's diagnosis result includes the presence or absence of abnormality, the degree of abnormality, an abnormal region, a candidate diagnosis name, the basis for diagnosis, and other comments. The doctor's diagnosis result on the section region 8 may be acquired by the acquisition function 151 acquiring a report or the like registered in an external information processing device, or the reception function 156 accepting input of information on the diagnosis by the user. Such information is also called annotation. The user's operation of inputting such information may also be called annotation.

[0098] FIG. 14 is a diagram showing an example of annotation setting according to the first embodiment. In the example shown in FIG. 14, the slice region 8 including the tissue region 62r is selected as a display target. Also, in FIG. 14, it is assumed that the reception function 156 receives that the user has drawn region designation images 94a and 94b indicating the region of interest on the tissue image 61 using the drawing tool 92. In this case, the meta information setting function 158 sets the region of interest display images 93a and 93b at positions corresponding to the region designation images 94a and 94b in the tissue region 62r included in the selected slice region 8. Note that the meta information setting function 158 may set the region of interest display images 93a and 93b outside the contour of the tissue region 62r so that the region of interest display images 93a and 93b do not cover the tissue region 62r. ​​The display control function 155 displays the region of interest display images 93a and 93b based on the setting by the meta information setting function 158. The region of interest display images 93a and 93b are assumed to be included in the meta information.

[0099] FIG. 15 is a diagram showing an example in which the selected slice region has changed from that shown in FIG. 14. In FIG. 15, slice region 8 including tissue region 62q is selected as the display target. In this case, region designation images 94a and 94b input in FIG. 14 for setting annotations for slice region 8 including tissue region 62r are not displayed. Then, the reception function 156 receives a user's operation to set a region designation image 94c for slice region 8 including currently selected tissue region 62q. Furthermore, the meta information setting function 158 sets a region of interest display image 93c at a position corresponding to the region designation image 94c in tissue region 62q included in the selected slice region 8.

[0100] Furthermore, the meta information setting function 158 may associate information such as an analysis result or a diagnosis result associated with the slice region 8 as meta information with a corresponding cut-out position on the sample image 51. For example, the display control function 155 may display information such as an analysis result or a diagnosis result associated with the slice region 8 on the sample image 51 in association with the cut-out position corresponding to the slice region 8.

[0101] 16 is a diagram showing an example of a state in which the selection of the slice region 8 in the first embodiment is cancelled. In the state shown in FIG. 15, when the reception function 156 receives that the user has clicked the background region 70 with the mouse, the selection of the slice region 8 is cancelled. In this case, the image processing function 157 cancels the display of the mask image. Note that the image processing function 157 continues to display the region of interest display images 93a to 93c even when the selection is cancelled.

[0102] 14-16 show examples of linear annotations, but the form of the displayed meta information is not limited to this. For example, text data or an image may be displayed on the image viewer 14. The display control function 155 may display various types of meta information related to the selected slice region 8, for example, in the background region 70 near the slice region 8 selected by the user. Also, a specific area in which meta information is displayed may be provided on the image viewer 14.

[0103] 1 , the correction function 159 corrects the slice area 8 recognized by the slice area recognition function 153 based on the user's operation accepted by the acceptance function 156. For example, when the user selects one or more tissue areas 62 on the image viewer 14, the correction function 159 may store the selected tissue areas 62 in the memory circuitry 120 as one slice area 8.

[0104] Next, a flow of the segment region recognition process executed by the information processing device 100 configured as above will be described.

[0105] FIG. 17 is a flowchart showing an example of the flow of the segment region recognition processing according to the first embodiment.

[0106] First, the acquisition function 151 acquires the sample image 51 from the sample image storage device 201 (S1).

[0107] Then, the tissue region recognition function 152 recognizes, by image processing, a plurality of tissue regions 62 in which the tissue specimen 6 is depicted from the tissue image 61 (S2).

[0108] Next, the slice area recognition function 153 groups the multiple tissue areas 62 according to a prescribed slice arrangement rule (S3). Note that the prescribed slice arrangement rule may be selected by the user when executing this flowchart, or may be determined in advance.

[0109] Then, the slice area recognition function 153 recognizes the tissue areas 62 in the same group as one slice area 8 (S4). The slice area recognition function 153 causes the memory circuitry 120 to store the association between the slice area 8 and the tissue area 62.

[0110] In addition, the display image generating function 154 generates a mask image 81 for highlighting each slice region 8 recognized by the slice region recognizing function 153 (S5). The display image generating function 154 stores the generated mask image 81 in the memory circuitry 120.

[0111] In addition, the meta information setting function 158 associates a number corresponding to each cut-out position with each section area 8 as an identifier based on the order of the cut-out positions included in the sample image 51 and the order of the section areas 8 in the tissue image 61 (S6).

[0112] Furthermore, the image processing function 157 performs image analysis for each slice region 8 to determine the degree of abnormality, the presence or absence of abnormality, the range, area, major axis, minor axis, average pixel value, most frequent pixel value, median pixel value, and the like of the abnormal region (S7). The meta information setting function 158 associates the analysis result by the image processing function 157 with the slice region 8 as meta information and stores it in the storage circuitry 120. At this point, the processing of this flowchart ends.

[0113] Next, a process flow relating to the display of the tissue image 61 in which the slice region 8 has been recognized will be described.

[0114] FIG. 18 is a flowchart showing an example of the flow of the segment region recognition processing according to the first embodiment.

[0115] When the reception function 156 receives the selection of a tissue image to be displayed by the user (S21 "Yes"), the display control function 155 causes the image viewer 14 to display the selected tissue image 61 (S22). When the reception function 156 does not receive the selection of a tissue image to be displayed (S21 "No"), the process returns to S21.

[0116] Furthermore, if the reception function 156 receives a selection of a slice region 8 by the user (S23 "Yes"), the display control function 155 identifies the selected slice region 8 (S24) and highlights the selected slice region 8 (S25). However, if the reception function 156 does not receive a selection of a slice region 8 (S23 "No"), the process returns to S23.

[0117] Furthermore, if meta information has already been set for the selected slice area 8, the display control function 155 displays the meta information in association with the selected slice area 8 (S26). Note that the user may be able to select whether to display or not display the meta information.

[0118] Furthermore, if the reception function 156 receives an operation to register or change meta information such as an annotation indicating a region of interest (S27 "Yes"), the meta information setting function 158 registers or changes the meta information in association with the selected slice region 8 (S28). The display control function 155 causes the image viewer 14 to display the newly registered or changed meta information. Note that if the reception function 156 does not receive an operation to register or change meta information (S27 "No"), the process returns to S27. Here, the process of this flowchart ends.

[0119] In this manner, the information processing device 100 of this embodiment recognizes a plurality of tissue regions 62 in which the tissue specimen 6 is depicted from the tissue image 61, and recognizes one or more tissue regions 62 corresponding to one slice among the recognized plurality of tissue regions 62 as one slice region 8. Therefore, according to the information processing device 100 of this embodiment, the plurality of tissue regions 62 included in the tissue image 61 can be managed in units of slices 60.

[0120] For example, since the state of each section 60 of a single specimen 5 is different, it is desirable to associate meta-information such as annotations with each section 60 individually. However, as a general comparative example, when a tissue image includes multiple tissue regions, it may be difficult to identify a group of tissue regions that constitute one section. In such an example, it may be difficult for the user to accurately specify a target region for setting meta-information.

[0121] In contrast, according to the information processing device 100 of this embodiment, multiple tissue regions 62 contained in a tissue image 61 are grouped by slice region 8 corresponding to one slice 60, making it possible to perform processing such as setting meta information for each individual slice 60.

[0122] Furthermore, the information processing device 100 of this embodiment recognizes the section region 8 based on the prescribed section arrangement rules (1) to (3). Therefore, according to the information processing device 100 of this embodiment, it is possible to recognize the section region 8 with high accuracy in accordance with the rules of the facility where the tissue image 61 was captured.

[0123] Second embodiment In the above-described first embodiment, the slice area recognition function 153 recognizes the slice area 8 based on a prescribed slice arrangement rule. In this second embodiment, the slice area recognition function 153 recognizes the slice area 8 based on the periodicity of the image feature amount of the tissue area 62.

[0124] Fig. 19 is a diagram showing an example of a recognition result of tissue regions 62t-62y according to the second embodiment. In the example shown in Fig. 19, the section region recognition function 153 assigns a number to each of the tissue regions 62t-62y included in the tissue image 61. The number is called a tissue region number.

[0125] Then, the slice region recognition function 153 analyzes the image feature amounts of the tissue regions 62t to 62y by image processing.

[0126] 20 is a diagram showing an example of an analysis result of image feature amounts according to the second embodiment. In this embodiment, the image feature amounts are the area of ​​tissue regions 62t to 62y, the average green color component, the green color standard deviation value, the position in the leftmost tissue image 61, and the position in the rightmost tissue image 61, etc. Note that indices representing the image feature amounts are not limited to these.

[0127] In the table of FIG. 20, the values ​​entered in the "region area" column are the ratios of the individual tissue regions 62t to 62y to the average region area of ​​the tissue regions 62t to 62y.

[0128] The slice region recognition function 153 evaluates the similarity of the tissue regions 62t to 62y and the periodicity of the image feature amounts of the tissue regions 62t to 62y based on the image feature amounts of the tissue regions 62t to 62y shown in FIG.

[0129] Fig. 21 is a diagram showing an example of evaluation of periodicity of image features according to the second embodiment. In Fig. 21, the periodicity of region areas is evaluated. For example, the slice region recognition function 153 classifies the region areas according to predetermined thresholds. In the example shown in Fig. 21, the thresholds are 25%, 50%, 150%, and 200%.

[0130] The slice area recognition function 153 judges whether there is periodicity in the evaluation of the area of ​​the regions classified by the threshold value, and the boundary of the period. In Fig. 21, there is periodicity in the evaluation of the area of ​​the regions for numbers 1 to 3 and numbers 4 to 6. Therefore, the slice area recognition function 153 recognizes the tissue areas 62t to 62v corresponding to numbers 1 to 3 as one group, and the tissue areas 62w to 62y corresponding to numbers 4 to 6 as another group. Then, as shown in Fig. 19, the slice area recognition function 153 recognizes the tissue areas 62t to 62v belonging to one group as slice area 8c, and the tissue areas 62w to 62y belonging to the other group as slice area 8d.

[0131] According to the method of this embodiment, even if the slice arrangement rule is not clearly defined or the slice arrangement rule at the time of imaging is unknown, it is possible to recognize the slice region 8.

[0132] Furthermore, an image feature detector that uses a trained model generated by a neural network may be used to evaluate the similarity and periodicity of the tissue regions 62t to 62y.

[0133] (Third embodiment) In the third embodiment, the slice area recognition function 153 recognizes the slice area 8 by using a trained model.

[0134] 22 is a diagram showing an example of input / output of the trained model 90 according to the third embodiment. The slice area recognition function 153 inputs a tissue image 61 to the trained model 90, thereby grouping a plurality of tissue areas 62 in units of slice areas 8. Specifically, the slice area recognition function 153 inputs the tissue image 61 to the trained model 90, thereby obtaining a bounding box 82 and a mask image 81 for each slice area 8 output from the trained model 90. A known instance segmentation technique can be applied to generate the trained model 90.

[0135] The trained model 90 is, for example, a trained model generated by deep learning such as a neural network or other machine learning. As a deep learning method, a deep convolutional neural network (DCNN), a convolutional neural network (CNN), a recurrent neural network (RNN), etc. can be applied, but are not limited to these. The trained model 90 is, for example, composed of a neural network and trained parameter data.

[0136] The trained model 90 is assumed to be stored in, for example, the memory circuitry 120. The slice area recognition function 153 reads out the trained model 90 from the memory circuitry 120 and inputs the tissue image 61. Alternatively, the trained model 90 may be incorporated in the slice area recognition function 153. In addition, the information processing device 100 may be provided with a learning function for generating the trained model 90, or may use a trained model 90 generated by another information processing device.

[0137] 23 is a diagram showing an example of a bounding box 82 according to the third embodiment. The bounding boxes 82a, 82b are boxes that group one or more tissue regions 62 in units of slice regions 8. Within one bounding box 82a, 82b, slice regions 8f, 8g are detected as one instance. Furthermore, when a part of a tissue region 62 belonging to another slice region 8 enters one bounding box 82a, 82b, the part of the tissue region 62 is pre-trained in the trained model 90 so as not to be detected.

[0138] FIG. 24 is a diagram showing an example of a mask image 81f according to the third embodiment. The trained model 90 outputs a mask image 81 for each slice region 8. The example shown in FIG. 24 is a mask image 81f for highlighting the slice region 8f. In the mask image 81f, a display slice region 83f that is an extension of the slice region 8f is transparent, and the other image regions are covered with semi-transparent black. In this embodiment, since the mask image 81 is generated by the trained model 90, the information processing device 100 does not need to include a display image generation function 154.

[0139] According to the method of this embodiment, even if the slice arrangement rule is not clearly defined or the slice arrangement rule at the time of imaging is unknown, it is possible to recognize the slice region 8.

[0140] (Variation 1) In the above-described first embodiment, an example in which an annotation is set in the slice region 8 has been described, but the example in which meta information is displayed in association with the slice region 8 is not limited to this.

[0141] Fig. 25 is a diagram showing an example of the display of the image viewer 14 according to Modification 1. In the example shown in Fig. 25, the display control function 155 displays each lesion site 95 of the tissue region 62 included in each slice region 8 in a manner different from other sites based on the lesion detection result for each slice region 8. The displayed lesion site 95 may be the analysis result by the image processing function 157 or the diagnosis result by a doctor.

[0142] (Variation 2) In the above-described embodiments, the information processing device 100 is a server device or a PC, but is not limited thereto. For example, the information processing device 100 may be a computer built into an imaging device that captures the tissue image 61. The imaging device is, for example, a WSS, a digital camera, or a digital microscope, but is not limited thereto.

[0143] The various data handled in this specification are typically digital data.

[0144] According to at least one of the embodiments described above, a plurality of tissue regions included in a tissue image can be managed on a slice-by-slice basis.

[0145] Although some embodiments have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These embodiments can be implemented in various other forms, and various omissions, substitutions, modifications, and combinations of embodiments can be made without departing from the spirit of the invention. These embodiments and their modifications are included in the scope of the invention and its equivalents as described in the claims, as well as in the scope and spirit of the invention. [Explanation of symbols]

[0146] 5. Samples 6 Tissue specimen 7. Slides 8,8a~8g,8n section area 14 Image Viewer 51 Sample images 60,60a~60e,60x section 61 Tissue Images 62,62a~62y tissue area 65,65a~65g Center of gravity 70 Background area 81,81f,81p Mask images 82,82a,82b Bounding Box 83f Display section area 90 trained models 91 List Box 93a~93c Region of interest display images 94a~94c Area-specified images 95 Lesion site 100 Information processing device 120 Memory circuit 140 Display 144 Operation tool display area 150 Processing circuit 151 Acquisition Function 152 Organizational area recognition function 153 Section area recognition function 154 Display image generation function 155 Display control function 156 Reception Function 157 Image Processing Functions 158 Meta information setting function 159 Fixing Features S Information Processing System

Claims

1. an acquisition unit that acquires a tissue image depicting a tissue specimen including at least one slice; a tissue region recognition unit that recognizes a plurality of tissue regions in which the tissue specimen is depicted from the tissue image; a slice region recognition unit that recognizes one or more tissue regions corresponding to one slice among the plurality of tissue regions as one slice region; Equipped with the section region recognition unit recognizes that, when the tissue specimen depicted in one tissue image includes multiple sections, the distance between multiple tissue regions constituting the same section is closer than the distance between multiple tissue regions constituting other sections, and based on the specified section arrangement rule, if the shortest distance between the contours of the multiple tissue regions is equal to or smaller than a threshold, or if multiple tissue regions that are relatively close to each other are identified based on the average or standard deviation of the shortest distances between all tissue regions included in the tissue image, the section region recognition unit recognizes that the multiple tissue regions are included in the same section region; Information processing device.

2. the tissue image includes a plurality of the tissue regions; When one slice is divided into a plurality of tissue regions and depicted, the slice region recognition unit recognizes the plurality of tissue regions corresponding to the one slice as one slice region. The information processing device according to claim 1 .

3. An acquisition unit for acquiring a tissue image depicting a tissue specimen including at least one slice; a tissue region recognition unit that recognizes a plurality of tissue regions in which the tissue specimen is depicted from the tissue image; a slice region recognition unit that recognizes one or more tissue regions corresponding to one slice among the plurality of tissue regions as one slice region; Equipped with the segment area recognition unit recognizes the segment area based on a prescribed segment arrangement rule; the specified slice arrangement rule is a rule that, when the tissue specimen depicted in one of the tissue images includes a plurality of slices, a plurality of tissue regions constituting one slice are arranged according to a shape of the slice when the slice is cut out from a specimen; The slice region recognition unit estimates a direction in which the slices are arranged and a range in which each of the slices may be arranged based on areas and center positions of the tissue regions, and recognizes one or more tissue regions included in the estimated range as one of the slice regions. Information processing device.

4. An acquisition unit for acquiring a tissue image depicting a tissue specimen including at least one slice; a tissue region recognition unit that recognizes a plurality of tissue regions in which the tissue specimen is depicted from the tissue image; a slice region recognition unit that recognizes one or more tissue regions corresponding to one slice among the plurality of tissue regions as one slice region; Equipped with the slice region recognition unit groups the tissue regions based on periodicity of image feature amounts of the tissue regions, and recognizes one or more tissue regions included in one group as one slice region; Information processing device.

5. An acquisition unit for acquiring a tissue image depicting a tissue specimen including at least one slice; a tissue region recognition unit that recognizes a plurality of tissue regions in which the tissue specimen is depicted from the tissue image; a slice region recognition unit that recognizes one or more tissue regions corresponding to one slice among the plurality of tissue regions as one slice region; Equipped with The slice area recognition unit inputs the tissue image to a trained model to group the plurality of tissue areas in units of slice areas. Information processing device.

6. The slice area recognition unit inputs the tissue image to the trained model and obtains a recognition result of the tissue area output from the trained model. The information processing device according to claim 5 .

7. The trained model is a model in which a correspondence between a plurality of the tissue images and the tissue regions in the plurality of the tissue images is trained by a deep learning technique. The information processing device according to claim 5 .

8. The trained model is an image feature detector generated by a neural network that outputs an assessment of the similarity of the tissue regions. The information processing device according to claim 5 .

9. The trained model is an image feature detector generated by a neural network that outputs an assessment of the periodicity of the tissue region. The information processing device according to claim 5 .

10. The slice region recognition unit inputs the tissue image into the trained model to obtain a bounding box output from the trained model and a mask image for each slice region, The mask image is an image in which a display slice region, which is an extension of the slice region, of the tissue image is covered with transparency, and an image region other than the display slice region is covered with semi-transparent black. The information processing device according to claim 5 .

11. A display control unit that displays the tissue image on a display unit. The information processing device according to claim 1 .

12. the display control unit displays a tissue region constituting one slice region among the plurality of tissue regions included in the tissue image in a manner different from tissue regions constituting other slice regions; The information processing device according to claim 11.

13. a reception unit that receives a user's operation to select one of the slice regions when the tissue image includes a plurality of slice regions; the display control unit displays a tissue region included in the slice region selected by the user's operation in a manner different from tissue regions included in other slice regions; The information processing device according to claim 12.

14. a meta-information setting unit for setting meta-information for each slice region when the tissue image includes a plurality of slice regions; The information processing device according to claim 13.

15. the meta-information setting unit sets meta-information in a slice region selected by a user from among a plurality of slice regions included in the tissue image. The information processing device according to claim 14.

16. The meta-information includes an analysis result of image processing on a slice region to which the meta-information is to be added.

16. The information processing device according to claim 14 or 15.

17. The meta-information includes a candidate diagnosis for the section region to which the meta-information is to be assigned. The information processing device according to any one of claims 14 to 16.

18. The meta information includes information regarding a diagnosis entered by a user. The information processing device according to any one of claims 14 to 17.

19. a correction unit that corrects the slice area recognized by the slice area recognition unit based on a user operation. The information processing device according to claim 1 .

20. An acquisition step in which an information processing device acquires a tissue image depicting a tissue specimen including at least one slice; a tissue region recognition step in which the information processing device recognizes a plurality of tissue regions in which the tissue specimen is depicted from the tissue image; a slice region recognition step in which the information processing device recognizes one or more tissue regions corresponding to one slice among the plurality of tissue regions as one slice region; Including, In the section area recognition step, the information processing device recognizes that when the tissue specimen depicted in one tissue image includes multiple sections, the distance between multiple tissue areas constituting the same section is closer than the distance between multiple tissue areas constituting other sections, and if the shortest distance between the contours of the multiple tissue areas is equal to or smaller than a threshold, or if multiple tissue areas that are relatively close to each other are identified based on the average or standard deviation of the shortest distances between all tissue areas included in the tissue image, the information processing device recognizes that the multiple tissue areas are included in the same section area. Information processing methods.

21. acquiring a tissue image depicting the tissue specimen including at least one section; a tissue region recognition step of recognizing a plurality of tissue regions in which the tissue specimen is depicted from the tissue image; a slice region recognition step of recognizing one or more tissue regions corresponding to one slice among the plurality of tissue regions as one slice region; on the computer, In the section area recognition step, when the tissue specimen depicted in one tissue image includes multiple sections, the distance between multiple tissue areas constituting the same section is closer than the distance between multiple tissue areas constituting other sections based on a specified section arrangement rule. If the shortest distance between the contours of the multiple tissue areas is equal to or smaller than a threshold, or if multiple tissue areas that are relatively close to each other are identified based on the average or standard deviation of the shortest distances between all tissue areas included in the tissue image, the multiple tissue areas are recognized as being included in the same section area. program.

Images