Information processing device, information processing method, and program

Abstract

To properly determine a position in which a tissue slice is clipped from a specimen.SOLUTION: An information processing apparatus includes a specimen image acquisition unit, a clipping policy acquisition unit, an interest region recognition unit, and a clipping position determination unit. The specimen image acquisition unit acquires a specimen image obtained by imaging a pathological specimen. The clipping policy acquisition unit acquires a clipping policy that defines a rule for clipping a tissue slice from the pathological specimen. The interest region recognition unit recognizes a first interest region from the specimen image. The clipping position determination unit determines a clipping position of the tissue slice in the specimen image on the basis of the first interest region and the clipping policy.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The embodiments disclosed in the present specification and drawings relate to an information processing device, an information processing method, and a program. [Background technology]

[0002] Conventionally, in the field of pathology, etc., specimen images in which the entire tissue fragment serving as the specimen is captured, and tissue images in which a tissue slice cut from the specimen is captured at a higher magnification than the specimen image, have been used by doctors and others for pathological tissue diagnosis, etc.

[0003] However, in such histopathological diagnosis, since the position at which the tissue slice is cut from the specimen is determined by the operator, it can be difficult to cut the tissue slice at the appropriate position depending on the operator's level of proficiency, which can result in inconsistencies in diagnostic accuracy as the lesion area in the specimen is not always cut out appropriately. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Laid-Open No. 2015-38467 [Patent Document 2] Japanese Patent Application Publication No. 2019-200090 Summary of the Invention [Problem to be solved by the invention]

[0005] One of the problems to be solved by the embodiments disclosed in this specification and the drawings is to appropriately determine the position at which to cut a tissue slice from a specimen. However, the problems to be solved by the embodiments disclosed in this specification and the drawings are not limited to the above problem. Problems corresponding to the effects of each configuration shown in the embodiments described below can also be positioned as other problems. [Means for solving the problem]

[0006] An information processing device according to an embodiment includes a specimen image acquisition unit, a cutout policy acquisition unit, a region of interest recognition unit, and a cutout position determination unit. The specimen image acquisition unit acquires a specimen image obtained by photographing a pathological specimen. The cutout policy acquisition unit acquires a cutout policy that defines rules for cutting out a tissue slice from the pathological specimen. The region of interest recognition unit recognizes a first region of interest from the specimen image. The cutout position determination unit determines a cutout position of the tissue slice in the specimen image based on the first region of interest and the cutout policy. The cut-out policy specifies that the cut-out position should be set so as to be perpendicular to the major axis of the pathological specimen. The cut-out policy acquisition unit acquires the cut-out policy by one of the following methods: accepting a user operation to input the cut-out policy, reading the cut-out policy from a storage unit, or acquiring the cut-out policy from an external device. [Brief explanation of the drawings]

[0007] [Figure 1] FIG. 1 is a diagram illustrating an example of the overall configuration of an information processing system according to the first embodiment. [Figure 2] FIG. 2 is a diagram illustrating a flow until a tissue image is captured according to the first embodiment. [Figure 3] FIG. 3 is a diagram showing an example of an outline of processing executed by the information processing device 100 according to the first embodiment. [Figure 4] FIG. 4 is a diagram illustrating recognition of a region of interest according to the first embodiment. [Figure 5] FIG. 5 is a diagram showing an example of an excision position based on the excision policy (8) "Excises so that the number of tissue slices collected is the greatest at a specified interval" according to the first embodiment. [Figure 6] FIG. 6 is a diagram showing an example of an excision position based on the excision policy (9) "Excision is performed so that the number of tissue slices collected is minimized at a specified interval" according to the first embodiment. [Figure 7] FIG. 7 is a flowchart showing an example of the flow of the cutout position determination process according to the first embodiment. [Figure 8] FIG. 8 is a diagram showing an example of a cutout position according to the first modification of the first embodiment. [Figure 9] FIG. 9 is a diagram showing an example of a cutout position according to the second modification of the first embodiment. [Figure 10]FIG. 10 is a diagram showing an example of a cutout position according to the fifth modification of the first embodiment. [Figure 11] FIG. 11 is a diagram showing an example of a cutout position according to the sixth modification of the first embodiment. [Figure 12] FIG. 12 is a diagram showing an example of a cutout position according to the seventh modification of the first embodiment. [Figure 13] FIG. 13 is a flowchart showing an example of the flow of the cutout position determination process according to the third embodiment. [Figure 14] FIG. 14 is a diagram showing an example of a cutout position according to the first modification of the third embodiment. [Figure 15] FIG. 15 is a diagram illustrating an example of the overall configuration of an information processing system according to the fourth embodiment. [Figure 16] FIG. 16 is a diagram showing an example of division position candidates according to the fourth embodiment. [Figure 17] FIG. 17 is a flowchart showing an example of the flow of a process for determining cutout positions and division position candidates according to the fourth embodiment. [Figure 18] FIG. 18 is a diagram showing an example of division position candidates according to the first modification of the fourth embodiment. [Figure 19] FIG. 19 is a diagram illustrating an example of the overall configuration of an information processing system according to the fifth embodiment. [Figure 20] FIG. 20 is a diagram showing an example of a sample image and a tissue image according to the fifth embodiment. [Figure 21] FIG. 21 is a diagram showing an example of a cutout position in a sample image according to the fifth embodiment. [Figure 22] FIG. 22 is a diagram showing an example of the correspondence between a plurality of cutout positions in a sample image and a plurality of tissue regions in a tissue image according to the fifth embodiment. [Figure 23] FIG. 23 is a diagram showing an example of an ID set by a user according to the fifth embodiment. [Figure 24] FIG. 24 is a diagram showing an example of the correspondence between the cutout line in the sample image and the tissue region on the tissue image according to the fifth embodiment. [Figure 25] FIG. 25 is a diagram showing an example of mapping of regions of interest based on the correspondence between the extraction positions and tissue regions shown in FIG. [Figure 26] FIG. 26 is a flowchart showing an example of the flow of the region of interest mapping process according to the fifth embodiment. [Figure 27] FIG. 27 is a flowchart showing an example of the flow of the region of interest mapping process according to the first modification of the fifth embodiment. [Figure 28] FIG. 28 is a diagram illustrating an example of the overall configuration of an information processing system according to the sixth embodiment. [Figure 29] FIG. 29 is a flowchart showing an example of the flow of the region of interest mapping process according to the sixth embodiment. [Figure 30] FIG. 30 is a diagram showing an example of a cutout position according to the seventh embodiment. [Figure 31] FIG. 31 is a diagram showing an example of mapping of a region of interest according to the seventh embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0008] Hereinafter, embodiments of an information processing device, an information processing method, and a program will be described in detail with reference to the drawings.

[0009] (First embodiment) 1 is a diagram showing an example of the overall configuration of an information processing system S according to the first embodiment. As shown in FIG. 1, the information processing system S includes an information processing device 100, a specimen image storage device 201, a tissue image storage device 202, a first image capturing device 401, and a second image capturing device 402.

[0010] The information processing device 100, the specimen image storage device 201, the tissue image storage device 202, the first photographing device 401, and the second photographing device 402 are communicatively connected to the specimen image storage device 201 and the tissue image storage device 202 via a network 300 such as an in-hospital LAN (Local Area Network).

[0011] The information processing system S may further include a specimen management system, a hospital information system (HIS), a laboratory information system (LIS), a radiology information system (RIS), etc. Alternatively, the information processing system S may be part of a hospital information system. The information processing system S may further include a terminal device such as a personal computer (PC) or a tablet terminal.

[0012] The information processing system S is installed, for example, in a medical institution such as a hospital, a research institute such as a university, an examination center, etc. Furthermore, some or all of the devices constituting the information processing system S may be installed in a cloud environment.

[0013] The first imaging device 401 captures an image of a tissue slice, which is a specimen. In this embodiment, an image of the entire tissue slice, which is a pathological specimen (hereinafter simply referred to as a specimen), is referred to as a specimen image. The first imaging device 401 is, for example, a digital camera, but is not limited to this. For example, the first imaging device 401 may be an analog camera or a video camera. The first imaging device 401 may also be an OCT (Optical Coherence Tomography) device, a MicroCT (Computed Tomography) device, a MicroMRI (Magnetic Resonance Imaging) device, an X-ray CT device, an MRI (Magnetic Resonance Imaging) device, or the like, which are capable of capturing three-dimensional tomographic images of the specimen. The first imaging device 401 may also be a three-dimensional scanner or the like that acquires a surface point cloud of the specimen. The specimen image may be processed, for example, by staining, to make abnormal regions more easily visible. The specimen image is also referred to as a macro image.

[0014] In this embodiment, the specimen is a partial tissue collected from a patient's body, an animal, etc. For example, the specimen is a tissue fragment such as the mucosa and submucosa of the digestive tract excised by endoscopic mucosal resection (EMR) and endoscopic submucosal dissection (ESD), but is not limited thereto and may also be a tissue fragment excised by laparotomy or the like.

[0015] The second imaging device 402 captures an image of at least one slice cut from a tissue slice, which is a specimen, at a magnification higher than that of a specimen image. In this embodiment, an image of at least one slice cut from a tissue slice, which is a specimen, at a magnification higher than that of a specimen image is referred to as a tissue image. The second imaging device 402 is, for example, a WSS (Whole Slide Scanner) or a digital microscope. A WSS is a device that captures a WSI (Whole Slide Imaging) image, which is a highly accurate digital image of the entire or part of a slice placed on a glass slide, but is not limited to this. The tissue image is also referred to as a micro-image.

[0016] The first image capturing device 401 and the second image capturing device 402 do not necessarily have to be included in the information processing system S.

[0017] The sample image storage device 201 is a device that stores sample images captured by the first image capturing device 401. Note that the sample image storage device 201 may store sample images captured by a device other than the first image capturing device 401.

[0018] The tissue image storage device 202 is a device that stores tissue images captured by the second imaging device 402. Note that the tissue image storage device 202 may store specimen images captured by a device other than the second imaging device 402.

[0019] The sample image storage device 201 and the tissue image storage device 202 are, for example, a server device or a PC. The sample image storage device 201 and the tissue image storage device 202 may be collectively referred to as an image storage device. Although the sample image storage device 201 and the tissue image storage device 202 are shown as separate devices in FIG. 1, the sample image storage device 201 and the tissue image storage device 202 may be configured as an integrated device. The sample image storage device 201, the tissue image storage device 202, and the information processing device 100 may be configured as an integrated device.

[0020] FIG. 2 is a diagram illustrating the flow up to capturing a tissue image according to the first embodiment. For example, a specimen 5 collected from a patient P by a diagnostician is transferred to a pathologist or a laboratory technician for pathological diagnosis. At this time, a pathology number is assigned for each request (order) for pathological diagnosis. A different specimen number is assigned for each specimen to be subjected to pathological diagnosis. Note that when there are multiple specimens to be diagnosed in one request for pathological diagnosis, multiple specimen numbers are associated with one pathology number. Hereinafter, in this embodiment, a pathologist or laboratory technician will be referred to as a pathologist or the like.

[0021] The processing performed by the information processing device 100 in this embodiment does not necessarily have to be for the purpose of pathological diagnosis. For example, the processing may be for the purpose of research by a research institute or for the purpose of creating a report at an examination center that performs pathological examinations on behalf of a medical institution.

[0022] A sample image ID is assigned as identification information for a sample image 51 obtained by capturing an image of the sample 5 using a digital camera or the like. The sample image ID may be registered as additional information for the sample image 51. Information related to the sample image 51, such as the pathology number of the sample 5 that is the subject of the image of the sample image 51, the sample number of the sample 5, the order number of the pathological diagnosis related to the sample 5, the patient ID of the patient P from whom the sample 5 was obtained, and the image date and time of the sample image 51, may be registered as additional information for the sample image 51. The sample image ID and other information related to the sample image 51 may be displayed as text information on the sample image 51.

[0023] A pathologist or the like cuts out tissue slices 60a to 60e from specimen 5, and after performing processing such as staining, for example, slices the cross sections of tissue slices 60a to 60e thinly and places them on glass slides 7. Hereinafter, when there is no need to distinguish between the individual tissue slices 60a to 60e, they will simply be referred to as tissue slices 60.

[0024] In this embodiment, the group of sections placed on the slide glass 7 is referred to as a tissue specimen 6. The tissue specimen 6 includes at least one tissue section 60.

[0025] In addition, in Figure 2, each tissue slice 60 is placed directly on the glass slide 7, but if the tissue slice 60 is long or depending on the shape of the cutting position, one tissue slice 60 may be divided into multiple pieces and placed on the glass slide 7.

[0026] In FIG. 2, as an example, the tissue slices 60a to 60e are imaged with their cross sections facing the imaging device, but the placement orientation is not limited to this. For example, the tissue slice 60 may be imaged in the same orientation as the specimen 5 when the specimen image 51 is captured. In addition, in FIG. 2, the tissue slice 60 is placed on the glass slide 7 so that the horizontal direction of the tissue image 61 corresponds to the longitudinal direction of the tissue region 62, but the placement orientation is not limited to this. For example, the tissue slice 60 may be placed on the glass slide 7 so that the vertical direction of the tissue image 61 corresponds to the longitudinal direction of the tissue region 62. Alternatively, the tissue slice 60 may be placed at an angle on the glass slide 7. The placement of the tissue slice 60 is assumed to be determined, for example, by the slice placement rules of each medical institution.

[0027] An image such as a WSI image obtained by capturing an image of the tissue specimen 6 placed on the slide glass 7 is a tissue image 61. In the tissue image 61, the tissue specimen 6 including tissue slices 60a to 60e is depicted.

[0028] A tissue image ID is assigned as identification information of the tissue image 61. The tissue image ID may be registered as additional information of the tissue image 61. Information related to the tissue image 61, such as the pathology number of the specimen 5 from which the section that is the subject of the imaging of the tissue image 61 was obtained, the specimen number of the specimen 5, the order number of the pathological diagnosis related to the specimen 5, the patient ID of the patient P from whom the specimen 5 was obtained, and the imaging date and time of the tissue image 61, may be registered as additional information of the tissue image 61. The tissue image ID and other information related to the tissue image 61 may be rendered as text information on the tissue image 61. The additional information of the tissue image 61 may also include identification information of the facility, such as a medical institution, that captured the tissue image 61. The identification information of the facility, such as a medical institution, that captured the imaging process of the tissue image 61 may be obtained, for example, from a clinical testing system or a specimen management system.

[0029] The tissue image 61 includes a background region 70 depicting the glass slide 7, and a plurality of tissue regions 62a to 62e depicting the tissue specimen 6. Hereinafter, the tissue regions 62a to 62e will be simply referred to as tissue regions 62 when they are not to be distinguished from one another.

[0030] The information processing device 100 of this embodiment determines the extraction position for extracting the tissue specimen 6 from the specimen 5 before the tissue image 61 is captured, thereby assisting the work of a technician or the like.

[0031] 3 is a diagram showing an example of an overview of processing executed by the information processing device 100 according to the first embodiment. As shown in FIG. 3, the information processing device 100 extracts a region of interest from a specimen image 51, and determines a cut-out position of the tissue slice based on the extracted region of interest and a tissue slice cut-out policy. The information processing device 100 displays a cut-out line 53 indicating the determined cut-out position superimposed on the specimen image 51, thereby presenting the cut-out position to a technician or the like.

[0032] The region of interest is, for example, an image region of the specimen image 51 in which a specific tissue that is important in pathological diagnosis is depicted. For example, the region of interest is a lesion region in which a lesion site is depicted. The lesion site is, for example, a tumor such as an adenoma or adenocarcinoma. The region of interest is not limited to a lesion, but may also be an abnormal region in which some abnormality has occurred. Hereinafter, the term "lesion region" in this embodiment may be read as "abnormal region."

[0033] Furthermore, the region of interest is an image region in which an abnormal region such as a tumor is depicted. Alternatively, the region of interest may include, for example, an image region in which normal tissues such as the mucosa, muscularis mucosa, submucosa, muscularis proper, or subserosa are depicted in the digestive tract, or normal regions such as blood vessels, muscle, fat, or parenchyma specific to various organs in other organs. Normal regions free of tumors are used, for example, for measuring tumors. The region of interest may include any one tissue or multiple types of tissue.

[0034] 3, the region of interest is a lesion region 511 included in a specimen region 510 in which the specimen 5 is depicted in the specimen image 51. Alternatively, the specimen region 510 may be the region of interest.

[0035] Returning to FIG. 1, the information processing device 100 is, for example, a server device or a PC, and includes a NW interface 110, a storage circuit 120, an input interface 130, a display 140, and a processing circuit 150.

[0036] The NW interface 110 is connected to the processing circuit 150 and controls the transmission and communication of various data between the information processing device 100 and the first image capturing device 401, the second image capturing device 402, the specimen image storage device 201, and the tissue image storage device 202. The NW interface 110 is realized by a network card, a network adapter, a NIC (Network Interface Controller), or the like.

[0037] The storage circuitry 120 stores in advance various types of information used by the processing circuitry 150. The storage circuitry 120 also stores various programs. The storage circuitry 120 also stores a cutout policy, which will be described later.

[0038] The input interface 130 may be realized by a trackball, switch buttons, a mouse, a keyboard, a touchpad that performs input operations by touching the operation surface, a touchscreen that integrates a display screen and a touchpad, a non-contact input circuit using an optical sensor, a voice input circuit, etc. The input interface 130 is connected to the processing circuit 150 and converts input operations received from an operator into electrical signals and outputs them to the processing circuit 150. Note that in this specification, the input interface is not limited to those that have physical operation components such as a mouse and a keyboard. For example, an electrical signal processing circuit that receives electrical signals corresponding to input operations from an external input device provided separately from the information processing device 100 and outputs these electrical signals to the processing circuit 150 is also included as an example of the input interface 130.

[0039] The display 140 is a liquid crystal display, an organic electro-luminescence (OEL) display, or the like. The input interface 130 and the display 140 may be integrated. For example, the input interface 130 and the display 140 may be realized by a touch panel. The display 140 is an example of a display unit.

[0040] The processing circuitry 150 is a processor that reads out programs from the storage circuitry 120 and executes them to realize functions corresponding to each program. The processing circuitry 150 of this embodiment includes a sample image acquisition function 151, a cutout policy acquisition function 152, a region of interest recognition function 153, a cutout position determination function 154, a display control function 155, and a reception function 156. The sample image acquisition function 151 is an example of a sample image acquisition unit. The cutout policy acquisition function 152 is an example of a cutout policy acquisition unit. The region of interest recognition function 153 is an example of a region of interest recognition unit. The cutout position determination function 154 is an example of a cutout position determination unit. The display control function 155 is an example of a display control unit. The reception function 156 is an example of a reception unit.

[0041] Here, for example, each processing function of the processing circuitry 150, i.e., the specimen image acquisition function 151, the cutout policy acquisition function 152, the region of interest recognition function 153, the cutout position determination function 154, the display control function 155, and the reception function 156, is stored in the storage circuitry 120 in the form of a computer-executable program. The processing circuitry 150 is a processor. For example, the processing circuitry 150 realizes the function corresponding to each program by reading and executing the program from the storage circuitry 120. In other words, the processing circuitry 150 in a state in which each program has been read out has each function shown in the processing circuitry 150 of FIG. 1. Note that, although FIG. 1 illustrates the processing functions performed by the specimen image acquisition function 151, the cutout policy acquisition function 152, the region of interest recognition function 153, the cutout position determination function 154, the display control function 155, and the reception function 156 being realized by a single processor, the processing circuitry 150 may be configured by combining multiple independent processors, and each processor may realize the function by executing a program. Furthermore, although FIG. 1 illustrates a single memory circuit 120 storing a program corresponding to each processing function, multiple memory circuits may be distributed and arranged, and the processing circuit 150 may read out the corresponding program from each memory circuit.

[0042] In the above description, an example has been described in which a "processor" reads and executes a program corresponding to each function from a storage circuit. However, the present embodiment is not limited to this. The term "processor" refers to a circuit such as a central processing unit (CPU), a graphics processing unit (GPU), an application-specific integrated circuit (ASIC), a programmable logic device (e.g., a simple programmable logic device (SPLD), a complex programmable logic device (CPLD), and a field programmable gate array (FPGA)). If the processor is a CPU, for example, the processor realizes a function by reading and executing a program stored in a storage circuit. On the other hand, if the processor is an ASIC, instead of storing a program in the storage circuit 120, the function is directly incorporated as a logic circuit within the processor circuit. Note that each processor in the present embodiment is not limited to being configured as a single circuit per processor, and may be configured as a single processor by combining multiple independent circuits to realize its function. Furthermore, the functions of the multiple components in FIG. 1 may be realized by integrating them into a single processor.

[0043] The sample image acquisition function 151 acquires a sample image 51 from the first image capturing device 401 or the sample image storage device 201 .

[0044] The specimen image 51 to be acquired is designated by, for example, a user operation received by a later-described reception function 156. The user of the information processing device 100 is a pathologist or the like.

[0045] The user specifies the specimen image 51 to be acquired by inputting, for example, a pathology number, a patient ID, a specimen number, an order number, an imaging date, or a combination of these.

[0046] The user may also select the sample image to be acquired from a list of sample images 51 stored in the sample image storage device 201. The sample image 51 may be stored in advance in the memory circuitry 120 and read out by a user operation.

[0047] The cut-out policy acquisition function 152 acquires a cut-out policy that defines the rules for cutting out a tissue slice 60 from a specimen 5 .

[0048] The cutout policy acquisition function 152 may, for example, accept a user operation to input a cutout policy to be used in processing. The cutout policy acquisition function 152 may also accept a user operation to select a cutout policy to be used in processing from among a plurality of cutout policies pre-stored in the memory circuitry 120. If a cutout policy is set as a default value in the memory circuitry 120, the cutout policy acquisition function 152 may acquire the cutout policy set as the default value. The default value of the cutout policy is set in advance by, for example, a user or an administrator. The cutout policy acquisition function 152 may also automatically select one cutout policy to be used in processing using various algorithms. The cutout policy acquisition function 152 may also acquire a cutout policy associated with the sample image 51 from the first imaging device 401, the sample image storage device 201, or another device.

[0049] The cutout policy in this embodiment is, for example, one of the following. (1) Set the cutting position so that it is perpendicular to the major axis of the specimen region. (2) Set the extraction position so that it is perpendicular to the major axis of the lesion area. (3) Set the cutting position so that it is perpendicular to the minor axis of the specimen region. (4) Set the extraction position so that it is perpendicular to the minor axis of the lesion area. (5) An arbitrary cut-out position is set so that it passes through the center of gravity of the specimen region. (6) An arbitrary extraction position is set so that it passes through the center of gravity of the lesion area. (7) A cutout position is temporarily set, and the temporarily set cutout position is rotated to set a cutout position that satisfies the conditions. (8) Cut tissue sections so that the number of tissue sections collected is maximized at specified intervals. (9) Cut the tissue sections so that the number of tissue sections collected is minimized at specified intervals. (10) Cut tissue sections to include the lesion area. (11) Cut each tissue section so that it is smaller than the specified size. (12) Cut each tissue section so that it is larger than the specified size. (13) Cut each tissue section to approximately the specified size. (14) Cut each tissue section so that it is as uniform in size as possible. (15) Cut each tissue section at equal intervals to the specified number. (16) The designated number of tissue sections are cut out in descending order of the probability that they represent a lesion area. (17) The specified number of cutouts is used to cut out the area that contains the most lesion area. (18) After the specified number of cutouts, the lesion area is cut out in descending order of probability. (19) Cut the tissue section so that it is at its thickest. (20) Cut the tissue section so that it is as thin as possible. (21) Cut out tissue sections containing the lesion area at the maximum number of intervals. (22) Do not set the cutout position.

[0050] The cut-out policy acquisition function 152 may select multiple cut-out policies from the above as long as the cut-out policies are not contradictory to each other. For example, the cut-out methods of the above cut-out policies (1) to (7) differ depending on whether the "region of interest" is defined as a specimen region or a lesion region. They can be combined with (8) to (21). Note that cut-out policies (1) to (6) are not limited to the above examples. The cut-out policies are also called slice cut-out rules.

[0051] The region of interest recognition function 153 recognizes a region of interest from the specimen image 51 .

[0052] Fig. 4 is a diagram illustrating recognition of a region of interest according to the first embodiment. In the example shown in Fig. 4, similar to Fig. 3 described above, a lesion region 511 included in a part of a specimen region 510 in which the specimen 5 is depicted in the specimen image 51 is assumed to be the region of interest.

[0053] When the sample image 51 is a two-dimensional image, the region of interest recognition function 153 recognizes a region of interest based on the surface features of the sample 5 depicted in the sample image 51.

[0054] More specifically, the region of interest recognition function 153 extracts a specimen region 510 and a lesion region 511 from the specimen image 51 by image segmentation processing.

[0055] In this embodiment, the lesion region 511 on the specimen image 51 is an example of a first region of interest in this embodiment. In addition, the specimen region 510 in the specimen image 51, in which the specimen 5 is depicted, may be an example of a first region of interest.

[0056] Note that the region of interest recognition function 153 is not limited to image segmentation processing, and may also perform recognition by rule-based image processing such as selective search, manual operation by a user, or inference using machine learning. For example, a trained model in which the specimen region 510 and the lesion region have been trained in advance by deep learning may be stored in the storage circuitry 120. In this case, the region of interest recognition function 153 reads out the trained model from the storage circuitry 120 and inputs the specimen image 51. The trained model may also be incorporated into the region of interest recognition function 153. Note that the region of interest recognition function 153 may recognize only either the specimen region 510 or the lesion region.

[0057] 4 illustrates an example in which the specimen image 51 is a two-dimensional image, but if the specimen image 51 is a three-dimensional tomographic image, the region of interest recognition function 153 recognizes a lesion region 511 inside the specimen 5 as the region of interest. For example, if the specimen image 51 is a three-dimensional image, the region of interest recognition function 153 recognizes the region of interest based on the surface features of the specimen 5 and the internal features of the specimen 5 depicted in the specimen image 51. Furthermore, if the specimen image 51 is a surface point cloud, the region of interest recognition function 153 recognizes the specimen region 510 as the region of interest.

[0058] Furthermore, the region of interest recognition function 153 may calculate the probability that each point on the sample image 51 is a region of interest, rather than uniquely identifying the range of the region of interest.

[0059] Returning to Figure 1, the cut-out position determination function 154 determines the cut-out position of the tissue slice 60 in the specimen image 51 based on the region of interest (e.g., specimen region 510 or lesion region 511) recognized by the region of interest recognition function 153 and the cut-out policy acquired by the cut-out policy acquisition function 152.

[0060] FIG. 5 is a diagram showing an example of an excision position based on the excision policy (8) "Excises so that the number of tissue slices collected is the greatest at a specified interval" according to the first embodiment.

[0061] In this case, the cut-out position determination function 154 identifies the major axis 521 of the specimen region 510. The major axis 521 of the specimen region 510 is the longest straight line among the straight lines connecting the two ends of the specimen region 510. The cut-out position determination function 154 sets cut-out positions perpendicular to the identified major axis 521 at a specified interval. The specified interval may be stored in advance in the memory circuitry 120, for example, or may be specified by the user when the process is executed.

[0062] The cut-out position determination function 154 defines the cut-out position by, for example, coordinates on the specimen image 51. In FIG.

[0063] Even if cut-out policy (1) "Calculate the major axis of the specimen region and set the cut-out position so that it is perpendicular to the major axis" is selected, the cut-out position determination function 154 determines the cut-out position using the same method as in Figure 5.

[0064] FIG. 6 is a diagram showing an example of a cut-out position based on cut-out policy (9) "Cut out so that the number of tissue slices collected is minimized at specified intervals" according to the first embodiment.

[0065] In this case, the cut-out position determination function 154 identifies the minor axis 522 of the specimen region 510. The minor axis 522 of the specimen region 510 is the longest straight line connecting the two ends of the specimen region 510 that is perpendicular to the major axis 521 of the specimen region 510. The cut-out position determination function 154 sets cut-out positions perpendicular to the identified minor axis 522 at specified intervals. In FIG. 6, the cut-out positions are illustrated by multiple cut-out lines 53.

[0066] Even when cut-out policy (3) "Set the cut-out position so that it is perpendicular to the minor axis of the specimen region" is selected, the cut-out position determination function 154 determines the cut-out position using the same method as in FIG.

[0067] Furthermore, when the cut-out policy (22) "Do not set cut-out position" is selected, the cut-out position determination function 154 does not set the cut-out position.

[0068] Returning to FIG. 1, the display control function 155 causes the display 140 to display the sample image 51 on which the cut-out line 53 indicating the cut-out position determined by the cut-out position determination function 154 is superimposed.

[0069] A technician or the like can cut out a tissue section 60 from the specimen 5 at the position of the cut-out line 53 on the specimen image 51 displayed on the display 140, thereby cutting out the tissue section 60 at an appropriate position in accordance with the cut-out policy.

[0070] The reception function 156 receives various operations from the user via the input interface 130 .

[0071] Next, the flow of the cutout position determination process executed by the information processing device 100 of this embodiment configured as above will be described.

[0072] FIG. 7 is a flowchart showing an example of the flow of the cutout position determination process according to the first embodiment.

[0073] First, the sample image acquisition function 151 acquires a sample image 51 from the first image capturing device 401 or the sample image storage device 201 (S101).

[0074] Then, the cut-out policy acquisition function 152 acquires a cut-out policy based on a user operation or a default setting (S102).

[0075] Next, the region of interest recognition function 153 recognizes a region of interest from the sample image 51 acquired in S101 by image segmentation processing or processing using a trained model (S103). The region of interest recognition function 153 also recognizes a sample region 510 from the sample image 51.

[0076] Then, the cut-out position determination function 154 determines the cut-out positions of the tissue slices 60 in the specimen image 51 based on the region of interest recognized by the region of interest recognition function 153 and the cut-out policy acquired by the cut-out policy acquisition function 152 (S104). For example, if policy (8) "Cut out so that the number of tissue slices collected is maximized at a specified interval" is acquired, the cut-out position determination function 154 determines the cut-out positions so that the cut-out positions perpendicular to the major axis 521 of the specimen region 510 are aligned at a specified interval, as shown in FIG. 5. The cut-out position determination function 154 stores the determined cut-out positions in the memory circuitry 120.

[0077] Then, the display control function 155 superimposes the cut-out line 53 representing the cut-out position determined by the cut-out position determination function 154 on the sample image 51 and displays it on the display 140 (S105). Note that the timing of this display is not particularly limited. For example, the display control function 155 may superimpose the cut-out line 53 representing the cut-out position on the sample image 51 immediately after the cut-out position determination function 154 has determined the cut-out position, or may display it when the reception function 156 receives an operation from the user to inquire about the cut-out position. At this point, the processing of this flowchart ends.

[0078] In this way, the information processing device 100 of this embodiment determines the cut-out position of the tissue slice 60 in the specimen image 51 based on the region of interest recognized from the specimen image 51 and the cut-out policy. Therefore, the information processing device 100 of this embodiment can appropriately determine the cut-out position of the tissue slice from the specimen 5, regardless of the level of proficiency of the technician or the like.

[0079] (Modification 1 of the first embodiment) In this modification, a method for determining the cut-out position when the cut-out policy (10) "Cut out the tissue slice so that it includes the lesion area" is acquired will be described.

[0080] FIG. 8 is a diagram showing an example of a cutout position according to the first modification of the first embodiment.

[0081] The cut-out position determination function 154 identifies the position of the center of gravity 523 of the lesion region 511 recognized by the region-of-interest recognition function 153 in the specimen image 51. The coordinates of the center of gravity 523 on the specimen image 51 are, for example, the average value of the coordinates of each pixel included in the lesion region 511 in the specimen image 51. The cut-out position determination function 154 calculates the positions of a first line that is the longest among the lines connecting the two ends of the specimen region 510 and that passes through the identified center of gravity 523, and a second line that is perpendicular to the first line, and determines these as the cut-out positions. In Figure 8, the cut-out position corresponding to the first line is illustrated as cut-out line 53a, and the cut-out position corresponding to the second line is illustrated as cut-out line 53b.

[0082] The cutout position determination function 154 may use the intersection of the major axis and minor axis of the lesion region 511 instead of the center of gravity 523. Alternatively, if the region of interest recognition function 153 calculates the probability that a point is likely to be a region of interest, the point with the highest probability may be used instead of the center of gravity 523. Furthermore, if there are multiple points with the highest probability, the cutout position determination function 154 may determine the cutout position so that the center of gravity of those points and the cutout position pass through those points.

[0083] Even when cut-out policy (6) "Set an arbitrary cut-out position so that it passes through the center of gravity of the focus area" is selected, the cut-out position determination function 154 determines the cut-out position in the same manner as in FIG.

[0084] (Modification 2 of the first embodiment) In this modification, a method for determining the cut-out position when cut-out policy (12) "Cut out each tissue slice so that it is larger than the specified size" is acquired will be described. The size of the tissue slice 60 represents the length of the tissue slice 60 in the longitudinal direction. The specified size may be stored in advance in the memory circuitry 120 or may be input by the user.

[0085] 9 is a diagram showing an example of cut-out positions according to Modification 2 of the first embodiment. The cut-out position determination function 154 provisionally sets cut-out positions at specified intervals that are perpendicular to the major axis of the specimen region 510. In FIG. 9, the provisionally set cut-out positions are indicated by provisional setting lines 530.

[0086] The excision position determination function 154 rotates the provisionally set excision position and determines, as the excision position, a position where the tissue slice 60 excised at the provisionally set excision position is larger than the specified size. Specifically, the excision position determination function 154 determines, as the excision position, a position where the length of the provisionally set line 530 on the specimen region 510 is longer than the specified size. In FIG. 9, the determined excision position is indicated by the excision line 53. The center of rotation is the center of gravity or center of the specimen region 510.

[0087] Furthermore, when the cut-out policy (11) "Cut out each tissue slice so that it is smaller than the specified size" is acquired, the cut-out position determination function 154 provisionally sets a cut-out position perpendicular to the minor axis, rotates the provisionally set cut-out position, and determines, as the cut-out position, a position where the tissue slice 60 cut out at the provisionally set cut-out position is smaller than the specified size. Specifically, the cut-out position determination function 154 determines, as the cut-out position, a position where the length of the provisionally set line 530 on the specimen region 510 is shorter than the specified size.

[0088] (Modification 3 of the first embodiment) In this modification, a method for determining the cut-out position when the cut-out policy (13) "Cut out each tissue slice so that it is close to a specified size" is acquired will be described.

[0089] In this case, the excision position determination function 154 identifies the major axis or minor axis of the specimen region 510 and provisionally sets excision positions perpendicular to the major axis or minor axis at specified intervals. The excision position determination function 154 rotates the provisionally set excision positions to determine the position at which the difference between the size of the tissue slice 60 excised at the provisionally set excision position and the specified size is smallest. The difference between the size of the tissue slice 60 and the specified size is, specifically, the difference between the length of the provisionally set line 530 on the specimen region 510 and the specified size.

[0090] Furthermore, when the cut-out policy (14) "Cut out each tissue slice so that it is as uniform in size as possible" is acquired, the cut-out position determination function 154 identifies the major axis or minor axis of the specimen region 510 and provisionally sets cut-out positions perpendicular to the major axis or minor axis at specified intervals. The cut-out position determination function 154 rotates the provisionally set cut-out positions and determines the positions at which the variance in the sizes of the tissue slices 60 cut out at the provisionally set cut-out positions is smallest as the cut-out positions. Specifically, the cut-out position determination function 154 determines the positions at which the variance in the lengths of the provisionally set lines 530 on the specimen region 510 is smallest as the cut-out positions.

[0091] (Fourth modification of the first embodiment) In this modification, a method for determining the cut-out position when the cut-out policy (15) "Cut out each tissue slice at equal intervals by the specified number" is acquired will be described. The number of tissue slices 60 is designated by input, for example, by the user.

[0092] In this case, the cut-out position determination function 154 identifies the major axis or minor axis of the specimen region 510, and sets a designated number of cut-out positions perpendicular to the identified major axis or minor axis at equal intervals.

[0093] (Fifth Modification of the First Embodiment) In this modified example, a method for determining the cut-out position when the cut-out policy (16) "Cut out the specified number of tissue slices in descending order of the probability that each tissue slice will be a lesion region" is acquired will be described. The number of tissue slices 60 is designated by input, for example, by the user. The probability that each tissue slice will be a region of interest is also called the probability of interest.

[0094] FIG. 10 is a diagram showing an example of a cutout position according to Modification 5 of the first embodiment. In this modification, the cutout position determination function 154 identifies the major axis 521 (or minor axis 522) of the specimen region 510. In this modification, the interest probability of each point on the specimen image 51 is calculated. The cutout position determination function 154 sets a specified number of cutout positions perpendicular to the identified major axis 521 (or minor axis 522) in descending order of interest probability. Note that the minimum value of the interval between cutout positions may be predetermined. In FIG. 10, the determined cutout positions are indicated by cutout lines 53.

[0095] (Modification 6 of the first embodiment) In this modification, a case where the sample image 51 is a three-dimensional image will be described.

[0096] Fig. 11 is a diagram showing an example of a cutout position according to Modification 6 of the first embodiment. A specimen image 51 shown in Fig. 11 is a three-dimensional image.

[0097] In this modification, it is assumed that the following cut-out policies have been acquired: (8) "Cut out tissue slices so that the number of tissue slices collected is maximized at specified intervals" and (10) "Cut out tissue slices so that they include the lesion region." In this case, the cut-out position determination function 154 identifies the major axis 524 of the lesion region 511. The cut-out position determination function 154 determines cut-out positions perpendicular to the identified major axis 524 at specified intervals.

[0098] When the specimen image 51 is a three-dimensional image as in this modification, the cut-out position is not limited to a line but may represent a cross section in three-dimensional space. For example, the cut-out position determination function 154 may determine, as the cut-out position, a plane oblique to the cut-out table on which the specimen 5 is placed.

[0099] Note that when cut-out policy (9) "Cut out so that the number of tissue slices collected is minimized at specified intervals" is selected instead of cut-out policy (8), the cut-out position determination function 154 identifies the minor axis of the lesion region 511. The cut-out position determination function 154 determines cut-out positions perpendicular to the identified minor axis at specified intervals.

[0100] (Seventh modification of the first embodiment) In this modified example, a case will be described in which the specimen image 51 is a three-dimensional image and the cutout policy (19) "Cut out so that the tissue slice is at its thickest" is selected.

[0101] 12 is a diagram showing an example of cut-out positions according to Modification 7 of the first embodiment. In this case, the cut-out position determination function 154 identifies the major axis 521 (or minor axis 522) of the specimen region 510. The cut-out position determination function 154 provisionally sets cut-out positions perpendicular to the identified major axis 521 (or minor axis 522) at specified intervals. In FIG. 12, the provisionally set cut-out positions are indicated by provisional setting lines 530.

[0102] The cut-out position determination function 154 moves the provisionally set cut-out position along the major axis 521 (or minor axis 522) and determines the position where the average thickness of the specimen 5 at the cut-out position is the thickest as the cut-out position. In FIG. 12, the determined cut-out position is indicated by a cut-out line 53.

[0103] In addition, if cut-out policy (20) "Cut out so that the thickness of the tissue slice is the thinnest" is selected instead of cut-out policy (8), the cut-out position determination function 154 moves the provisionally set cut-out position along the long diameter 521 (or short diameter 522) and determines the position where the average thickness of the specimen 5 at the cut-out position is the thinnest as the cut-out position.

[0104] (Second embodiment) In the first embodiment described above, the region of interest recognition function 153 recognized a region of interest from the sample image 51 by image segmentation processing or processing using a trained model, regardless of the cut-out policy. In this second embodiment, the region of interest recognition function 153 identifies the features of the region of interest to be recognized from the sample image 51 based on the cut-out policy, and then recognizes the region of interest from the sample image 51 according to the features.

[0105] Similar to the first embodiment, the processing circuit 150 of the information processing device 100 of this embodiment includes a specimen image acquisition function 151, a cutout policy acquisition function 152, a region of interest recognition function 153, a cutout position determination function 154, a display control function 155, and a reception function 156. The specimen image acquisition function 151, the cutout policy acquisition function 152, the cutout position determination function 154, the display control function 155, and the reception function 156 have the same functions as those of the first embodiment.

[0106] In this embodiment, if the cut-out policy acquired by the cut-out policy acquisition function 152 contains a description regarding the definition of a region of interest, the region of interest recognition function 153 recognizes the region in the sample image 51 that corresponds to the description as a region of interest.

[0107] For example, assume that the cut-out policy (21) "Cut out so that the number of tissue slices containing the lesion region is maximized at a specified interval" is acquired by the cut-out policy acquisition function 152. Since this policy includes a description of the lesion region 511, the region-of-interest recognition function 153 recognizes the lesion region 511 as a region of interest.

[0108] As another example, assume that cut-out policy (1) "Set the cut-out position so that it is perpendicular to the major axis of the specimen region." is acquired. In this case, since the policy includes a description of the specimen region 510, the region of interest recognition function 153 recognizes the specimen region 510 as a region of interest. In this case, the region of interest recognition function 153 does not need to recognize the lesion region 511.

[0109] Furthermore, if the cut-out policy acquired by the cut-out policy acquisition function 152 does not specify a description regarding the definition of a region of interest, the region of interest recognition function 153 may recognize the specimen region 510 as a region of interest. For example, a cut-out policy that does not specify a description regarding the definition of a region of interest, such as cut-out policy (8) "Cut out so that the number of tissue sections collected at a specified interval is the greatest," specifies the cut-out position based on the entire specimen 5. Therefore, if the cut-out policy does not specify a description regarding the definition of a region of interest, the region of interest recognition function 153 recognizes the specimen region 510 as a region of interest. In this case, the region of interest recognition function 153 does not need to recognize the lesion region 511.

[0110] In this way, according to the information processing device 100 of this embodiment, the characteristics of the region of interest to be recognized from the sample image 51 are identified based on the cut-out policy, and then the region of interest corresponding to the characteristics is recognized from the sample image 51. In addition to the same effects as the first embodiment, unnecessary recognition processing can be reduced.

[0111] (Third embodiment) In the first and second embodiments described above, the cut-out policy acquisition function 152 acquires a cut-out policy by user input or a predetermined setting, regardless of the type of sample 5. In this third embodiment, the cut-out policy acquisition function 152 acquires a cut-out policy to be applied to cut-out of the sample 5 from among a plurality of cut-out policy candidates, depending on the type of sample 5.

[0112] The storage circuitry 120 of this embodiment stores a plurality of candidate cutout policies and the types of specimens 5 in association with each other.

[0113] The type of specimen 5 indicates, for example, the anatomical tissue from which the specimen 5 is excised. For example, the types of specimen 5 include "gastric mucosa," "colon mucosa," "mammary gland tissue," and "renal tissue."

[0114] A plurality of types may be associated with one cut-out policy candidate in the storage circuitry 120. The plurality of cut-out policy candidates may be, for example, the cut-out policies (1) to (21) exemplified in the first embodiment, but are not limited to these.

[0115] Similar to the first embodiment, the processing circuit 150 of the information processing device 100 of this embodiment includes a specimen image acquisition function 151, a cutout policy acquisition function 152, a region of interest recognition function 153, a cutout position determination function 154, a display control function 155, and a reception function 156. The specimen image acquisition function 151, the region of interest recognition function 153, the cutout position determination function 154, the display control function 155, and the reception function 156 have the same functions as those of the first embodiment.

[0116] The cut-out policy acquisition function 152 of this embodiment acquires a cut-out policy to be applied to cut-out of the sample 5 from a plurality of cut-out policy candidates stored in the storage circuitry 120 according to the type of the sample 5.

[0117] More specifically, the cutout policy acquisition function 152 of this embodiment identifies the type of the sample 5 based on the region of interest recognized by the region of interest recognition function 153. For example, the cutout policy acquisition function 152 identifies the type of the sample image 51 acquired by the sample image acquisition function 151 using a trained model that has been trained by associating a sample image with the type of sample depicted in the sample image. The trained model may be stored in the memory circuitry 120. In this case, the cutout policy acquisition function 152 reads out the trained model from the memory circuitry 120 and inputs the sample image 51. Furthermore, the trained model may be incorporated into the cutout policy acquisition function 152.

[0118] Alternatively, the type of sample 5 may be associated in advance with the sample image 51. For example, the type of sample 5 may be registered as additional information of the sample image 51. In this case, the cutout policy acquisition function 152 reads out the type of sample 5 registered as additional information of the sample image 51. The method of identifying the type of sample 5 is not limited to these, and may be input by the user, for example.

[0119] Then, the cut-out policy acquisition function 152 acquires the cut-out policy associated with the type of the identified sample 5 from among the multiple cut-out policy candidates stored in the storage circuit 120.

[0120] FIG. 13 is a flowchart showing an example of the flow of the cutout position determination process according to the third embodiment.

[0121] The process of acquiring the sample image 51 in S101 is the same as that in the first embodiment described with reference to Fig. 7. The process of recognizing the region of interest from the sample image 51 in S201 is the same as that in S103 in the first embodiment.

[0122] Next, the cut-out policy acquisition function 152 of this embodiment identifies the type of specimen 5 based on the region of interest recognized by the region of interest recognition function 153, and acquires the cut-out policy associated with the identified type of specimen 5 from among the multiple cut-out policy candidates stored in the memory circuit 120 (S202).

[0123] The process from determining the cutout position in S104 to displaying the cutout line 53 on the specimen image 51 in S105 is the same as in the first embodiment. Here, the process of this flowchart ends.

[0124] (Modification 1 of the third embodiment) In the above-described third embodiment, an extraction policy to be applied to extracting the sample 5 is acquired from among a plurality of extraction policy candidates according to the type of the sample 5. In this modification, an extraction policy to be applied to extracting the sample 5 is acquired according to the region of interest of the sample image 51.

[0125] FIG. 14 is a diagram showing an example of a cutout position according to the first modification of the third embodiment.

[0126] For example, the region of interest recognition function 153 of this modification recognizes a specimen region 510 and a lesion region 511 as regions of interest, as shown in FIG. 14, before acquiring the cutout policy.

[0127] The characteristics of the specimen region 510 and the lesion region 511 and the cut-out policies corresponding to each characteristic are stored, for example, in the memory circuitry 120. The characteristics of the specimen region 510 and the lesion region 511 are, for example, "the aspect ratio of the lesion region 511 is equal to or greater than a threshold," "the aspect ratio of the lesion region 511 is less than a threshold," "the size of the specimen region 510 is equal to or less than a threshold," "the size of the lesion region 511 is equal to or less than a threshold," etc.

[0128] For example, it is assumed that the characteristic of the lesion region 511, "the aspect ratio of the lesion region 511 is equal to or greater than a threshold," is associated with the cut-out policy (8). It is also assumed that the characteristics of the lesion region 511, "the aspect ratio of the lesion region 511 is less than a threshold" and "the size of the lesion region 511 is equal to or less than a threshold," are associated with the cut-out policy (10). It is also possible that the characteristic of the specimen region 510, "the size of the specimen region 510 is equal to or less than a threshold," is associated with no cut-out policy but "no cut-out position is set."

[0129] In this case, if the aspect ratio of the lesion region 511 recognized as the region of interest is equal to or greater than a threshold, the cut-out policy acquisition function 152 acquires cut-out policy (8) from cut-out policies (1) to (21), which is "Cut out so that the number of tissue slices collected is the greatest at a specified interval." If the aspect ratio of the lesion region 511 is less than the threshold, the cut-out policy acquisition function 152 selects cut-out policy (10), which is "Cut out tissue slices so that they include the lesion region. Cut out so that they include the region of interest." Figure 14 illustrates an example in which the aspect ratio of the lesion region 511 is equal to or greater than a threshold. In this case, the cut-out policy acquisition function 152 selects cut-out policy (8).

[0130] Furthermore, the selection of a cut-out policy according to the region of interest of the specimen image 51 is not limited to this. For example, when the size of the specimen region 510 recognized as the region of interest is equal to or smaller than a threshold, the cut-out policy acquisition function 152 does not select any cut-out policy but selects "do not set a cut-out position." Furthermore, when the size of the lesion region 511 recognized as the region of interest is equal to or smaller than a threshold, the cut-out policy acquisition function 152 selects cut-out policy (10) "cut out the tissue slice so that it includes the lesion region." Note that the association between the characteristics of the specimen region 510 and the lesion region 511 and the cut-out policies corresponding to each characteristic is not limited to the above example.

[0131] In this way, according to the information processing device 100 of this embodiment, a cut-out policy to be applied to cut-out of the specimen 5 is obtained from among multiple cut-out policy candidates based on the region of interest extracted from the specimen image 51, and therefore an appropriate cut-out policy can be selected according to the characteristics of the region of interest.

[0132] (Fourth embodiment) In the above-described first to third embodiments, the information processing device 100 has been described as having a function of determining an excision position for excising the tissue slice 60 from the specimen 5. In this fourth embodiment, the information processing device 100 further determines whether or not it is necessary to divide the tissue slice 60 excised from one excision position, and suggests an appropriate division position.

[0133] 15 is a diagram showing an example of the overall configuration of an information processing system S according to the fourth embodiment. Similar to the first to third embodiments, the information processing system S according to this embodiment includes an information processing device 100, a specimen image storage device 201, a tissue image storage device 202, a first image capturing device 401, and a second image capturing device 402.

[0134] The processing circuit 150 of the information processing device 100 of this embodiment includes a specimen image acquisition function 151, a cutout policy acquisition function 152, a region of interest recognition function 153, a cutout position determination function 154, a display control function 155, a reception function 156, and a division position candidate determination function 157. The division position candidate determination function 157 is an example of a division position candidate determination unit. The specimen image acquisition function 151, the cutout policy acquisition function 152, the region of interest recognition function 153, the cutout position determination function 154, and the reception function 156 have the same functions as those of the first embodiment.

[0135] The division position candidate determination function 157 determines a division position candidate for dividing the tissue slice 60 into multiple pieces when the size of the tissue slice 60 cut out at the cut-out position determined by the cut-out position determination function 154 is equal to or larger than a threshold. Furthermore, the division position candidate determination function 157 determines that division is unnecessary when the size of the tissue slice 60 cut out at the cut-out position determined by the cut-out position determination function 154 is smaller than the threshold. The size of the tissue slice 60 is specifically the length of the tissue slice 60 in the longitudinal direction.

[0136] More specifically, the division position candidate determination function 157 calculates the longitudinal length of the tissue slice 60 from the length of the portion of the cutout line 53 of the sample image 51 that is within the sample region 510 and the magnification ratio of the sample image 51. The magnification ratio of the sample image 51 may be determined for each first imaging device 401, for example, or may be included in the supplementary information of the sample image 51.

[0137] Then, the division position candidate determination function 157 determines that a tissue slice 60 whose longitudinal length is equal to or greater than a threshold value is to be divided. The threshold value is not particularly limited. The threshold value may be input by the user or may be determined in advance.

[0138] Furthermore, the division position candidate determination function 157 determines division position candidates for dividing the tissue slice 60 determined to be the division target into multiple pieces. The division position candidates are, for example, regions other than the lesion region 511, and positions where the longitudinal length of each slice after division is less than a threshold value.

[0139] The division position candidate determination function 157 may determine a recommended division region and a quasi-recommended division region according to the degree of division recommendation. For example, if there is a region other than the lesion region 511 in the tissue slice 60 to be divided and the longitudinal length of each slice after division is less than a threshold, the division position candidate determination function 157 determines that region as a recommended division region.

[0140] Furthermore, when there is no recommended division region within the tissue slice 60 to be divided, the division position candidate determination function 157 determines a region that is included in the lesion region 511 but in which the longitudinal length of each slice after division is less than a threshold as a recommended quasi-division region. The recommended division region and the recommended quasi-division region are examples of division position candidates in this embodiment.

[0141] Furthermore, the division position candidate determination function 157 determines an area within the tissue slice 60 to be divided that is neither a recommended division area nor a semi-recommended division area as a non-recommended division area. Furthermore, for a tissue slice 60 that does not require division, the division position candidate determination function 157 determines the entire tissue slice 60 as a non-recommended division area.

[0142] 16 is a diagram showing an example of division position candidates according to the fourth embodiment. In the example shown in FIG. 16, the longitudinal length d1 of the second tissue slice from the top is assumed to be less than a threshold. In this case, the division position candidate determination function 157 determines that the tissue slice is not to be divided. The division position candidate determination function 157 determines that the entire tissue slice is a region not recommended for division.

[0143] Furthermore, the longitudinal length d2 of the third tissue slice from the top is assumed to be equal to or greater than the threshold value. In this case, the division position candidate determination function 157 determines that tissue slice is to be divided. If there is a region among the tissue slices determined to be the target of division that is not included in the lesion region 511 and where the longitudinal length of each slice after division is less than the threshold value, the division position candidate determination function 157 designates that region as a recommended division region. The division position candidate determination function 157 determines whether or not division is necessary for each tissue slice within the specimen region 510, and determines a recommended division region, a semi-recommended division region, or a non-recommended division region.

[0144] The display control function 155 of this embodiment has the same functions as those of the first embodiment, and displays a division candidate position image 590 representing a division recommended region, a semi-division recommended region, and a division non-recommended region superimposed on the sample image 51. For example, as shown in FIG. 16 , the display control function 155 may display rectangles representing the division recommended region, the semi-division recommended region, and the division non-recommended region in different display modes superimposed on the sample image 51.

[0145] Although Figure 16 illustrates an example in which each tissue slice is divided into a maximum of two parts, the division position candidate determination function 157 may determine division position candidates that divide the tissue slice into three or more parts depending on the size of the tissue slice.

[0146] Next, the flow of the process of determining cut-out positions and division position candidates executed by the information processing device 100 of this embodiment configured as above will be described.

[0147] FIG. 17 is a flowchart showing an example of the flow of a process for determining cutout positions and division position candidates according to the fourth embodiment.

[0148] The process from acquiring the sample image 51 in S101 to determining the cut-out position in S104 is the same as that in the first embodiment described with reference to FIG.

[0149] Next, the division position candidate determination function 157 of this embodiment determines a slice division position candidate (S301). Specifically, the division position candidate determination function 157 determines whether each tissue slice 60 is a division target based on the longitudinal lengths of the multiple tissue slices 60 cut out at each cutting position. The division position candidate determination function 157 determines a region other than the lesion region 511 within the tissue slice 60 determined to be a division target, where the longitudinal length of each slice after division is less than a threshold, as a recommended division region. Furthermore, if there is no recommended division region within the tissue slice 60 to be divided, the division position candidate determination function 157 determines a region that is included in the lesion region 511 but where the longitudinal length of each slice after division is less than a threshold, as a quasi-recommended division region. The division position candidate determination function 157 determines a region within the specimen region 510 that is neither a recommended division region nor a quasi-recommended division region as a non-recommended division region. The division position candidate determination function 157 stores the position information of the recommended division area, semi-recommended division area, and non-recommended division area in the sample image 51 in the storage circuitry 120.

[0150] Then, the display control function 155 superimposes the cutout line 53 and the division candidate position image 590 indicating the degree of division recommendation on the specimen image 51, and displays them on the display 140 (S302).

[0151] In this way, according to the information processing device 100 of this embodiment, when the size of the tissue slice 60 cut out at the cut-out position is equal to or larger than a threshold value, candidate division positions for dividing the tissue slice 60 into multiple pieces are determined, thereby assisting technicians and the like in dividing the tissue slice 60 at appropriate positions.

[0152] (Modification 1 of the fourth embodiment) In this modified example, the division position candidate determining function 157 further determines a dividable region common to the plurality of tissue slices 60 to be divided as a division position candidate for the plurality of tissue slices 60.

[0153] FIG. 18 is a diagram showing an example of division position candidates according to Modification 1 of the fourth embodiment. In the example shown on the left side of FIG. 18, a recommended division region and a recommended quasi-division region are determined for each tissue section, as in FIG. 16. In this modification, as shown on the right side of FIG. 18, if there is a recommended division region or recommended quasi-division region that is common to multiple consecutive tissue sections, the division position candidate determination function 157 leaves only the common region and changes the other recommended division regions and recommended quasi-division regions to non-recommended division regions. As a result, as shown in FIG. 18, recommended division regions and recommended quasi-division regions are consecutively arranged on a straight line that is perpendicular to the cut-out position in multiple consecutive tissue sections.

[0154] In addition, the display control function 155 of this modified example displays a division candidate position image 590 representing a division recommended area and a quasi-division recommended area that are common to multiple consecutive tissue sections, superimposed on the specimen image 51, as shown in Figure 18.

[0155] Therefore, according to the information processing device 100 of this modified example, when dividing a plurality of tissue slices 60, a technician or the like can easily grasp the position where a plurality of consecutive tissue slices 60 can be cut at once.

[0156] (Fifth embodiment) In the first to fourth embodiments described above, the information processing device 100 determines the cut-out position in the specimen image 51. In this fifth embodiment, after each tissue slice 60 cut out from the specimen 5 is photographed, the information processing device 100 maps a region of interest in the specimen image 51 to a region of interest in the tissue image 61 in which each tissue slice 60 is depicted.

[0157] 19 is a diagram showing an example of the overall configuration of an information processing system S according to the fifth embodiment. Similar to the first to fourth embodiments, the information processing system S according to this embodiment includes an information processing device 100, a specimen image storage device 201, a tissue image storage device 202, a first image capturing device 401, and a second image capturing device 402.

[0158] The processing circuit 150 of the information processing device 100 of this embodiment includes a specimen image acquisition function 151, a cutout policy acquisition function 152, a region of interest recognition function 153, a cutout position determination function 154, a display control function 155, a reception function 156, a tissue image acquisition function 158, a tissue region recognition function 159, a cutout position acquisition function 160, a cutout position association function 161, and a region of interest association function 162. The tissue image acquisition function 158 is an example of a tissue image acquisition unit. The tissue region recognition function 159 is an example of a tissue region recognition unit. The cutout position acquisition function 160 is an example of a cutout position acquisition unit and a cutout position identification unit. The cutout position association function 161 is an example of a cutout position association unit. The region of interest association function 162 is an example of a region of interest association unit.

[0159] The specimen image acquisition function 151, the cut-out policy acquisition function 152, the region of interest recognition function 153, the cut-out position determination function 154, and the reception function 156 have the same functions as those in the first embodiment.

[0160] The tissue image acquisition function 158 acquires a tissue image 61 from the second image capturing device 402 or the tissue image storage device 202. In the tissue image 61, a group of tissue slices cut out from the specimen 5 are depicted.

[0161] The tissue region recognition function 159 recognizes each region in which multiple tissue slices 60 are depicted in the tissue image 61 as a tissue region 62. Note that multiple tissue slices 60 are collectively referred to as a tissue slice group. The tissue region recognition function 159 recognizes the tissue region 62 by image segmentation processing, rule-based processing, manual operation by a user, a trained model based on machine learning, or the like.

[0162] 20 is a diagram showing an example of a sample image 51 and a tissue image 61 according to the fifth embodiment. In the example shown in FIG. 20, three tissue slices 60 cut out from the sample 5 depicted in the sample image 51 are depicted as tissue regions 62a to 62c on the tissue image 61. The length of each of the tissue regions 62a to 62c linearly corresponds to the length of the cut-out position on the sample region 510 on the sample image 51. Furthermore, in the tissue regions 62a to 62c, a lesion is depicted in a range corresponding to a lesion region 511 within the sample region 510.

[0163] 19 , the cut-out position acquisition function 160 acquires a plurality of cut-out positions representing the collection positions of a plurality of tissue slices 60 in the specimen image 51. For example, the cut-out position acquisition function 160 reads out the cut-out positions determined by the cut-out position determination function 154 from the memory circuitry 120.

[0164] Alternatively, the cut-out position acquisition function 160 may acquire the cut-out position recorded in another pathology diagram system, LIS, or the like. Furthermore, the cut-out position acquisition function 160 may acquire the cut-out position by optically reading a line indicating the cut-out position written on a sheet of paper on which the specimen image 51 is printed. Alternatively, the cut-out position acquisition function 160 may acquire the cut-out position from the position of the cut-out line 53 input by the user on the specimen image 51 displayed on the display 140. The cut-out position in this embodiment is intended to indicate the range from which the tissue piece was actually collected, and may not reach the edge of the specimen 5 or may be interrupted depending on the actual situation.

[0165] Fig. 21 is a diagram showing an example of the cut-out position in the specimen image 51 according to the fifth embodiment. In Fig. 21, the portion of the cut-out line 53 from which the tissue slice 60 has actually been cut out is shown as a solid cut-out line 533, and the portion of the cut-out line 53 from which the tissue slice 60 has not actually been cut out is shown as a dashed display cut-out line 532.

[0166] In the cutting pattern 1 shown in FIG. 21, an actual cutting line 533 is drawn vertically across the specimen region 510, so that the specimen 5 is cut out as a tissue slice 60 from one end to the other.

[0167] Furthermore, in cut-out pattern 2, only a portion of the specimen 5 is cut out as the tissue slice 60, rather than from end to end. For example, when the cut-out position recorded in another pathology drawing system, LIS, or the like indicates that only a portion of the specimen region 510 has been cut out, the cut-out position acquisition function 160 identifies the actual cut-out position in accordance with the record.

[0168] Returning to FIG. 19, the cut-out position associating function 161 identifies the correspondence between a plurality of cut-out positions in the specimen region 510 and a plurality of tissue regions 62 in the tissue image 61.

[0169] Fig. 22 is a diagram showing an example of the correspondence relationship between a plurality of cut-out positions in a specimen image 51 according to the fifth embodiment and a plurality of tissue regions 62d to 62f in a tissue image 61. In Fig. 22, the cut-out positions indicated by cut-out lines 53c to 53e correspond one-to-one to the tissue regions 62d to 62f.

[0170] The correspondence between the multiple cut-out positions in the specimen region 510 and the multiple tissue regions 62 in the tissue image 61 may be performed automatically by the cut-out position correspondence function 161, or may be performed manually by the user, or may be performed automatically and then modified by the user.

[0171] For example, when an ID is set for each of the cut-out positions and the tissue regions 62, the cut-out position associating function 161 associates the IDs of the cut-out positions with the IDs of the tissue regions 62 that match.

[0172] The IDs of the excision positions and the tissue regions 62 may be set in another pathology diagram creation system or LIS. Alternatively, the IDs of the multiple excision positions in the specimen region 510 and the IDs of the multiple tissue regions 62 in the tissue image 61 may be determined according to operation rules predetermined for each medical institution that operates the information processing device 100. The operation rules for each medical institution may be, for example, a section arrangement rule for placing the tissue sections 60 on the glass slide 7. For example, if the section arrangement rule stipulates that the tissue sections 60 are placed on the glass slide 7 in the order in which they are excised from the top of the specimen image 51, the IDs of the excision positions and the IDs of the tissue regions 62 are assigned in the order in which they are depicted at the top of the specimen image 51 and the tissue image 61, respectively. The operation rules for each medical institution may be stored, for example, in the memory circuitry 120.

[0173] Alternatively, a user may operate to register a corresponding ID for each cut-out position in the sample image 51 and each of the multiple tissue regions 62 in the tissue image 61. In this case, based on the user's operation accepted by the accepting function 156, the cut-out position associating function 161 acquires the IDs of each cut-out position and each of the tissue regions 62. The user may be, for example, a technician.

[0174] Fig. 23 is a diagram showing an example of an ID set by a user according to the fifth embodiment. Fig. 23 illustrates an image 55d that is a part of a specimen image 51, and images 65d and 65e that are parts of a tissue image 61. Fig. 23 illustrates a case in which two tissue slices 60 are cut out from one cutout line 53 (53g, 53f), and each tissue slice 60 is depicted in a different image 65 (65e, 65d).

[0175] 23, it is assumed that the user has input ID "1" at the cut-out position indicated by cut-out line 53f, ID "2" at the cut-out position indicated by cut-out line 53g, and ID "3" at the cut-out position indicated by cut-out line 53h. Within one sample image 51, the ID of each cut-out position is unique.

[0176] Tissue region 62g rendered in image 65d is tissue slice 60 cut out from the cut-out position indicated by cut-out line 53f in image 55d. In this case, the user sets ID "1" for tissue region 62g, which is the same as the ID of the cut-out position indicated by cut-out line 53f. Furthermore, tissue region 62h rendered in image 65e is tissue slice 60 cut out from the cut-out position indicated by cut-out line 53g in image 55d. In this case, the user sets ID "2" for tissue region 62h, which is the same as the ID of the cut-out position indicated by cut-out line 53g.

[0177] Returning to Figure 19, the region of interest matching function 162 identifies the positions of multiple regions of interest in multiple tissue regions 62 corresponding to the region of interest in the tissue image 61 based on the region of interest in the specimen image 51 and the correspondence between multiple cut-out positions and multiple tissue regions 62.

[0178] The regions of interest in the tissue regions 62 are an example of the second regions of interest in this embodiment. The region of interest in this embodiment is, for example, the lesion region 511.

[0179] More specifically, the regions of interest in the tissue regions 62 are ranges in the tissue regions 62 on the tissue image 61 that correspond to the range between the intersections of the cutout line 53 and the lesion region 511 on the specimen image 51 .

[0180] Fig. 24 is a diagram showing an example of the correspondence between the cutout line 53i in the sample image 51 and the tissue region 62i on the tissue image 61 according to the fifth embodiment. Fig. 24 illustrates an image 55e that is a part of the sample image 51 and an image 65f that is a part of the tissue image 61.

[0181] In FIG. 24, the length d11 of the cut-out position indicated by the cut-out line 53i and the length d12 of the tissue region 62i correspond linearly.

[0182] Fig. 25 is a diagram showing an example of mapping of a region of interest based on the correspondence between the cut-out position and tissue region 62i shown in Fig. 24. Region of interest correspondence function 162 specifies a range on tissue region 62i that corresponds to the range of lesion region 511 on cut-out line 53i (range of first region of interest) based on the scale relationship between length d11 of the cut-out position and length d12 of tissue region 62i. Region of interest correspondence function 162 maps the specified range to tissue region 62i as the range of a second region of interest.

[0183] The region-of-interest matching function 162 stores position information of the range of the second region of interest on the identified tissue region 62i in the memory circuitry 120. The region-of-interest matching function 162 may also match the position information of the range of the second region of interest on the identified tissue region 62i with the tissue image 61 and transmit the information to an external system such as an LIS or other pathology report system. The processing circuitry 150 may also include a transmission function that executes processing for transmission to the external system, separate from the region-of-interest matching function 162. The transmission function is an example of a transmission unit.

[0184] Returning to FIG. 19 , the display control function 155 of this embodiment has the same functions as those of the first embodiment, and superimposes images representing multiple second regions of interest on the tissue image 61 and displays them on the display 140. For example, as shown in FIG. 25 , the display control function 155 displays an image representing the second region of interest mapped on the tissue region 62 on the tissue image 61 or an image 65 corresponding to a portion of the tissue image 61. Note that in FIG. 25 , the image representing the second region of interest is depicted as opaque for the sake of explanation, but the display control function 155 may overlay and display a semi-transparent image at a position on the tissue image 61 corresponding to the second region of interest. Alternatively, the display control function 155 may represent multiple second regions of interest using a frame surrounding a position on the tissue image 61 corresponding to the second region of interest, or a line positioned near a position on the tissue image 61 corresponding to the second region of interest.

[0185] Furthermore, the display control function 155 causes the sample image 51 and a tissue image 61, on which a plurality of second regions of interest are superimposed, to be displayed in association with each other on the display 140. For example, as shown in Fig. 25, the display control function 155 causes the sample image 51 or a portion of the sample image 51 to be displayed in association with a tissue image 61 or a portion of the tissue image 61, which is an image of a tissue slice 60 cut out from the sample 5 depicted in the sample image 51.

[0186] For example, the display control function 155 may display on the display 140 a tissue image 61 in which multiple second regions of interest are superimposed, and when a selection operation such as a click on a second region of interest is received from the user, the display control function 155 may display on the display 140 a specimen image 51 including an excision position corresponding to the selected second region of interest.

[0187] Furthermore, the display control function 155 may superimpose an image representing the first region of interest on the specimen image 51 in a range corresponding to the first region of interest, and display the image on the display 140.

[0188] 26 is a flowchart showing an example of the flow of the process of mapping a region of interest according to the fifth embodiment. As a premise of the process of this flowchart, it is assumed that a specimen image 51 and a tissue image 61 have been photographed and acquired by the specimen image acquisition function 151 and the tissue image acquisition function 158.

[0189] First, the region of interest recognition function 153 recognizes a region of interest in the specimen image 51 (S401). The region of interest in this flowchart is, for example, a lesion region 511.

[0190] Then, the tissue region recognition function 159 recognizes each of the regions in which the multiple tissue slices 60 are depicted in the tissue image 61 as a tissue region 62 (S402).

[0191] Then, the cut-out position acquisition function 160 acquires a plurality of cut-out positions representing the collection positions of the plurality of tissue slices 60 in the specimen image 51 (S403).

[0192] Then, the cut-out position associating function 161 identifies the correspondence between the plurality of cut-out positions in the specimen region 510 and the plurality of tissue regions 62 in the tissue image 61 (S404).

[0193] The region of interest matching function 162 maps the range of the lesion region 511 at the cut-out position of the specimen image 51 (the range of the first region of interest) to the range of the second region of interest in the tissue region 62 corresponding to the cut-out position on the tissue image 61 (S405).

[0194] Then, the reception function 156 displays the mapping result of S406 on the display 140 (S406). For example, the display control function 155 displays, on the display 140, a specimen image 51 on which an image representing a first region of interest is superimposed and a tissue image 61 on which images representing a plurality of second regions of interest are superimposed, in association with each other. At this point, the processing of this flowchart ends.

[0195] In this way, according to the information processing device 100 of this embodiment, by identifying the positions of multiple second regions of interest in the tissue image 61 that correspond to the first region of interest in the specimen image 51, it is possible to assist a pathologist or the like in identifying regions of interest such as the lesion region 511 in the tissue image 61 when observing the tissue image 61 and making a pathological diagnosis.

[0196] For example, as a comparative example, when a tissue image and a specimen image are simply displayed separately, even if a pathologist or the like tries to refer to the lesion area of ​​the specimen image from which the tissue section was cut out to assist in making a pathological tissue diagnosis, it may be difficult to understand where the lesion area on the specimen image corresponds on the tissue image.

[0197] In contrast, by specifying the positions of a first region of interest in the specimen image 51 and a plurality of second regions of interest corresponding to the first region of interest in the tissue image 61 as in the information processing device 100 of this embodiment, it is possible for a pathologist or the like to easily understand where on the tissue image 61 the lesion region 511 on the specimen image 51 corresponds. Therefore, the information processing device 100 of this embodiment improves the ease with which a pathologist or the like can refer to the specimen image 51 while observing the tissue image 61, and can reduce erroneous diagnosis by helping the pathologist or the like accurately understand the correspondence between the first region of interest and a plurality of second regions of interest.

[0198] Furthermore, according to the information processing device 100 of this embodiment, images representing multiple second regions of interest are superimposed on the tissue image 61 and displayed on the display 140, thereby allowing a pathologist or the like to easily grasp the second regions of interest on the tissue image 61.

[0199] Furthermore, according to the information processing device 100 of this embodiment, the specimen image 51 and the tissue image 61 on which images representing multiple second regions of interest are superimposed are displayed in correspondence with each other on the display 140, thereby enabling a pathologist or the like to easily understand the correspondence between the specimen image 51 and the second regions of interest on the tissue image 61.

[0200] (Modification 1 of the fifth embodiment) In this modified example, the cut-out position acquisition function 160 identifies the cut-out position on the sample image 51 based on, for example, a cut-out line 53 that indicates the cut-out position on the sample image 51.

[0201] More specifically, the cut-out position acquisition function 160 identifies the actual cut-out position, for example, by removing the portion of the cut-out line 53 corresponding to the cut-out position determined by the cut-out position determination function 154 that extends outside the specimen area 510.

[0202] 21, the cutout line 53 may include a portion where the tissue section 60 has not actually been cut out, such as a range outside the specimen region 510. As shown in Fig. 21, the cutout line 53 may be drawn in a manner different from the portion where the tissue section 60 has actually been cut out, such as a dashed line, for a portion of the cutout line 53 where the tissue section 60 has not actually been cut out and is drawn as a guide, but the cutout line 53 may also be drawn in the same display manner regardless of whether it is inside or outside the specimen region 510.

[0203] FIG. 27 is a flowchart showing an example of the flow of the region of interest mapping process according to the first modification of the fifth embodiment.

[0204] First, the region of interest recognition function 153 recognizes a region of interest in the specimen image 51 (S501). In the modified example, the region of interest is, for example, a specimen region 510 and a lesion region 511.

[0205] Then, the tissue region recognition function 159 recognizes each of the regions in which the multiple tissue slices 60 are depicted in the tissue image 61 as a tissue region 62 (S502).

[0206] Then, the cutout position acquisition function 160 acquires a plurality of cutout lines 53 on the sample image 51 (S503). The cutout position acquisition function 160 may acquire the cutout lines by optically reading the cutout lines written on a sheet of paper on which the sample image 51 is printed. Alternatively, the cutout position acquisition function 160 may read the cutout lines 53 determined by the cutout position determination function 154 from the memory circuitry 120.

[0207] The cut-out position acquisition function 160 removes the portions of each cut-out line 53 that extend outside the specimen region 510, and acquires a plurality of actual cut-out lines 533 that correspond to the actual cut-out positions (S504).

[0208] Then, the cut-out position associating function 161 identifies the correspondence between a plurality of actual cut-out lines 533 corresponding to the actual cut-out positions acquired by the cut-out position acquiring function 160 and a plurality of tissue regions 62 in the tissue image 61 (S505).

[0209] The mapping process in S506 and the display process in S507 are the same as the processes in S405 and S406 described with reference to Fig. 26. Here, the process of this flowchart ends.

[0210] In this way, according to the information processing device 100 of this embodiment, even when the actual cut-out position is unknown, the actual cut-out position can be identified from the cut-out line 53 and the specimen region 510.

[0211] (Sixth embodiment) In the sixth embodiment, the user can modify the cutout position and the region of interest.

[0212] 28 is a diagram showing an example of the overall configuration of an information processing system S according to the sixth embodiment. Similar to the first to fifth embodiments, the information processing system S according to this embodiment includes an information processing device 100, a specimen image storage device 201, a tissue image storage device 202, a first image capturing device 401, and a second image capturing device 402.

[0213] The processing circuit 150 of the information processing device 100 of this embodiment includes a specimen image acquisition function 151, a cutout policy acquisition function 152, a region of interest recognition function 153, a cutout position determination function 154, a display control function 155, a reception function 156, a tissue image acquisition function 158, a tissue region recognition function 159, a cutout position acquisition function 160, a cutout position association function 161, a region of interest association function 162, a cutout position correction function 163, and a region of interest correction function 164. The cutout position correction function 163 is an example of a cutout position correction unit. The region of interest correction function 164 is an example of a region of interest correction unit.

[0214] The specimen image acquisition function 151, the cut-out policy acquisition function 152, the region of interest recognition function 153, the cut-out position determination function 154, the display control function 155, the reception function 156, the tissue image acquisition function 158, the tissue region recognition function 159, the cut-out position acquisition function 160, the cut-out position matching function 161, and the region of interest matching function 162 have the same functions as those in the fifth embodiment.

[0215] The cut-out position correction function 163 corrects the cut-out position when the cut-out position determined by the cut-out position determination function 154 or the cut-out position identified by the cut-out position acquisition function 160 differs from the collection position of the tissue slice 60 corresponding to the tissue area 62 in the actual specimen image 51.

[0216] Generally, the cutout line 53 on the specimen image 51 is drawn before the technician or the like actually cuts out the tissue slice 60 from the specimen 5, and therefore may differ from the actual cutout position. In such a case, for example, the cutout position correction function 163 may move or extend the cutout position on the specimen image 51 based on a user's operation to move or extend the cutout line 53. The cutout position correction function 163 stores the corrected cutout position in the memory circuitry 120.

[0217] Furthermore, the region of interest correction function 164 corrects the second region of interest that has been mapped to the tissue region 62 by the region of interest association function 162 .

[0218] More specifically, when receiving a user operation to move or expand / contract an image representing the second region of interest on tissue image 61 displayed on display 140, region of interest correction function 164 may move or expand the second region of interest representing lesion region 511 in tissue image 61 based on the user operation. Region of interest correction function 164 stores the position of the second region of interest after the correction in memory circuitry 120. Note that region of interest correction function 164 may perform correction based on the user operation on not only the second region of interest but also the first region of interest.

[0219] Alternatively, when the region of interest correction function 164 receives a user operation to correct or confirm a second region of interest mapped to the tissue region 62 by the region of interest matching function 162, it may consider the second region of interest to have been diagnosed and transmit the tissue image 61 including the second region of interest to an external system such as a pathology report system.

[0220] FIG. 29 is a flowchart showing an example of the flow of the region of interest mapping process according to the sixth embodiment.

[0221] The process from the recognition process of the region of interest in S601 to the acquisition of the group of cutout positions in S603 is the same as the process from S401 to S403 described with reference to FIG.

[0222] Then, the cut-out position correction function 163 corrects the cut-out position determined by the cut-out position determination function 154 based on the user's operation to match the actual situation (S604).

[0223] The process from S605 for associating the cutout position with the tissue region 62 to the display process in S607 is the same as the process from S404 to S406 described with reference to Fig. 26. Here, the process of this flowchart ends.

[0224] Although the region of interest correction process is omitted in FIG. 29, the region of interest correction process may be executed by the region of interest correction function 164 after S607.

[0225] (Seventh embodiment) In each of the above-described embodiments, the tissue slice 60 is imaged with its cross section facing the imaging device, but the tissue slice 60 may also be imaged in the same orientation as the specimen 5 when the specimen image 51 is captured.

[0226] Fig. 30 is a diagram showing an example of the cut-out position according to the seventh embodiment. When the specimen surface in the specimen image 51 and the thin-section surface of the tissue slice 60 face in the same direction, the cut-out position is represented by a cut-out rectangle 54 rather than a cut-out line 53, as shown in Fig. 30. The cut-out rectangle 54 is shaped so as not to extend beyond the specimen region 510.

[0227] The cut-out position acquisition function 160 acquires a plurality of cut-out regions based on the cut-out rectangle 54 indicating the cut-out position and the specimen region 510. The cut-out position association function 161 identifies the correspondence between the plurality of cut-out regions acquired by the cut-out position acquisition function 160 and a plurality of tissue regions 62. Furthermore, when aligning the cut-out regions with the corresponding tissue regions 62, the cut-out position association function 161 may deform the cut-out regions to match the shape of the tissue regions 62.

[0228] 31 is a diagram showing an example of mapping of a region of interest according to the seventh embodiment. The region of interest associating function 162 may apply a deformation field obtained when the cut-out region is aligned with the corresponding tissue region 62 to a first region of interest (e.g., lesion region 511) in the specimen region 510, and map a second region of interest onto the tissue image 61. In this case, the region of interest associating function 162 may remove any region of the mapped second region of interest that extends outside the tissue region 62.

[0229] The various data handled in this specification are typically digital data.

[0230] According to at least one of the embodiments described above, the position at which the tissue slice is to be cut from the specimen can be appropriately determined.

[0231] Although several embodiments have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These embodiments can be implemented in various other forms, and various omissions, substitutions, modifications, and combinations of embodiments can be made without departing from the spirit of the invention. These embodiments and their modifications are included within the scope and spirit of the invention, as well as within the scope of the invention and its equivalents as defined in the claims. [Explanation of symbols]

[0232] 5 specimens 6 Tissue specimen 7. Glass slides 51 Sample images 53, 53a~53i Cutting lines 60,60a~60e tissue section 61 Tissue Images 62,62a~62j Organization area 100 Information processing device 110 Network Interface 120 Memory circuit 130 Input Interface 140 Display 150 Processing Circuit 151 Sample image acquisition function 152 Cutting policy acquisition function 153 Region of interest recognition function 154 Cutout position determination function 155 Display control function 156 Reception Function 157 Division position candidate determination function 158 Tissue image acquisition function 159 Organizational area recognition function 160 Cutout position acquisition function 161 Cutout position matching function 162 Region of interest matching function 163 Cutout position correction function 164 Region of interest correction function 201 Sample Image Storage Device 202 Tissue Image Storage Device 300 Network 401 First imaging device 402 Secondary Camera 510 Specimen Area 511 Lesion area 590 Split candidate position image S Information Processing System

Claims

1. a specimen image acquisition unit that acquires a specimen image obtained by photographing a pathological specimen; a cutout policy acquisition unit that acquires a cutout policy that defines rules for cutting out tissue slices from a pathological specimen; a region of interest recognition unit that recognizes a first region of interest from the sample image; an excision position determination unit that determines an excision position of the tissue slice in the specimen image based on the first region of interest and the excision policy; Equipped with the cut-out policy defines at least that the cut-out position is set so as to be perpendicular to the major axis of the pathological specimen; the cut-out policy acquisition unit acquires the cut-out policy by one of the following methods: accepting a user operation to input the cut-out policy; reading the cut-out policy from a storage unit; and acquiring the cut-out policy from an external device. Information processing device.

2. The cut-out policy further defines that a cut-out position of the pathological specimen is temporarily set, and the temporarily set cut-out position is rotated to set a cut-out position that satisfies a condition. The information processing device according to claim 1 .

3. The cutting policy further specifies that each tissue section cut from the pathology specimen is to be larger than a specified size. The information processing device according to claim 2 .

4. The segmentation policy further defines that the segmentation should be performed so as to include the largest possible lesion area within a specified number of segmentations; The lesion area is an area in the specimen image where a lesion site of the pathological specimen is depicted. The information processing device according to claim 2 .

5. The segmentation policy further defines that the segmentation is performed so that the number of tissue sections including the lesion area is maximized within a specified interval; The lesion area is an area in the specimen image where a lesion site of the pathological specimen is depicted. The information processing device according to claim 2 .

6. When the cut-out policy includes a description regarding any region in the sample image, the region of interest recognition unit recognizes the region described in the cut-out policy as the first region of interest from the sample image. The information processing device according to claim 1 .

7. a division position candidate determination unit that determines division position candidates for dividing the tissue slice into a plurality of pieces when the size of the tissue slice cut out at the cut-out position determined by the cut-out position determination unit is equal to or larger than a threshold value; The information processing device according to claim 1 .

8. a display control unit that displays the sample image on a display unit with an excision line representing the excision position superimposed thereon, The information processing device according to claim 1 .

9. the specimen image is a two-dimensional image, the region of interest recognition unit recognizes the first region of interest based on features of the specimen surface depicted in the specimen image; The information processing device according to claim 1 .

10. the specimen image is a three-dimensional image, the region of interest recognition unit recognizes the first region of interest based on features of the surface and interior of the specimen depicted in the specimen image; The information processing device according to claim 1 .

11. a tissue image acquisition unit for acquiring tissue images of a plurality of tissue slices cut out from the pathological specimen; a tissue region recognition unit that recognizes regions in which the plurality of tissue slices are depicted in the tissue image as a plurality of tissue regions; a cutout position acquisition unit that acquires a plurality of cutout positions representing collection positions of the plurality of tissue slices in the sample image; an excision position associating unit that identifies a correspondence between the plurality of excision positions and the plurality of tissue regions; a region-of-interest matching unit that identifies positions of a plurality of second regions of interest corresponding to the first regions of interest in the tissue image in the plurality of tissue regions based on the first regions of interest in the sample image and the correspondence relationship, The information processing device according to claim 1 .

12. the cut-out position acquisition unit specifies a cut-out position in the sample image based on a cut-out line representing the cut-out position; The information processing device according to claim 11.

13. a cut-out position correcting unit that corrects the cut-out position determined by the cut-out position determining unit or the cut-out position acquired by the cut-out position acquiring unit when the cut-out position is different from the actual collection position of the tissue slice corresponding to the tissue region in the sample image; 13. The information processing device according to claim 11 or 12.

14. the cutout position associating unit specifies a correspondence relationship between the plurality of cutout positions and the plurality of tissue regions in accordance with a predetermined operation rule; The information processing device according to any one of claims 11 to 13.

15. the cut-out position associating unit specifies a correspondence relationship between the plurality of cut-out positions and the plurality of tissue regions based on an operation by a user; The information processing device according to any one of claims 11 to 13.

16. a display control unit that causes a display unit to display images representing the plurality of second regions of interest superimposed on the tissue image, The information processing device according to any one of claims 11 to 13.

17. the display control unit causes the display unit to display the specimen image and the tissue image on which the images representing the second regions of interest are superimposed in association with each other. The information processing device according to claim 16.

18. the display control unit causes the display unit to superimpose an image representing the first region of interest on the sample image; The information processing device according to claim 17.

19. when a user's operation on the second region of interest displayed on the display unit is accepted, the sample image including the cut-out position corresponding to the second region of interest is displayed on the display unit.

19. The information processing device according to claim 17 or 18.

20. a region of interest correction unit that corrects the second region of interest based on a user operation; 20. The information processing device according to claim 11.

21. A specimen image acquisition step in which a processing circuit acquires a specimen image obtained by photographing a pathological specimen; a cutting policy acquisition step in which the processing circuit acquires a cutting policy that defines rules for cutting tissue slices from a pathological specimen; a region of interest recognition step in which the processing circuit recognizes a first region of interest from the specimen image; an extraction position determination step in which the processing circuit determines an extraction position of the tissue slice in the specimen image based on the first region of interest and the extraction policy; Including, the cut-out policy defines at least that the cut-out position is set so as to be perpendicular to the major axis of the pathological specimen; In the cut-out policy acquisition step, the processing circuit acquires the cut-out policy by one of the following methods: accepting a user operation to input the cut-out policy; reading the cut-out policy from a storage unit; and acquiring the cut-out policy from an external device. Information processing methods.

22. a sample image acquisition step of acquiring a sample image obtained by photographing a pathological sample; a cutout policy acquisition step of acquiring a cutout policy that defines rules for cutting out tissue slices from a pathological specimen; a region of interest recognition step of recognizing a first region of interest from the sample image; an extraction position determination step of determining an extraction position of the tissue slice in the specimen image based on the first region of interest and the extraction policy; on the computer, the cut-out policy defines at least that the cut-out position is set so as to be perpendicular to the major axis of the pathological specimen; In the cut-out policy acquisition step, the cut-out policy is acquired by any one of accepting a user's operation to input the cut-out policy, reading the cut-out policy from a storage unit, and acquiring the cut-out policy from an external device. program.

Images