Information processing device, method for operating information processing device, and program for operating information processing device

Abstract

This information processing device is provided with a processor, in which the processor acquires an organ image in which an organ of a subject is imaged, detects an abnormal part in which it is estimated that an abnormality occurs from the organ image, aligns a reference image including positional information about a plurality of anatomic parts of the organ with the organ image to define the anatomic parts in the organ image, and outputs degree information that shows the degree of occurrence of the abnormal part in each of the plurality of anatomic parts.

Description

Information processing device, operating method for information processing device, and operating program for information processing device

[0001] The technology of the present disclosure relates to an information processing device, an operating method for an information processing device, and an operating program for an information processing device.

[0002] In the medical field, various evaluations are performed based on organ images of a subject's organs. For example, in a test in which a candidate drug substance is administered to a subject, such as a rat, to evaluate the efficacy and toxicity of the candidate substance, slice images of tissue slices (brain slices, heart slices, etc.) collected by autopsy of the subject are used as organ images. Traditionally, evaluations were performed by experts, such as pathologists, observing organ images. However, recent advances in image analysis technology have led to the development of techniques that allow automatic evaluation without the need for experts. For example, JP 2020-533725 A describes a technique in which a machine learning model is applied to multiple patch images obtained by dividing a slice image, and whether or not an abnormality, such as cancer, has occurred in the tissue slices depicted in the patch images.

[0003] Some organs have multiple anatomical regions. For example, the brain has anatomical regions such as the cerebral cortex, hippocampus, thalamus, and amygdala. In evaluations using organ images, there is a need to understand the extent of abnormalities in each of these anatomical regions. However, while the technology described in JP 2020-533725 A can determine whether or not an abnormality has occurred in a tissue slice captured in a patch image, it has been difficult to determine the extent of abnormalities in each anatomical region.

[0004] One embodiment of the technology of the present disclosure provides an information processing device, an operating method of the information processing device, and an operating program of the information processing device that can easily grasp the degree of abnormality occurring in each anatomical part of an organ.

[0005] The information processing device of the present disclosure includes a processor that acquires organ images of the subject's organs, detects abnormal parts from the organ images where abnormalities are presumed to have occurred, defines anatomical parts in the organ image by aligning the organ image with a reference image having positional information of multiple anatomical parts of the organ, and outputs degree information that indicates the degree of occurrence of abnormal parts in each of the multiple anatomical parts.

[0006] Preferably, the processor extracts features of a region of interest in the organ image and determines whether the region of interest is an abnormal part based on the difference between the features extracted from the region of interest and a reference feature.

[0007] The reference feature is preferably a feature extracted from a region surrounding the region of interest.

[0008] The organ images are images used in evaluation tests to evaluate drug candidate substances, and the reference feature amounts are preferably feature amounts extracted from organ images of subjects to whom the candidate substances have not been administered.

[0009] The processor preferably controls the display to superimpose the defined anatomical region on the organ image.

[0010] The processor preferably accepts instructions to modify the defined anatomical region.

[0011] The processor preferably outputs the degree information generated based on the anatomical region modified in response to the modification instruction.

[0012] The extent information preferably includes the number of abnormal portions for each of the plurality of anatomical regions and the area ratio of the abnormal portions for each of the plurality of anatomical regions.

[0013] The organ image is preferably a slice image of a tissue slice of the organ.

[0014] The method of operating the information processing device disclosed herein includes obtaining an organ image of a subject's organ, detecting abnormal parts from the organ image where abnormalities are presumed to have occurred, defining anatomical parts in the organ image by aligning the organ image with a reference image having positional information of multiple anatomical parts of the organ, and outputting degree information indicating the degree of occurrence of abnormal parts in each of the multiple anatomical parts.

[0015] The operating program of the information processing device disclosed herein causes a computer to perform processes including acquiring organ images of the subject's organs, detecting abnormal parts from the organ images where abnormalities are presumed to have occurred, defining anatomical parts in the organ images by aligning the organ images with a reference image having positional information of multiple anatomical parts of the organ, and outputting degree information indicating the degree of occurrence of abnormal parts in each of the multiple anatomical parts.

[0016] According to the technology of the present disclosure, it is possible to provide an information processing device, an operating method of the information processing device, and an operating program of the information processing device that can easily grasp the degree of abnormality occurring in each anatomical part of an organ.

[0017] 1 is a diagram illustrating the steps of an evaluation test, a section image, and a drug discovery support device. FIG. 2 is a diagram illustrating an administration group and a control group. FIG. 3 is a block diagram illustrating a computer constituting the drug discovery support device. FIG. 4 is a block diagram illustrating a processing unit of a CPU of the drug discovery support device. FIG. 5 is a diagram illustrating a patch image obtained by dividing a section image. FIG. 6 is a diagram illustrating how features are extracted from a patch image by a feature extractor. FIG. 7 is a diagram illustrating the structure of a feature extractor. FIG. 8 is a diagram illustrating processing in the learning phase of an autoencoder. FIG. 9 is a diagram illustrating processing by a detection unit. FIG. 10 is a diagram illustrating detection results. FIG. 11 is a diagram illustrating a reference image. FIG. 12 is a diagram illustrating processing by a demarcation unit. FIG. 13 is a diagram illustrating demarcation results. FIG. 14 is a diagram illustrating degree information. FIG. 15 is a drug discovery support screen displaying a section image. FIG. 16 is a drug discovery support screen displaying degree information. FIG. 17 is a flowchart illustrating the processing procedure of the drug discovery support device. FIG. 18 is a diagram illustrating another example of degree information displayed on a drug discovery support screen. FIG. 19 is a diagram illustrating processing by a detection unit of a second embodiment. FIG. 19 is a diagram illustrating a drug discovery support screen displaying a demarcated anatomical site superimposed on a section image. FIG. 19 is a diagram illustrating a drug discovery support screen for accepting an instruction to correct an anatomical site. FIG. 10 is a diagram showing a third embodiment in which extent information is generated based on an anatomical portion corrected in accordance with a correction instruction.

[0018] First Embodiment As shown in FIG. 1 , a drug discovery support device 10 is an example of an "information processing device" according to the technology of the present disclosure, and is used to evaluate the efficacy and toxicity of a drug candidate substance 26 (see FIG. 2 ). The drug discovery support device 10 is, for example, a desktop personal computer, and includes a display 11 that displays various screens, and an input device 12 that includes a keyboard, a mouse, a touch panel, and / or a microphone for voice input. The drug discovery support device 10 is operated by an operator, such as a staff member at a pharmaceutical facility involved in drug development. The display 11 is an example of a "display" according to the technology of the present disclosure.

[0019] A slice image 15 is input to the drug discovery support device 10. The slice image 15 is generated, for example, by the following procedure. First, a subject S, such as a rat, prepared for evaluation of a candidate substance 26 is autopsied, and multiple tissue slices (hereinafter referred to as brain slices) BS of a coronal section of an organ, in this case, the brain B, of the subject S are collected. Next, the collected brain slices BS are attached one by one to glass slides 16, and then stained, in this case, with hematoxylin and eosin dye. Next, the stained brain slices BS are covered with a cover glass 17 to complete a slide specimen 18. The slide specimen 18 is then placed in an imaging device 19, such as a digital optical microscope, and the slice image 15 is captured by the imaging device 19. The slice image 15 thus obtained is assigned a subject ID (Identification Data) for uniquely identifying the subject S, an image ID for uniquely identifying the slice image 15, the slice position of the coronal section, the date and time of imaging, etc. The slice image 15 is an example of an "organ image" and an "image used in an evaluation test for evaluating a drug candidate substance" according to the technology of the present disclosure. Note that the staining may be with hematoxylin dye alone, nuclear fast red dye, or the like.

[0020] As an example, as shown in FIG. 2 , the slice images 15 input to the drug discovery support device 10 include a first slice image 151 and a second slice image 152. The first slice image 151 is an image of a brain slice BS of a subject S in the administration group 25. The administration group 25 is composed of multiple subjects S to whom a drug candidate substance 26 has been administered. The number of subjects S constituting the administration group 25 is, for example, approximately 5 to 10. Since multiple first slice images 151 are obtained from one subject S, the number of first slice images 151 obtained from the administration group 25 is calculated by multiplying the number obtained from one subject S by the number of subjects S. For example, if the number obtained from one subject S is 10 and the number of subjects S constituting the administration group 25 is 10, 10 × 10 = 100 first slice images 151 are obtained.

[0021] The second slice image 152 is an image of a brain slice BS of a subject S in the control group 27. Unlike the treatment group 25, the control group 27 is composed of multiple subjects S who were not administered the drug candidate substance 26. The subjects S in the control group 27 have the same attributes as the subjects S in the treatment group 25 and are kept in the same rearing environment. The same attributes include, for example, the same age in weeks and / or the same gender. The same attributes also include the same age composition ratio and / or the same gender composition ratio (e.g., five males and five females). The same rearing environment includes, for example, the same food, the same temperature and humidity of the rearing space, and / or the same size of the rearing space. The term "same" in the term "same rearing environment" refers not only to "completely the same" but also to "same" in the sense of including a margin of error generally acceptable in the technical field to which the technology of the present disclosure pertains and that does not contradict the spirit of the technology of the present disclosure. The number of subjects S constituting the control group 27 is also, for example, about 5 to 10, similar to the administration group 25. Like the first slice images 151, a plurality of second slice images 152 are obtained from one subject S, and therefore the number of second slice images 152 obtained from the control group 27 is calculated by multiplying the number obtained from one subject S by the number of subjects S. In the following description, unless there is a particular need to distinguish between them, the first slice images 151 and the second slice images 152 will be collectively referred to as slice images 15.

[0022] The administration group 25 includes a plurality of groups with different doses of the candidate substance 26. For example, the dose of the candidate substance 26 is differentiated into three levels: a high dose group, a medium dose group, and a low dose group. This allows the influence of the dose of the candidate substance 26 on the subject S to be ascertained.

[0023] 3, the computer constituting the drug discovery support apparatus 10 includes, in addition to the display 11 and input device 12, a storage 30, a memory 31, a CPU (Central Processing Unit) 32, and a communication unit 33. These are interconnected via a bus line 34.

[0024] The storage 30 is a hard disk drive built into the computer constituting the drug discovery support apparatus 10 or connected via a cable or network. Alternatively, the storage 30 is a disk array with multiple hard disk drives connected in series. The storage 30 stores control programs such as an operating system, various application programs, and various data associated with these programs. Note that a solid state drive may be used instead of a hard disk drive.

[0025] The memory 31 is a work memory for the CPU 32 to execute processing. The CPU 32 loads programs stored in the storage 30 into the memory 31 and executes processing in accordance with the programs. In this way, the CPU 32 comprehensively controls each part of the computer. The CPU 32 is an example of a "processor" according to the technology of the present disclosure. Note that the memory 31 may be built into the CPU 32. The communication unit 33 controls the transmission of various information to and from external devices such as the imaging device 19.

[0026] As an example, as shown in Figure 4, an operating program 40 is stored in the storage 30 of the drug discovery support apparatus 10. The operating program 40 is an application program for causing a computer to function as the drug discovery support apparatus 10. In other words, the operating program 40 is an example of an "operating program for an information processing device" according to the technology of the present disclosure. The storage 30 also stores a feature extractor 41, setting range information 42, a reference image 43, and the like.

[0027] When the operating program 40 is started, the CPU 32 of the computer constituting the drug discovery support device 10 works in cooperation with the memory 31 and the like to function as a read / write (hereinafter abbreviated as RW (Read Write)) control unit 50, a detection unit 51, a definition unit 52, a generation unit 53, and a display control unit 54.

[0028] The RW control unit 50 controls the storage of various data in the storage 30 and the reading of various data from the storage 30. For example, the RW control unit 50 stores the slice image 15 from the imaging device 19 in the storage 30.

[0029] The RW control unit 50 acquires the slice images 15 by reading them from the storage 30. The RW control unit 50 outputs the slice images 15 to the detection unit 51, the demarcation unit 52, and the display control unit 54. The slice images 15 output from the RW control unit 50 to the detection unit 51, etc., are targets for determining the degree of occurrence of abnormal portions in the brain slice BS. Hereinafter, the slice images 15 for determining the degree of occurrence of abnormal portions in the brain slice BS will be referred to as target slice images 15T (see FIG. 5, etc.). The target slice image 15T is, for example, all of the multiple slice images 15 obtained from one subject S among the multiple subjects S that make up the administration group 25.

[0030] The RW control unit 50 reads the feature extractor 41 and the setting range information 42 from the storage 30, and outputs the feature extractor 41 and the setting range information 42 to the detection unit 51. The RW control unit 50 also reads the reference image 43 from the storage 30, and outputs the reference image 43 to the demarcation unit 52.

[0031] The detection unit 51 uses the feature extractor 41 to detect abnormal portions in the target slice image 15T where abnormalities are presumed to have occurred. An abnormal portion is a portion where a lesion not seen in a normal brain slice BS, such as a tumor, inflammation, cyst, or infiltration, has occurred. The detection unit 51 outputs the detection result 60 of the abnormal portion to the generation unit 53.

[0032] The demarcation unit 52 aligns the reference image 43 with the target slice image 15T to demarcate a plurality of anatomical regions of the brain B in the target slice image 15T. The demarcation unit 52 outputs a demarcation result 61 of the anatomical regions of the brain B to the generation unit 53. Note that the demarcation unit 52 performs a process of demarcating the anatomical regions in parallel with the process of detecting abnormal portions by the detection unit 51.

[0033] Based on the detection result 60 and the demarcation result 61, the generation unit 53 generates degree information 62 that indicates the degree of occurrence of abnormal portions in each of the multiple anatomical regions of the brain B. The generation unit 53 outputs the degree information 62 to the display control unit 54.

[0034] The display control unit 54 controls the display of various screens on the display 11. The various screens include a drug discovery support screen 100A (see FIG. 15) that displays a slice image 15, and a drug discovery support screen 100B (see FIG. 16) that displays degree information 62. In addition to these processing units 50 to 54, the CPU 32 also includes an instruction receiving unit 120 (see FIG. 23) that receives various operation instructions from the input device 12.

[0035] As an example, as shown in Fig. 5 , the detection unit 51 uses a well-known image recognition technique to recognize a brain slice BS that appears in the target slice image 15T, and subdivides the recognized brain slice BS into a plurality of patch images 65. The patch images 65 have a preset size that can be handled by the feature extractor 41. Note that, although in Fig. 5 , the patch images 65 do not have any areas that overlap with other patch images 65, the patch images 65 may partially overlap with other patch images 65.

[0036] As an example, as shown in FIG. 6, the detection unit 51 causes the feature extractor 41 to extract a feature 68 for each of a plurality of patch images 65 obtained by dividing the target slice image 15T.

[0037] As an example, as shown in FIG. 7 , the feature extractor 41 is configured by reusing an encoder unit 71 of an autoencoder 70. The autoencoder 70 includes a decoder unit 72 in addition to the encoder unit 71. A patch image 65 is input to the encoder unit 71. The encoder unit 71 converts the patch image 65 into a feature 68. The encoder unit 71 passes the feature 68 to the decoder unit 72. The decoder unit 72 generates a restored image 73 of the patch image 65 from the feature 68.

[0038] As is well known, the encoder unit 71 includes a convolution layer that performs convolution processing using a filter, a pooling layer that performs pooling processing such as maximum value pooling, and the like. The decoder unit 72 is similar. The encoder unit 71 extracts features 68 by repeatedly performing convolution processing using the convolution layer and pooling processing using the pooling layer on the input patch image 65. The extracted features 68 represent the shape and texture characteristics of the brain slice BS captured in the patch image 65. The features 68 are a collection of multiple numerical values. In other words, the features 68 are multidimensional data. The number of dimensions of the features 68 is, for example, 512, 1024, or 2048.

[0039] As an example, as shown in FIG. 8 , during a learning phase before the encoder unit 71 is converted into the feature extractor 41, the autoencoder 70 receives training patch images 65L and performs training. The autoencoder 70 outputs training reconstructed images 73L for the training patch images 65L. A loss calculation is performed on the autoencoder 70 using a loss function based on the training patch images 65L and the training reconstructed images 73L. Various coefficients (e.g., filter coefficients of the convolutional layer) of the autoencoder 70 are updated based on the results of the loss calculation, and the autoencoder 70 is updated according to the update settings. Here, the training patch images 65L are supplied from a plurality of patch images 65 obtained by subdividing a brain slice BS depicted in a slice image 15 obtained in an evaluation test of a candidate substance for a drug developed in the past.

[0040] During the learning phase of the autoencoder 70, the above-described series of processes, including input of the training patch images 65L to the autoencoder 70, output of the training restored images 73L from the autoencoder 70, loss calculation, update setting, and updating of the autoencoder 70, are repeatedly performed while the training patch images 65L are exchanged. The repetition of the above-described series of processes is terminated when the accuracy of restoration from the training patch images 65L to the training restored images 73L reaches a predetermined set level. The encoder unit 71 of the autoencoder 70 whose restoration accuracy has thus reached the set level is stored in the storage 30 of the drug discovery support apparatus 10 as the feature extractor 41. Note that learning may be terminated when the above-described series of processes has been repeated a set number of times, regardless of the accuracy of restoration from the training patch images 65L to the training restored images 73L.

[0041] The learning of the autoencoder 70 may be performed by the drug discovery support device 10, or may be performed by a device separate from the drug discovery support device 10. In the latter case, the feature extractor 41 is transmitted from the separate device to the drug discovery support device 10, and the feature extractor 41 is stored in the storage 30 by the RW control unit 50.

[0042] As shown in FIG. 9 as an example, the detection unit 51 determines whether the region of interest 80 is an abnormal area based on the difference Δ between the feature of interest 68IN extracted from the region of interest 80 and the surrounding feature 68S extracted from the surrounding region 81. The region of interest 80 is the area to be determined as an abnormal area. The region of interest 80 is, for example, an area of ​​one or more (four, nine, sixteen, etc.) patch images 65. The surrounding region 81 is the area surrounding the region of interest 80. The detection unit 51 moves the region of interest 80 one by one and makes a determination each time. The detection unit 51 also changes the size of the region of interest 80 (the number of patch images 65 constituting the region of interest 80) in several stages to make the determination. In FIG. 9, the surrounding region 81 is eight, but if the region of interest 80 is located near the edge of the brain slice BS, the surrounding region 81 will be less than eight. The surrounding feature amount 68S is a representative value, for example, an average value, of the feature amounts 68 extracted from the plurality of surrounding regions 81. The surrounding feature amount 68S is an example of a "reference feature amount" according to the technology of the present disclosure.

[0043] The detection unit 51 compares the difference Δ between the feature of interest 68IN and the surrounding feature 68S with the set range of the set range information 42. The set range information 42 includes a lower limit value and an upper limit value that indicate the set range. If the difference Δ is within the set range (lower limit value≦difference Δ≦upper limit value), the detection unit 51 outputs a determination result 82 that the region of interest 80 is not an abnormal portion. On the other hand, if the difference Δ is outside the set range (difference Δ<lower limit value, or difference Δ>upper limit value), the detection unit 51 outputs a determination result 82 that the region of interest 80 is an abnormal portion.

[0044] 10 , the detection result 60 is information in which a determination result 82 is registered for each position coordinate of a patch image 65. The position coordinate of the patch image 65 is, for example, the position coordinate of a diagonal point of the patch image 65.

[0045] As an example, as shown in FIG. 11 , the reference image 43 is an image containing position information for multiple anatomical regions of the brain B. More specifically, the reference image 43 is an image in which pixel values ​​representing anatomical regions (e.g., 0 for the cerebral cortex, 1 for the hippocampus, etc.) are registered for each corresponding pixel. The reference image 43 is called an atlas. In FIG. 11 , as shown in legend 85, the cerebral cortex, hippocampus, thalamus, hypothalamus, and amygdala are illustrated as anatomical regions. Other anatomical regions include the cerebellum and cranial nerves (see FIG. 14 ). A reference image 43 is prepared for each coronal slice position of the brain slice BS. The demarcation unit 52 aligns the target slice image 15T at the same slice position with the reference image 43. The reference image 43 can be publicly available data, such as the Allen Brain Atlas provided by the Allen Institute for Brain Science in Seattle, USA. The reference image 43 is not limited to publicly available data; it can also be an image independently created by a pharmaceutical facility.

[0046] 12 as an example, the demarcation unit 52 first performs grayscale processing 90 on the target slice image 15T to highlight the contour 93 of the brain slice BS, thereby converting the target slice image 15T into a grayscale image 91. Next, the demarcation unit 52 performs registration processing 92 to align the grayscale image 91 with the reference image 43. Note that the grayscale processing 90 is not necessarily performed.

[0047] The registration process 92 is a non-rigid registration process, and is performed, for example, by the following procedure. Specifically, multiple control points CP are set on the contour 93 of the brain slice BS in the grayscale image 91. Each control point CP is then assigned to an equivalent position on the contour 94 of the brain slice BS in the reference image 43. The demarcation unit 52 derives the amount of deformation of each pixel of the reference image 43 when aligning the target section image 15T and the reference image 43 from the amount of movement of each control point CP of the contour 94 relative to each control point CP of the contour 93, for example, by thin plate spline transformation. The demarcation unit 52 deforms the reference image 43 using the derived amount of deformation, thereby aligning the reference image 43 with the target section image 15T. In FIG. 12 , the distortion of the grid points of the reference image 43 resulting from the deformation for alignment with the target section image 15T is represented by thin lines. Instead of the non-rigid registration process using the thin plate spline transformation, for example, non-rigid registration process using a B-spline curve or non-rigid registration process taking into account the brightness information of the target slice image 15T may be performed.

[0048] As an example, as shown in FIG. 13, the demarcation result 61 is information in which an anatomical part is registered for each position coordinate of the target slice image 15T.

[0049] 14 , the extent information 62 includes the number of abnormal portions for each of a plurality of anatomical regions and the area ratio of the abnormal portions for each of the plurality of anatomical regions. The number and area ratio of the abnormal portions are values ​​derived for all of a plurality of slice images 15 obtained from a single subject S. The area ratio of the abnormal portions is the ratio of the number of pixels in the abnormal portions to the number of pixels in each anatomical region in all of a plurality of slice images 15 obtained from a single subject S. For example, if the number of pixels defined as the cerebral cortex by the defining unit 52 is 10,000 and the number of pixels in the region of interest 80 detected as an abnormal portion by the detecting unit 51 is 500 in all of a plurality of slice images 15 obtained from a single subject S, the area ratio is (500 / 10,000)×100=5%.

[0050] 15 as an example, the display control unit 54 controls the display of a drug discovery support screen 100A on the display 11. The display control unit 54 displays the drug discovery support screen 100A when an operator issues an instruction to display a slice image 15 via the input device 12 and the instruction to display is accepted by the instruction accepting unit 120. The drug discovery support screen 100A is provided with a pull-down menu 101 for selecting either the administration group 25 or the control group 27, and a pull-down menu 102 for selecting the subject ID of the subject S. FIG. 15 shows an example in which the administration group 25 is selected in the pull-down menu 101, and the subject ID "R001" is selected in the pull-down menu 102.

[0051] The drug discovery support screen 100A displays a plurality of slice images 15 arranged side by side. These slice images 15 are all of the slice images 15 obtained from one subject S. Fig. 15 shows an example in which ten first slice images 151 with image IDs "SP001" to "SP010" obtained from the subject S with subject ID "R001" in the administration group 25 are displayed side by side.

[0052] An abnormality determination button 103 is provided at the bottom of the drug discovery support screen 100A. When the operator selects the abnormality determination button 103, an abnormality determination instruction is received by the instruction receiving unit 120. The abnormality determination instruction is an instruction to cause the detection unit 51, the demarcation unit 52, and the generation unit 53 to detect abnormal portions from the slice images 15 displayed on the drug discovery support screen 100A, demarcate anatomical sites in the slice images 15 displayed on the drug discovery support screen 100A, and generate extent information 62 for all of the slice images 15 displayed on the drug discovery support screen 100A. In other words, the slice image 15 displayed on the drug discovery support screen 100A (here, the first slice image 151 obtained from the subject S with the subject ID "R001" in the administration group 25) corresponds to the target slice image 15T.

[0053] When the abnormality determination button 103 is selected, the abnormal portion is detected, the anatomical site is defined, and the degree information 62 is generated, the display control unit 54 pops up a drug discovery support screen 100B on the drug discovery support screen 100A, as shown in FIG. 16 as an example. The degree information 62 is displayed on the drug discovery support screen 100B. The drug discovery support screen 100B disappears when the OK button 105 is selected.

[0054] Next, the operation of the above configuration will be described with reference to the flowchart shown in Fig. 17. First, when the operating program 40 is started in the drug discovery support device 10, the CPU 32 of the drug discovery support device 10 functions as an RW control unit 50, a detection unit 51, a demarcation unit 52, a generation unit 53, and a display control unit 54, as shown in Fig. 4.

[0055] First, the imaging device 19 captures a slice image 15 of a brain slice BS of the subject S. The slice image 15 is output from the imaging device 19 to the drug discovery support device 10. In the drug discovery support device 10, the slice image 15 from the imaging device 19 is stored in the storage 30 by the RW control unit 50 (step ST100).

[0056] When the operator issues an instruction to display the slice image 15 via the input device 12 (YES in step ST110), the RW control unit 50 reads the slice image 15 specified in the display instruction from the storage 30 (step ST120). The slice image 15 is output from the RW control unit 50 to the display control unit 54. As shown in Fig. 15, the slice image 15 is displayed on the display 11 via the drug discovery support screen 100A under the control of the display control unit 54 (step ST130).

[0057] When the operator selects the abnormality determination button 103 on the drug discovery support screen 100A and issues an abnormality determination instruction (YES in step ST140), the RW control unit 50 reads out and acquires the slice image 15 currently displayed on the drug discovery support screen 100A from the storage 30 as a target slice image 15T (step ST150). The target slice image 15T is output from the RW control unit 50 to the detection unit 51 and the demarcation unit 52.

[0058] The RW control unit 50 reads the feature extractor 41 and the setting range information 42 from the storage 30 and outputs them to the detection unit 51. The RW control unit 50 also reads the reference image 43 from the storage 30 and outputs it to the demarcation unit 52.

[0059] 5, the detection unit 51 subdivides the target slice image 15T into a plurality of patch images 65 (step ST160). Subsequently, as shown in FIG. 6, the detection unit 51 causes the feature extractor 41 to extract features 68 from the patch images 65 (step ST170).

[0060] 9, the detection unit 51 calculates a difference Δ between an interest feature 68IN of the region of interest 80 and a surrounding feature 68S of the surrounding region 81. Then, the difference Δ is compared with the set range of the set range information 42, and a determination result 82 indicating whether the region of interest 80 is an abnormal portion is output (step ST180). Based on this determination result 82, the detection result 60 shown in FIG. 10 is generated. The detection result 60 is output from the detection unit 51 to the generation unit 53.

[0061] 12, the demarcation unit 52 aligns the target slice image 15T and the reference image 43 at the same slice position, thereby demarcating a plurality of anatomical regions of the brain B in the target slice image 15T (step ST190). Then, the demarcation result 61 shown in FIG. 13 is generated. The demarcation result 61 is output from the demarcation unit 52 to the generation unit 53.

[0062] 14 based on the detection result 60 and the demarcation result 61, the generation unit 53 generates degree information 62 indicating the degree of occurrence of abnormal parts for each of the plurality of anatomical regions (step ST200). The degree information 62 is output from the generation unit 53 to the display control unit 54.

[0063] 16, the level information 62 is displayed on the display 11 through the drug discovery support screen 100B under the control of the display control unit 54 (step ST210). The operator evaluates the candidate substance 26 using the level information 62 as a starting point.

[0064] As described above, the CPU 32 of the drug discovery support device 10 includes the RW control unit 50, the detection unit 51, the demarcation unit 52, and the display control unit 54. The RW control unit 50 acquires the target slice image 15T, which depicts the brain B of the subject S, by reading it from the storage 30. The detection unit 51 detects an abnormal portion estimated to have occurred in the target slice image 15T. The demarcation unit 52 demarcates the anatomical portion in the target slice image 15T by aligning the reference image 43, which has position information of multiple anatomical portions of the brain B, with the target slice image 15T. The display control unit 54 outputs degree information 62, which indicates the degree of occurrence of the abnormal portion in each of the multiple anatomical portions, by displaying it on the drug discovery support screen 100B. This makes it possible to easily grasp the degree of abnormality occurring in each anatomical portion of the brain B.

[0065] One method for defining anatomical regions in the target section image 15T is to apply a relatively simple image recognition technique such as pattern matching. However, image recognition techniques such as pattern matching have a relatively low accuracy in defining anatomical regions. Therefore, the technique disclosed herein improves the accuracy of defining anatomical regions by aligning the reference image 43 with the target section image 15T to define anatomical regions in the target section image 15T.

[0066] The detection unit 51 extracts a feature of interest 68IN of a region of interest 80 in the target slice image 15T, and determines whether the region of interest 80 is an abnormal portion based on the difference Δ between the feature of interest 68IN and a surrounding feature 68S, which is a reference feature. This makes it easy to determine whether the region of interest 80 is an abnormal portion.

[0067] An abnormal part is a part that has a shape and texture different from the surrounding area, such as a tumor, inflammation, cyst, or infiltration. Therefore, if the reference feature is set to the surrounding feature 68S extracted from the surrounding area 81 of the region of interest 80, the accuracy of detecting the abnormal part can be improved.

[0068] The degree information 62 includes the number of abnormal parts for each of a plurality of anatomical regions and the area ratio of the abnormal parts for each of a plurality of anatomical regions, which makes it possible to more accurately grasp the degree of abnormality occurring in each of the anatomical regions.

[0069] The organ image is a slice image 15 of a brain slice BS. In the slice image 15, the brain slice BS is entirely stained with a dye, making it difficult to see the structural characteristics of the brain B. For this reason, if a relatively simple image recognition technique such as pattern matching is applied, the accuracy of defining the anatomical regions may be very low. Therefore, by defining the anatomical regions by aligning them with the reference image 43, the effect of improving the accuracy of defining the anatomical regions can be more effectively achieved.

[0070] The pull-down menu 102 of the drug discovery support screen 100A may be provided with an option to specify all of the subjects S constituting the administration group 25 or the control group 27. Then, as an example, as shown in FIG. 18 , the extent information 62 may be displayed, generated by using the section images 15 of all of the subjects S in the administration group 25 as the target section image 15T. In this manner, the operator can know the extent to which abnormalities thought to be caused by the candidate substance 26 have occurred in each anatomical region in the administration group 25. Furthermore, when the control group 27 is selected in the pull-down menu 102, the operator can know the extent to which abnormalities thought to be caused by the attributes and / or rearing environment of the subjects S, rather than the candidate substance 26, have occurred in each anatomical region in the control group 27.

[0071] Alternatively, the pull-down menu 101 of the drug discovery support screen 100A may be provided with an option to simultaneously specify both the administration group 25 and the control group 27. Then, as an example, the degree information 62A and 62B may be displayed side by side, as in the drug discovery support screen 100B shown in FIG. 19 . The degree information 62A is the degree information 62 generated using the section images 15 of all the subjects S constituting the administration group 25 as the target section images 15T. The degree information 62B is the degree information 62 generated using the section images 15 of all the subjects S constituting the control group 27 as the target section images 15T. This allows the operator to easily grasp the extent to which abnormalities thought to be caused by the candidate substance 26 and abnormalities thought to be caused by the attributes and / or rearing environment of the subjects S occur in each anatomical region.

[0072] Second Embodiment In the first embodiment, the surrounding feature 68S extracted from the surrounding region 81 of the region of interest 80 is exemplified as the reference feature, but the present invention is not limited to this.

[0073] 20, as an example, in the second embodiment, a representative feature 68R is used as a reference feature. The representative feature 68R is a representative value, such as an average value, of a plurality of feature values ​​68 extracted from a plurality of patch images 65 obtained by subdividing a second slice image 152 obtained from a subject S constituting a control group 27. The processing of the detection unit 51 is the same as the processing shown in FIG. 9 in the first embodiment, except that the surrounding feature 68S becomes the representative feature 68R, and therefore a description thereof will be omitted.

[0074] As described above, the control group 27 is a collection of subjects S who were not administered the drug candidate substance 26. Naturally, subjects S who were not administered the candidate substance 26 will not develop abnormalities caused by the candidate substance 26. Therefore, by using the representative feature 68R extracted from the second slice image 152 of the subjects S who were not administered the candidate substance 26 as the reference feature, abnormal portions can be detected with high accuracy.

[0075] In addition, the representative feature 68R of the feature 68 extracted from the second slice images 152 of multiple subjects S who were not administered the candidate substance in evaluation tests of a candidate drug developed in the past may be used as the reference feature.

[0076] 21 as an example, in the 3_1 embodiment, the display control unit 54 controls the display of a drug discovery support screen 100C on the display 11. The display control unit 54 displays the drug discovery support screen 100C before the generation unit 53 generates the degree information 62 (before the drug discovery support screen 100B is displayed) or after the generation unit 53 generates the degree information 62 (after the drug discovery support screen 100B is displayed).

[0077] One target slice image 15T is displayed on the drug discovery support screen 100C. The anatomical region defined by the defining unit 52 is superimposed on the target slice image 15T. A legend 85 of the anatomical region is displayed to the right of the target slice image 15T. In addition, a back button 110 and a forward button 111 are provided below the target slice image 15T for switching the display of the target slice image 15T. The drug discovery support screen 100C disappears when an OK button 112 is selected.

[0078] As described above, in the third embodiment, the display control unit 54 performs control to superimpose the defined anatomical region on the target slice image 15T and display it on the display 11. This allows the operator to confirm the validity of the defined anatomical region.

[0079] 22, in the third embodiment, the operator can modify the defined anatomical region on the drug discovery support screen 100C. Specifically, a modification button 113 is provided at the bottom of the drug discovery support screen 100C, and while the modification button 113 is selected, the operator can modify the outline of the anatomical region on the target slice image 15T using, for example, a mouse cursor 114. The modification of the anatomical region is confirmed when the OK button 112 is selected.

[0080] As an example, as shown in FIG. 23 , in the third embodiment, the instruction receiving unit 120 receives an instruction 121 to modify an anatomical region from an operator via the input device 12. The modification instruction 121 includes position coordinates of pixels of the contour of the anatomical region modified by the operator using the cursor 114. The instruction receiving unit 120 outputs the modification instruction 121 to the demarcation unit 52. The demarcation unit 52 modifies the demarcation result 61 in accordance with the modification instruction 121 to generate a modified demarcation result 61C. The demarcation unit 52 outputs the modified demarcation result 61C to the generation unit 53.

[0081] The generation unit 53 generates the degree information 62 based on the corrected demarcation result 61C. In an aspect in which the drug discovery support screen 100C is displayed before the generation unit 53 generates the degree information 62 (before the drug discovery support screen 100B is displayed), the degree information 62 displayed on the drug discovery support screen 100B is generated from the beginning based on the corrected demarcation result 61C. On the other hand, in an aspect in which the drug discovery support screen 100C is displayed after the generation unit 53 generates the degree information 62 (after the drug discovery support screen 100B is displayed), the degree information 62 displayed on the drug discovery support screen 100B switches from that generated based on the pre-correction demarcation result 61 to that generated based on the corrected demarcation result 61C.

[0082] In this way, in the third embodiment, the instruction receiving unit 120 receives an instruction 121 to correct the defined anatomical region. Therefore, the operator can manually correct the anatomical region defined by the defining unit 52.

[0083] Furthermore, the display control unit 54 outputs the degree information 62 generated based on the anatomical region corrected in response to the correction instruction 121. Therefore, the degree information 62 based on the anatomical region corrected by the operator can be viewed by the operator.

[0084] In the above embodiments, the slide specimen 18 on which only a brain slice BS is mounted has been exemplified, but this is not limiting. It is also possible to handle slide specimens on which tissue slices from multiple organs, such as heart slices and bone marrow slices, are mounted on a single slide glass 16 in addition to the brain slice BS. In this case, an identification unit for identifying the tissue slices of each organ is configured in the CPU 32, and the tissue slices of each organ are identified by the identification unit before subsequent processing, such as detecting abnormal portions, is performed. This makes it possible to handle slide specimens on which tissue slices from multiple organs are mounted. Slide specimens on which tissue slices from multiple organs are mounted are more common than the slide specimen 18 on which a tissue slice from a single organ is mounted, as in the first embodiment. This allows for drug discovery support that is more suited to general operations.

[0085] Although the encoder unit 71 of the autoencoder 70 has been exemplified as the feature extractor 41, the present invention is not limited to this. The encoder unit of a convolutional neural network trained to output a determination result indicating whether or not an abnormality exists in a certain class, for example, a brain slice BS shown in the patch image 65, in response to an input of the patch image 65 may be used as the feature extractor 41. Furthermore, other machine learning models such as a support vector machine may be used as the feature extractor 41.

[0086] In the tissue slice shown in the learning patch image 65L, areas that are likely to be mistakenly detected as abnormal may be masked before use for learning. Examples of areas that are likely to be mistakenly detected as abnormal include blood vessel areas in the brain slice BS and dust particles that adhered to the slide specimen 18 during preparation.

[0087] The feature amount 68 is not limited to that extracted by the feature amount extractor 41. It may be the average value, maximum value, minimum value, mode, or variance of the pixel values ​​of the patch image 65, for example.

[0088] The anatomical regions are not limited to the cerebral cortex, hippocampal formation, amygdala, etc. For example, the hippocampal formation may be further classified into the dentate gyrus, hippocampus, subiculum, entorhinal cortex, etc., and the hippocampus may be further classified into CA1, CA2, and CA3 regions. Multiple types of reference images 43 with different classifications of anatomical regions may be stored and used depending on the purpose.

[0089] The organ is not limited to the brain B, but may be the stomach, lung, small intestine, or large intestine. The subject S is not limited to a rat, but may be a mouse, guinea pig, gizzard shad, hamster, ferret, rabbit, dog, cat, monkey, or the like.

[0090] The organ image is not limited to the illustrated slice image 15. It may be a tomographic image obtained by computed tomography (CT), a tomographic image obtained by magnetic resonance imaging (MRI), a tomographic image obtained by positron emission tomography (PET), or a tomographic image obtained by single photon emission computed tomography (SPECT).

[0091] The output form of the degree information 62 is not limited to the drug discovery support screen 100B exemplified in the first embodiment, but also includes printout onto a paper medium, or file output as an attachment to an email, etc.

[0092] The drug discovery support device 10 may be a personal computer installed in a pharmaceutical facility as shown in FIG. 1, or may be a server computer installed in a data center independent of the pharmaceutical facility.

[0093] When the drug discovery support device 10 is configured as a server computer, the slice images 15 are transmitted from personal computers installed in each pharmaceutical facility to the server computer via a network such as the Internet. The server computer distributes the drug discovery support screen 100A, etc. to the personal computer in the form of screen data for web distribution created using a markup language such as XML (Extensible Markup Language). The personal computer reproduces the drug discovery support screen 100A, etc. to be displayed on a web browser based on the screen data and displays it on a display. Note that other data description languages ​​such as JSON (Javascript (registered trademark) Object Notation) may be used instead of XML.

[0094] The drug discovery support device 10 according to the technique of the present disclosure can be widely used across all stages of drug development, from the earliest stage of setting drug discovery targets to the final stage of clinical trials.

[0095] The hardware configuration of the computer constituting the drug discovery support device 10 according to the technology of the present disclosure can be modified in various ways. For example, the drug discovery support device 10 can be configured with multiple computers separated as hardware in order to improve processing power and reliability. For example, the functions of the detection unit 51 and the demarcation unit 52 can be distributed and performed by two computers. In this case, the drug discovery support device 10 is configured with two computers.

[0096] In this way, the hardware configuration of the computer of the drug discovery support apparatus 10 can be changed as appropriate depending on the required performance, such as processing power, safety, and reliability. Furthermore, not only the hardware, but also application programs such as the operating program 40 can be duplicated or stored in multiple storage devices in order to ensure safety and reliability.

[0097] In each of the above embodiments, the following various processors can be used as the hardware structure of processing units that perform various processes, such as the RW control unit 50, the detection unit 51, the definition unit 52, the generation unit 53, the display control unit 54, and the instruction reception unit 120. The various processors include the CPU 32, which is a general-purpose processor that executes software (operation program 40) and functions as various processing units, as described above, as well as dedicated electrical circuits that are processors having a circuit configuration specifically designed to perform specific processes, such as a programmable logic device (PLD) that is a processor whose circuit configuration can be changed after manufacture, such as an FPGA (Field Programmable Gate Array), and an ASIC (Application Specific Integrated Circuit).

[0098] A single processing unit may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (e.g., a combination of multiple FPGAs and / or a combination of a CPU and an FPGA).Furthermore, multiple processing units may be configured with a single processor.

[0099] Examples of configuring multiple processing units with a single processor include, first, a form in which one processor is configured with a combination of one or more CPUs and software, as typified by computers such as client and server, and this processor functions as multiple processing units. Second, a form in which a processor is used to realize the functions of the entire system including multiple processing units with a single IC (Integrated Circuit) chip, as typified by systems on chips (SoCs). In this way, various processing units are configured using one or more of the above-mentioned various processors as a hardware structure.

[0100] Furthermore, more specifically, the hardware structure of these various processors can be an electric circuit (circuitry) that combines circuit elements such as semiconductor elements.

[0101] The technology of the present disclosure can be appropriately combined with the various embodiments and / or various modified examples described above. Furthermore, it is not limited to the above-described embodiments, and various configurations can be adopted without departing from the spirit of the present disclosure. Furthermore, the technology of the present disclosure extends not only to programs but also to storage media that non-temporarily store programs.

[0102] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[0103] In this specification, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed by connecting them with "and / or."

[0104] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

Claims

1. A processor is provided. The processor, acquiring organ images of the subject's organs, the organ images being used in an evaluation test for evaluating a candidate drug substance; Detecting an abnormal portion in which an abnormality is presumed to occur from the organ image; defining the anatomical parts in the organ image by registering the organ image with a reference image having position information of a plurality of anatomical parts of the organ; outputting degree information representing the degree of occurrence of the abnormal portion in each of the plurality of anatomical sites; In detecting the abnormal portion, all of the subjects constituting an administration group to which the candidate substance is administered and / or all of the subjects constituting a control group to which the candidate substance is not administered can be targeted; As the degree information, it is possible to output degree information generated for all of the subjects constituting the administration group, and / or degree information generated for all of the subjects constituting the control group. Information processing device.

2. The processor, Extracting features of a region of interest in the organ image; The information processing apparatus according to claim 1 , wherein the apparatus judges whether the region of interest is the abnormal portion based on a difference between a feature amount extracted from the region of interest and a reference feature amount.

3. The information processing apparatus according to claim 2 , wherein the reference feature amount is a feature amount extracted from a region surrounding the region of interest.

4. the organ image is an image used in an evaluation test for evaluating a candidate substance for a drug; The information processing apparatus according to claim 2 , wherein the reference feature amount is a feature amount extracted from an organ image of a subject to which the candidate substance has not been administered.

5. The processor, The information processing apparatus according to claim 1 , further comprising control for superimposing the defined anatomical region on the organ image and displaying the same on a display device.

6. The processor, The information processing device according to claim 5 , further comprising an instruction to modify the defined anatomical portion.

7. The processor, The information processing device according to claim 6 , wherein the degree information is generated based on an anatomical part corrected in response to the correction instruction and is output.

8. The information processing apparatus according to claim 1 , wherein the degree information includes the number of the abnormal parts for each of the plurality of anatomical sites and an area ratio of the abnormal parts for each of the plurality of anatomical sites.

9. The information processing apparatus according to claim 1 , wherein the organ image is a slice image obtained by capturing a tissue slice of the organ.

10. Obtaining organ images of a subject's organs, the organ images being used in an evaluation test for evaluating a candidate drug substance; detecting an abnormal portion in which an abnormality is presumed to occur from the organ image; defining the anatomical parts in the organ image by registering the organ image with a reference image having position information of a plurality of anatomical parts of the organ; and outputting degree information representing the degree of occurrence of the abnormal portion in each of the plurality of anatomical sites; Including, In detecting the abnormal portion, all of the subjects constituting an administration group to which the candidate substance is administered and / or all of the subjects constituting a control group to which the candidate substance is not administered can be targeted; As the degree information, it is possible to output degree information generated for all of the subjects constituting the administration group, and / or degree information generated for all of the subjects constituting the control group. A method for operating an information processing device.

11. Obtaining organ images of a subject's organs, the organ images being used in an evaluation test for evaluating a candidate drug substance; detecting an abnormal portion in which an abnormality is presumed to occur from the organ image; defining the anatomical parts in the organ image by registering the organ image with a reference image having position information of a plurality of anatomical parts of the organ; and outputting degree information representing the degree of occurrence of the abnormal portion in each of the plurality of anatomical sites; causing a computer to execute a process including In detecting the abnormal portion, all of the subjects constituting an administration group to which the candidate substance is administered and / or all of the subjects constituting a control group to which the candidate substance is not administered can be targeted; As the degree information, it is possible to output degree information generated for all of the subjects constituting the administration group, and / or degree information generated for all of the subjects constituting the control group. An operating program for an information processing device.

12. The processor, The information processing apparatus according to claim 2 , wherein the size of the region of interest is changed stepwise to extract the feature amount and to determine whether the region of interest is the abnormal portion.