Information processing device, method for operating information processing device, and program for operating information processing device

By detecting abnormal parts in the organ image and demarcating anatomical parts, the problem of difficulty in evaluating the degree of abnormalities in each anatomical part of the organ in the prior art is solved, and accurate evaluation and precise demarcation of organ abnormalities is achieved.

CN119031878APending Publication Date: 2024-11-26FUJIFILM CORP
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202380030809.6
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Priority Date
2022-03-29
Filing Date
2023-01-30
Publication Date
2024-11-26

AI Technical Summary

Technical Problem

Existing organ image evaluation techniques are difficult to accurately grasp the degree of abnormality in each anatomical part of the organ, especially in organs with multiple anatomical parts such as the brain.

Method used

By obtaining organ images, detecting abnormal parts, and using position information of multiple anatomical parts with organs to align with the organ image, demarcate the anatomical parts, and outputting the abnormality degree information of each anatomical part, including the number and area of ​​abnormal parts ratio.

Benefits of technology

The accuracy of abnormalities in each anatomical part of the organ is achieved, the accuracy of demarcation of anatomical part is improved, and the occurrence of abnormalities in the organ can be better grasped.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119031878A_ABST
    Figure CN119031878A_ABST
Patent Text Reader

Abstract

This information processing device is provided with a processor that performs: a process for acquiring an organ image in which an organ of a subject is captured; detecting an abnormal part estimated to be abnormal according to the organ image; demarcating an anatomical site in the organ image by aligning a reference image having positional information of a plurality of anatomical sites of the organ with the organ image; and outputting degree information indicating the degree of occurrence of each abnormal portion of the plurality of anatomical sites.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The technology of the present invention relates to an information processing device, an operating method of the information processing device, and an operating program of the information processing device. Background Art

[0002] In the medical field, various evaluations are being conducted based on organ images obtained by photographing organs of a subject. For example, in an experiment in which candidate substances of a drug are administered to a subject such as a rat to evaluate the efficacy and toxicity of the candidate substances, slice images obtained by photographing tissue slices (brain slices, heart slices, etc.) collected by autopsy of the subject are used as organ images. In the past, evaluations were performed by experts such as pathologists observing organ images, but with recent advances in image analysis technology, technologies have been developed that automatically perform evaluations without the need for expert operation. For example, Japanese Patent Publication No. 2020-533725 describes a technology that applies a machine learning model to multiple block images after subdividing a slice image, and determines whether abnormalities such as cancer occur in the tissue slices reflected in the block images. Summary of the invention

[0003] Technical issues to be solved by the invention

[0004] Among organs, there are organs with multiple anatomical parts. For example, the brain has anatomical parts such as the cerebral cortex, hippocampus, thalamus, and tonsils. In the evaluation using organ images, it is hoped to understand the degree of abnormality that has occurred in each of these anatomical parts. However, in the technology described in Japanese Patent Publication No. 2020-533725, although it is possible to understand whether an abnormality has occurred in a tissue slice reflected in a block image, it is difficult to understand the degree of abnormality that has occurred in each anatomical part.

[0005] One embodiment of the technology according to the present invention provides an information processing device, an operating method of the information processing device, and an operating program of the information processing device that can easily grasp the degree of abnormality that has occurred in each anatomical part of an organ.

[0006] Means for solving technical problems

[0007] The information processing device of the present invention has a processor that performs the following processing: acquiring an organ image obtained by photographing an organ of a subject; detecting an abnormal part estimated to be abnormal based on the organ image; demarcating anatomical parts in the organ image by aligning a reference image having position information of multiple anatomical parts of the organ with the organ image; and outputting degree information representing the degree of occurrence of each abnormal part of the multiple anatomical parts.

[0008] Preferably, the processor performs the following processing: extracting a feature quantity of a region of interest in the organ image; and determining whether the region of interest is an abnormal portion based on a difference between the feature quantity extracted from the region of interest and a reference feature quantity.

[0009] Preferably, the reference feature quantity is a feature quantity extracted from a region surrounding the region of interest.

[0010] Preferably, the organ image is an image of an evaluation test for evaluating a candidate substance of a drug, and the reference feature amount is a feature amount extracted from an organ image of a subject to which the candidate substance has not been administered.

[0011] Preferably, the processor controls the delineated anatomical region to be superimposed on the organ image and displayed on the display.

[0012] Preferably, the processor receives an indication of a revision of the delineated anatomical region.

[0013] Preferably, the processor outputs degree information generated based on the anatomical part modified according to the modification instruction.

[0014] Preferably, the degree information includes the number of each abnormal portion of the plurality of anatomical portions and the area ratio of each abnormal portion of the plurality of anatomical portions.

[0015] Preferably, the organ image is a slice image obtained by photographing a tissue slice of the organ.

[0016] The working method of the information processing device of the present invention includes the following steps: acquiring an organ image obtained by photographing an organ of a subject; detecting an abnormal part estimated to be abnormal based on the organ image; delineating the anatomical part in the organ image by aligning a reference image having position information of multiple anatomical parts of the organ with the organ image; and outputting degree information representing the degree of occurrence of each abnormal part of the multiple anatomical parts.

[0017] The working program of the information processing device of the present invention enables a computer to execute processing including the following steps: acquiring an organ image obtained by photographing an organ of a subject; detecting an abnormal part estimated to be abnormal based on the organ image; delineating anatomical parts in the organ image by aligning a reference image having position information of multiple anatomical parts of the organ with the organ image; and outputting degree information representing the degree of occurrence of each abnormal part of the multiple anatomical parts.

[0018] Effects of the Invention

[0019] According to the technology of the present invention, it is possible to provide an information processing device, an operating method of the information processing device, and an operating program of the information processing device that can easily grasp the degree of abnormality occurring in each anatomical part of an organ. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 The diagram shows the process of an evaluation test, slice images, and a drug development support device.

[0021] Figure 2 It is a graph showing the drug-treated group and the control group.

[0022] Figure 3 This is a block diagram showing a computer constituting the drug development support device.

[0023] Figure 4 This is a block diagram showing a processing unit of a CPU of a drug development support device.

[0024] Figure 5 A diagram showing a block image obtained by subdividing a slice image.

[0025] Figure 6 1 is a diagram showing a state where a feature quantity is extracted from a block image by a feature quantity extractor.

[0026] Figure 7 This is a diagram showing the structure of a feature extractor.

[0027] Figure 8 is a diagram showing the processing in the learning phase of the autoencoder.

[0028] Fig. 9 It is a diagram showing the processing of the detection unit.

[0029] Fig.10 It is a graph showing the detection result.

[0030] Fig.11 is a diagram representing a reference image.

[0031] Fig.12 This is a diagram showing the processing of the delimiting unit.

[0032] Fig.13 It is a diagram showing the delineation result.

[0033] Fig.14 It is a graph showing degree information.

[0034] Fig.15 This figure shows a drug discovery support screen displaying a slice image.

[0035] Fig.16 This figure shows a drug development support screen that displays degree information.

[0036] Fig.17 This is a flowchart showing the processing procedure of the drug development support device.

[0037] Fig.18 This is a diagram showing another example of the degree information displayed on the drug development support screen.

[0038] Fig.19 This is a diagram showing another example of the degree information displayed on the drug development support screen.

[0039] Fig. 20 It is a diagram showing the processing of the detection unit according to the second embodiment.

[0040] Fig.21 This figure shows a drug development support screen in which a delineated anatomical part is displayed superimposed on a slice image.

[0041] Fig. 22 This is a diagram showing a drug development support screen for receiving an instruction to correct an anatomical part.

[0042] Fig.23 This is a diagram showing a 3_2nd embodiment of generating degree information based on the anatomical part corrected according to the correction instruction. DETAILED DESCRIPTION

[0043] [First embodiment]

[0044] As an example, Figure 1 As shown, the drug development support device 10 is an example of an "information processing device" involved in the technology of the present invention, and is used to evaluate candidate substances 26 of a drug (see Figure 2 )'s efficacy and toxicity. The drug development support device 10 is, for example, a desktop personal computer, and is provided with a display 11 for displaying various screens and an input device 12 composed of a keyboard, a mouse, a touch panel and / or a microphone for voice input. The drug development support device 10 is operated by an operator such as a staff member of a pharmaceutical facility involved in drug development. In addition, the display 11 is an example of a "display" involved in the technology of the present invention.

[0045] The slice image 15 is input into the drug development support device 10. The slice image 15 is generated, for example, by the following sequence. First, a subject S such as a rat prepared for evaluating a candidate substance 26 is dissected, and organs of the subject S are collected, and multiple tissue slices (hereinafter referred to as brain slices) BS of the coronal section of the brain B are collected. Next, after the collected brain slices BS are pasted one by one on a slide 16, the brain slices BS are stained, and hematoxylin-eosin is used for staining. Next, the stained brain slices BS are covered with a cover glass 17 to complete a slide specimen 18. Then, the slide specimen 18 is placed on a photographic device 19 such as a digital optical microscope, and the slice image 15 is photographed by the photographic device 19. In the slice image 15 obtained in this way, a subject ID (Identification Data) for uniquely identifying the subject S, an image ID for uniquely identifying the slice image 15, a slice position of the coronal section, and a shooting date, etc. are attached. The slice image 15 is an example of an “organ image” and an “image for evaluating a candidate drug substance” according to the present invention. In addition, the staining may be staining with hematoxylin alone or staining with nuclear red pigment.

[0046] As an example, Figure 2 As shown, the slice images 15 input to the drug development support device 10 include a first slice image 151 and a second slice image 152. The first slice image 151 is an image obtained by photographing a brain slice BS of a subject S of the dosing group 25. The dosing group 25 is composed of a plurality of subjects S to which a candidate substance 26 of a drug is administered. The number of subjects S constituting the dosing group 25 is, for example, about 5 to 10. A plurality of first slice images 151 are obtained from one subject S, and therefore, the number of first slice images 151 obtained by multiplying the number obtained from one subject S by the number of subjects S is obtained from the dosing group 25. For example, when the number obtained from one subject S is 10 and the number of subjects S constituting the dosing group 25 is 10, 10×10=100 first slice images 151 can be obtained.

[0047] The second slice image 152 is an image obtained by photographing a brain slice BS of a subject S of the control group 27. In contrast to the drug administration group 25, the control group 27 is composed of a plurality of subjects S to which the candidate substance 26 is not administered. The subjects S constituting the control group 27 have the same attributes as the subjects S constituting the drug administration group 25 and are subjects S in the same breeding environment. The same attributes refer to, for example, the same age and / or the same gender. In addition, the same attributes also include the same age composition ratio and / or the same gender composition ratio (5 males and 5 females, etc.). The same breeding environment refers to, for example, the same feed, the same temperature and humidity of the breeding space, and / or the same width of the breeding space. The "same" in the same breeding environment refers to the same meaning of the error that is generally allowed in the technical field to which the technology of the present invention belongs and does not deviate from the technical spirit of the present invention, in addition to being completely the same. The number of subjects S constituting the control group 27 is also the same as that of the drug administration group 25, for example, about 5 to 10. The second slice image 152 is obtained from a single subject S in the same manner as the first slice image 151, and therefore a number of second slice images 152 obtained by multiplying the number of slice images obtained from a single subject S by the number of subjects S is obtained from the control group 27. In the following description, the first slice image 151 and the second slice image 152 are collectively referred to as slice images 15 unless they need to be distinguished.

[0048] In addition, in the administration group 25, there are a plurality of groups with different administration doses of the candidate substance 26. For example, the administration doses of the candidate substance 26 are made different in three stages, namely, a high administration dose group, a medium administration dose group, and a low administration dose group. In this way, the effect of the administration dose of the candidate substance 26 on the subject S can be determined.

[0049] As an example, Figure 3 As shown, the computer constituting the drug development support apparatus 10 includes a storage device 30 , a memory 31 , a CPU (Central Processing Unit) 32 , and a communication unit 33 in addition to the display 11 and the input device 12 . These are connected to each other via a bus 34 .

[0050] The storage device 30 is a hard disk drive built into the computer constituting the drug development support device 10 or connected via a cable or a network. Alternatively, the storage device 30 is a disk array formed by connecting multiple hard disk drives. The storage device 30 stores control programs such as an operating system, various application programs, and various data attached to these programs. In addition, a solid-state drive can also be used instead of a hard disk drive.

[0051] The memory 31 is a working memory for the CPU 32 to perform processing. The CPU 32 loads the program stored in the storage device 30 into the memory 31, and performs processing according to the program. Thus, the CPU 32 centrally controls each part of the computer. The CPU 32 is an example of a "processor" involved in the technology of the present invention. In addition, the memory 31 can be built into the CPU 32. The communication unit 33 performs transmission control of various information with external devices such as the photographic device 19.

[0052] As an example, Figure 4 As shown, a working program 40 is stored in the storage device 30 of the drug development support apparatus 10. The working program 40 is an application program for causing the computer to function as the drug development support apparatus 10. That is, the working program 40 is an example of the "working program of the information processing device" involved in the technology of the present invention. The storage device 30 also stores a feature quantity extractor 41, setting range information 42, and a reference image 43.

[0053] When the operation program 40 is activated, the CPU 32 of the computer constituting the drug development support device 10 cooperates with the memory 31 and the like to function as a read / write (hereinafter referred to as RW (Read Write)) control unit 50 , a detection unit 51 , a delimiting unit 52 , a generating unit 53 and a display control unit 54 .

[0054] The RW control unit 50 controls storage of various data in the storage device 30 and reading of various data in the storage device 30. For example, the RW control unit 50 stores the slice image 15 from the imaging device 19 in the storage device 30.

[0055] The RW control unit 50 obtains the slice image 15 by reading it from the storage device 30. The RW control unit 50 outputs the slice image 15 to the detection unit 51, the delimitation unit 52, and the display control unit 54. The slice image 15 output from the RW control unit 50 to the detection unit 51 and the like is a target for grasping the degree of occurrence of the abnormal part of the brain slice BS. Hereinafter, the target slice image 15 for grasping the degree of occurrence of the abnormal part of the brain slice BS is denoted as a target slice image 15T (reference Figure 5 The target slice image 15T is, for example, all of the plurality of slice images 15 obtained from one subject S among the plurality of subjects S constituting the medication group 25 .

[0056] The RW control unit 50 reads the feature extractor 41 and the setting range information 42 from the storage device 30, and outputs the feature extractor 41 and the setting range information 42 to the detection unit 51. In addition, the RW control unit 50 reads the reference image 43 from the storage device 30, and outputs the reference image 43 to the delimitation unit 52.

[0057] The detection unit 51 detects an abnormal part estimated to be abnormal from the target slice image 15T using the feature quantity extractor 41. The abnormal part refers to a part where a lesion (e.g., tumor, inflammation, cyst, infiltration, etc.) not observed in a normal brain slice BS occurs. The detection unit 51 outputs the detection result 60 of the abnormal part to the generation unit 53.

[0058] The delimiting unit 52 delimits a plurality of anatomical parts of the brain B in the target slice image 15T by aligning the reference image 43 with the target slice image 15T. The delimiting unit 52 outputs a delimitation result 61 of the anatomical parts of the brain B to the generating unit 53. The delimiting unit 52 performs the process of delimiting the anatomical parts in parallel with the process of detecting the abnormal part by the detecting unit 51.

[0059] The generating unit 53 generates degree information 62 indicating the degree of occurrence of each abnormal portion in a plurality of anatomical parts of the brain B based on the detection result 60 and the demarcation result 61 . The generating unit 53 outputs the degree information 62 to the display control unit 54 .

[0060] The display control unit 54 controls the display of various screens on the display 11. Among the various screens, there is a drug development support screen 100A (see Fig.15 ) and the drug development support screen 100B showing the degree information 62 (reference Fig.16 ) etc. In addition, in the CPU 32, in addition to the processing units 50 to 54, an instruction receiving unit 120 (see Fig.23 )wait.

[0061] As an example, Figure 5 As shown, the detection unit 51 uses a well-known image recognition technology to identify the brain slice BS shown in the target slice image 15T, and divides the identified brain slice BS into a plurality of block images 65. The block image 65 has a predetermined size that can be processed by the feature quantity extractor 41. Figure 5 In the example, the block image 65 does not have an area overlapping with other block images 65 , but the block image 65 may overlap with a part of other block images 65 .

[0062] As an example, Figure 6 As shown, the detection unit 51 extracts the feature amount 68 for each of the plurality of block images 65 obtained by segmenting the target slice image 15T using the feature amount extractor 41 .

[0063] As an example, Figure 7As shown in FIG. 4 , the encoder unit 71 of the automatic encoder 70 is used in the feature quantity extractor 41. The automatic encoder 70 includes a decoder unit 72 in addition to the encoder unit 71. The block image 65 is input to the encoder unit 71. The encoder unit 71 converts the block image 65 into a feature quantity 68. The encoder unit 71 transmits the feature quantity 68 to the decoder unit 72. The decoder unit 72 generates a restored image 73 of the block image 65 based on the feature quantity 68.

[0064] As is well known, the encoder unit 71 includes a convolution layer for performing convolution processing using a filter and a pooling layer for performing pooling processing such as maximum pooling processing. The same is true for the decoder unit 72. The encoder unit 71 extracts a feature 68 by repeatedly performing convolution processing based on a convolution layer and pooling processing based on a pooling layer on the input block image 65. The extracted feature 68 represents the features of the shape and texture of the brain slice BS reflected in the block image 65. The feature 68 is a set of multiple numerical values. That is, the feature 68 is multidimensional data. The dimension of the feature 68 is, for example, 512, 1024, or 2048.

[0065] As an example, Figure 8 As shown, the auto encoder 70 inputs the learning block image 65L in the learning stage before the encoder unit 71 is transferred to the feature quantity extractor 41 to perform learning. The auto encoder 70 outputs the learning restoration image 73L for the learning block image 65L. Based on these learning block images 65L and the learning restoration image 73L, the loss calculation of the auto encoder 70 using the loss function is performed. Then, based on the result of the loss calculation, the update setting of various coefficients of the auto encoder 70 (coefficients of the filter of the convolution layer, etc.) is performed, and the auto encoder 70 is updated according to the update setting. Here, the learning block image 65L is supplied from a plurality of block images 65 obtained by subdividing the brain slice BS shown in the slice image 15 obtained in the evaluation test of the candidate substance of the drug developed in the past.

[0066] In the learning phase of the auto encoder 70, the above series of processing of inputting the learning block image 65L to the auto encoder 70, outputting the learning restored image 73L from the auto encoder 70, loss calculation, updating setting, and updating of the auto encoder 70 is repeated while the learning block image 65L is replaced. The repetition of the above series of processing ends when the restoration accuracy from the learning block image 65L to the learning restored image 73L reaches a preset level. In this way, the encoder unit 71 of the auto encoder 70 whose restoration accuracy reaches the set level is stored as the feature quantity extractor 41 in the storage device 30 of the drug development support apparatus 10. In addition, the learning may end when the above series of processing is repeated a set number of times regardless of the restoration accuracy from the learning block image 65L to the learning restored image 73L.

[0067] Such learning of the autoencoder 70 may be performed by the drug development support apparatus 10 or by a device different from the drug development support apparatus 10. In the latter case, the feature quantity extractor 41 is sent from another device to the drug development support apparatus 10, and the feature quantity extractor 41 is stored in the storage device 30 by the RW control unit 50.

[0068] As an example, Fig. 9 As shown, the detection unit 51 determines whether the attention region 80 is an abnormal part based on the difference Δ between the attention feature amount 68IN extracted from the attention region 80 and the surrounding feature amount 68S extracted from the surrounding area 81. The attention region 80 is a region that is the object of determination of whether it is an abnormal part. The attention region 80 is, for example, a region of one or more (4, 9, 16, etc.) block images 65. The surrounding area 81 is the surrounding area of ​​the attention region 80. The detection unit 51 makes determinations while moving the attention region 80 one by one. Furthermore, the detection unit 51 changes the size of the attention region 80 (the number of block images 65 constituting the attention region 80) in multiple stages to make determinations. Fig. 9 In the example, there are 8 surrounding regions 81, but when the end of the brain slice BS has the region of interest 80, there are fewer than 8 surrounding regions 81. The surrounding feature amount 68S is a representative value, such as an average value, of the feature amounts 68 extracted from the plurality of surrounding regions 81. The surrounding feature amount 68S is an example of a “reference feature amount” involved in the technology of the present invention.

[0069] The detection unit 51 compares the difference Δ between the attention feature quantity 68IN and the surrounding feature quantity 68S with the setting range of the setting range information 42. The setting range information 42 includes a lower limit value and an upper limit value indicating the setting range. When the difference Δ is within the setting range (lower limit value ≤ difference Δ ≤ upper limit value), the detection unit 51 outputs a determination result 82 indicating that the attention area 80 is not an abnormal part. On the other hand, when the difference Δ is outside the setting range (difference Δ < lower limit value or difference Δ > upper limit value), the detection unit 51 outputs a determination result 82 indicating that the attention area 80 is an abnormal part.

[0070] As an example, Fig.10 As shown, the detection result 60 is information in which the determination result 82 is registered for each position coordinate of the block image 65. For example, the position coordinate of the block image 65 is the position coordinate of the diagonal point of the block image 65.

[0071] As an example, Fig.11As shown, the reference image 43 is an image having position information of multiple anatomical parts of the brain B. More specifically, the reference image 43 is an image registered in each pixel corresponding to the pixel value representing the anatomical part (for example, 0 for the cerebral cortex, 1 for the hippocampus, etc.). The reference image 43 is called an atlas (ATLAS). Fig.11 In FIG. 85, as anatomical parts, the cerebral cortex, hippocampus, thalamus, hypothalamus and amygdala are shown. In addition to these, the anatomical parts also include cerebellum, cranial nerves, etc. (refer to Fig.14 ). The reference image 43 is prepared for each slice position of the coronal section of the brain slice BS. The demarcation unit 52 aligns the object slice image 15T at the same slice position with the reference image 43. As the reference image 43, public data such as the Allen Brain Atlas provided by the Allen Institute for Brain Science in Seattle, USA can be used. In addition, the reference image 43 is not limited to public data, and can also be an image produced independently by a pharmaceutical facility.

[0072] As an example, Fig.12 As shown, the delimiting unit 52 first performs grayscale processing 90 on the target slice image 15T to highlight the outline 93 of the brain slice BS and use the target slice image 15T as a grayscale image 91. Next, the delimiting unit 52 performs alignment processing 92 for aligning the grayscale image 91 with the reference image 43. In addition, the grayscale processing 90 may not necessarily be performed.

[0073] The alignment processing 92 is a non-rigid alignment processing, and is performed, for example, in the following order. That is, a plurality of control points CP are set on the contour 93 of the brain slice BS of the grayscale image 91. Then, each control point CP is assigned to the same position of the contour 94 of the brain slice BS of the reference image 43. The delineating unit 52 derives the deformation amount of each pixel of the reference image 43 when the object slice image 15T is aligned with the reference image 43, based on the movement amount of each control point CP of the contour 94 relative to each control point CP of the contour 93, for example, by thin plate spline conversion. The delineating unit 52 deforms the reference image 43 using the derived deformation amount, and aligns the reference image 43 with the object slice image 15T. Fig.12 , the distortion of the grid points of the reference image 43 due to deformation for alignment with the target slice image 15T is expressed by thin lines. In addition, for example, a non-rigid alignment process using a B-spline curve or a non-rigid alignment process taking into account the brightness information of the target slice image 15T may be performed instead of the non-rigid alignment process based on the thin plate spline transformation.

[0074] As an example, Fig.13 As shown, the demarcation result 61 is information in which the anatomical part is registered for each position coordinate of the target slice image 15T.

[0075] As an example, Fig.14 As shown, the degree information 62 includes the number of abnormal parts in each of the multiple anatomical parts and the area ratio of each of the multiple abnormal parts in the multiple anatomical parts. The number of abnormal parts and the area ratio are values ​​derived from all of the multiple slice images 15 obtained from one subject S as an object. The area ratio of the abnormal part is the ratio of the number of pixels of the abnormal part to the number of pixels of each anatomical part in all of the multiple slice images 15 obtained from one subject S. For example, in the case where the number of pixels demarcated as the cerebral cortex by the demarcation unit 52 is 10000 and the number of pixels of the focus area 80 detected as the abnormal part by the detection unit 51 is 500 in all of the multiple slice images 15 obtained from one subject S, the area ratio is (500 / 10000)×100=5%.

[0076] As an example, Fig.15 As shown, the display control unit 54 controls the display of the drug development support screen 100A on the display 11. The display control unit 54 displays the drug development support screen 100A when the operator issues a display instruction for the slice image 15 via the input device 12 and the display instruction is received by the instruction receiving unit 120. The drug development support screen 100A includes a pull-down menu 101 for selecting one of the drug administration group 25 and the control group 27 and a pull-down menu 102 for selecting the subject ID of the subject S. Fig.15 , an example is shown in which the medication group 25 is selected in the pull-down menu 101 and the subject ID “R001” is selected in the pull-down menu 102 .

[0077] A plurality of slice images 15 are displayed side by side on the drug development support screen 100A. These plurality of slice images 15 are all slice images 15 obtained from one subject S. Fig.15 , an example is shown in which ten first slice images 151 with image IDs “SP001” to “SP010” obtained from the subject S with subject ID “R001” of the medication group 25 are displayed side by side.

[0078] An abnormality determination button 103 is provided at the bottom of the drug development support screen 100A. When the operator selects the abnormality determination button 103, an abnormality determination instruction is received by the instruction receiving unit 120. The abnormality determination instruction is an instruction for causing the detection unit 51, the delimitation unit 52, and the generation unit 53 to detect an abnormal portion from the slice image 15 displayed on the drug development support screen 100A, delimit the anatomical part in the slice image 15 displayed on the drug development support screen 100A, and generate the degree information 62 for all the slice images 15 displayed on the drug development support screen 100A. That is, the slice image 15 (here, the first slice image 151 obtained from the subject S of the subject ID "R001" of the medication group 25) displayed on the drug development support screen 100A corresponds to the target slice image 15T.

[0079] When the abnormality determination button 103 is selected and abnormal portion detection, anatomical site demarcation, and degree information 62 generation are performed, as an example, Fig.16 As shown, the display control unit 54 displays the drug development support screen 100B as a pop-up on the drug development support screen 100A. The drug development support screen 100B displays the degree information 62. The drug development support screen 100B disappears when the OK button 105 is selected.

[0080] Next, as an example, refer to Fig.17 The flowchart shown in FIG. 1 illustrates the function of the above structure. First, if the working program 40 is started in the drug development support device 10, as shown in FIG. Figure 4 As shown, the CPU 32 of the drug development support device 10 functions as an RW control unit 50 , a detection unit 51 , a delimiting unit 52 , a generating unit 53 , and a display control unit 54 .

[0081] First, a slice image 15 is obtained by photographing a brain slice BS of a subject S using the photographing device 19. The slice image 15 is output from the photographing device 19 to the drug development support apparatus 10. In the drug development support apparatus 10, the slice image 15 from the photographing device 19 is stored in the storage device 30 by the RW control unit 50 (step ST100).

[0082] When the operator issues a display instruction for the slice image 15 via the input device 12 ("Yes" in step ST110), the RW control unit 50 reads the slice image 15 specified by the display instruction from the storage device 30 (in step ST120). The slice image 15 is output from the RW control unit 50 to the display control unit 54. Fig.15 As shown, the slice image 15 is displayed on the display 11 via the drug development support screen 100A under the control of the display control unit 54 (step ST130 ).

[0083] When the operator selects the abnormality determination button 103 on the drug development support screen 100A and issues an abnormality determination instruction ("Yes" in step ST140), the RW control unit 50 reads the slice image 15 displayed on the drug development support screen 100A from the storage device 30 as the target slice image 15T (step ST150). The target slice image 15T is output from the RW control unit 50 to the detection unit 51 and the delimitation unit 52.

[0084] The RW control unit 50 reads the feature quantity extractor 41 and the setting range information 42 from the storage device 30 and outputs them to the detection unit 51. In addition, the RW control unit 50 reads the reference image 43 from the storage device 30 and outputs it to the delimitation unit 52.

[0085] like Figure 5 As shown in FIG. 1 , the detection unit 51 divides the target slice image 15T into a plurality of block images 65 (step ST160). Figure 6 As shown, in the detection unit 51, the feature amount 68 is extracted from the block image 65 by the feature amount extractor 41 (step ST170).

[0086] like Fig. 9 As shown in FIG. 1 , in the detection unit 51, the difference Δ between the attention feature quantity 68IN of the attention area 80 and the surrounding feature quantity 68S of the surrounding area 81 is calculated. Then, the difference Δ is compared with the setting range of the setting range information 42, and a determination result 82 indicating whether the attention area 80 is an abnormal part is output (step ST180). Based on the determination result 82, a Fig.10 The detection result 60 is shown. The detection result 60 is output from the detection unit 51 to the generation unit 53.

[0087] like Fig.12 As shown, in the delimiting unit 52, the target slice image 15T at the same slice position is aligned with the reference image 43, thereby delimiting a plurality of anatomical parts of the brain B in the target slice image 15T (step ST190). Fig.13 The delimitation result 61 is shown. The delimitation result 61 is output from the delimiting unit 52 to the generating unit 53.

[0088] In the generating unit 53, based on the detection result 60 and the demarcation result 61, the following is generated: Fig.14 The degree information 62 indicating the degree of occurrence of abnormal portions in each of the plurality of anatomical parts is output from the generator 53 to the display controller 54 (step ST200 ).

[0089] like Fig.16As shown, the degree information 62 is displayed on the display 11 through the drug development support screen 100B under the control of the display control unit 54 (step ST210 ). The operator evaluates the candidate substance 26 based on the degree information 62 .

[0090] As described above, the CPU 32 of the drug development support device 10 includes an RW control unit 50, a detection unit 51, a demarcation unit 52, and a display control unit 54. The RW control unit 50 obtains the target slice image 15T obtained by reading the image of the brain B of the subject S from the storage device 30. The detection unit 51 detects the abnormal part estimated to be abnormal based on the target slice image 15T. The demarcation unit 52 demarcates the anatomical part in the target slice image 15T by aligning the reference image 43 having the position information of multiple anatomical parts of the brain B with the target slice image 15T. The display control unit 54 outputs the degree information 62 indicating the degree of occurrence of each abnormal part of the multiple anatomical parts by displaying it on the drug development support screen 100B. Therefore, it is possible to easily grasp the degree of abnormality in each anatomical part of the brain B.

[0091] As a method for delimiting the anatomical part in the target slice image 15T, a method using a relatively simple image recognition technology such as pattern matching can be considered. However, in image recognition technologies such as pattern matching, the accuracy of delimiting the anatomical part is relatively low. Therefore, in the technology of the present invention, the accuracy of delimiting the anatomical part is improved by aligning the reference image 43 and the target slice image 15T and delimiting the anatomical part in the target slice image 15T.

[0092] The detection unit 51 extracts the attention feature amount 68IN of the attention region 80 in the target slice image 15T, and determines whether the attention region 80 is an abnormal portion based on the difference Δ between the attention feature amount 68IN and the surrounding feature amount 68S serving as a reference feature amount. Therefore, it is possible to easily determine whether the attention region 80 is an abnormal portion.

[0093] The abnormal part is a part having a shape and texture different from the surrounding part, such as a tumor, inflammation, cyst, or infiltration. Therefore, if the reference feature quantity is set to the surrounding feature quantity 68S extracted from the surrounding area 81 of the focus area 80, the detection accuracy of the abnormal part can be improved.

[0094] The degree information 62 includes the number of abnormal parts in each of the plurality of anatomical parts and the area ratio of the abnormal parts in each of the plurality of anatomical parts. Therefore, it is possible to more accurately grasp the degree of abnormality occurring in each anatomical part.

[0095] The organ image is a slice image 15 obtained by photographing a brain slice BS. Since the brain slice BS is stained with pigment as a whole, it is difficult to understand the structural features of the brain B in the slice image 15. Therefore, when a relatively simple image recognition technology such as pattern matching is applied, the accuracy of delineating the anatomical part may become very low. Therefore, by delineating the anatomical part by aligning with the reference image 43, the effect of further improving the accuracy of delineating the anatomical part can be exerted.

[0096] In addition, an option for designating all the subjects S constituting the administration group 25 or the control group 27 may be provided in the pull-down menu 102 of the drug development support screen 100A. Fig.18 As shown in the drug development support screen 100B, for example, the degree information 62 in which the slice images 15 of all the subjects S in the drug administration group 25 are generated as the target slice images 15T can be displayed. In this way, the operator can know the degree of abnormality in each anatomical part that is considered to be caused by the candidate substance 26 in the drug administration group 25. In addition, when the control group 27 is selected in the pull-down menu 102, the operator can know the degree of abnormality in each anatomical part that is considered to be caused by the attributes and / or the breeding environment of the subject S rather than the candidate substance 26 in the control group 27.

[0097] Furthermore, an option to simultaneously designate both the administration group 25 and the control group 27 may be provided in the pull-down menu 101 of the drug development support screen 100A. Fig.19 As shown in the drug development support screen 100B, the degree information 62A and 62B may be displayed side by side. The degree information 62A is the degree information 62 in which the slice images 15 of all the subjects S constituting the drug administration group 25 are generated as the target slice images 15T. The degree information 62B is the degree information 62 in which the slice images 15 of all the subjects S constituting the control group 27 are generated as the target slice images 15T. In this way, the operator can easily understand the degree of abnormality that is considered to be caused by the candidate substance 26 and the abnormality that is considered to be caused by the properties of the subject S and / or the breeding environment, etc., in each anatomical part.

[0098] [Second embodiment]

[0099] In the first embodiment described above, the surrounding feature amount 68S extracted from the surrounding area 81 of the target area 80 is exemplified as the reference feature amount, but the present invention is not limited to this.

[0100] As an example, Fig. 20As shown in the second embodiment, the representative feature 68R is used as the reference feature. The representative feature 68R is a representative value, such as an average value, of the plurality of feature 68 extracted from the plurality of block images 65 obtained by subdividing the second slice image 152 obtained from the subject S constituting the control group 27. The processing of the detection unit 51 is the same as that of the first embodiment described above, except that the surrounding feature 68S becomes the representative feature 68R. Fig. 9 The processing shown is the same, so the description is omitted.

[0101] As described above, the control group 27 is a collection of subjects S that have not been administered with the candidate substance 26. In the subjects S that have not been administered with the candidate substance 26, abnormalities caused by the candidate substance 26 naturally do not occur. Therefore, by setting the reference feature amount to the representative feature amount 68R extracted from the second slice image 152 of the subject S that has not been administered with the candidate substance 26, the abnormal part can be detected with high accuracy.

[0102] Alternatively, the representative feature amount 68R of the feature amounts 68 extracted from the second slice images 152 of a plurality of subjects S to which no candidate substance was administered in evaluation tests of candidate substances of drugs developed in the past may be used as a reference feature amount.

[0103] [Implementation Method 3_1]

[0104] As an example, Fig.21 As shown, in the third embodiment, the display control unit 54 controls the display of the drug development support screen 100C on the display 11. The display control unit 54 displays the drug development support screen 100C before the generation unit 53 generates the degree information 62 (before displaying the drug development support screen 100B) or after the generation unit 53 generates the degree information 62 (after displaying the drug development support screen 100B).

[0105] A single target slice image 15T is displayed on the drug development support screen 100C. The anatomical part delimited by the delimiting unit 52 is displayed superimposed on the target slice image 15T. An anatomical part legend 85 is displayed on the right side of the target slice image 15T. In addition, a return button 110 and a send button 111 for switching the display of the target slice image 15T are provided at the bottom of the target slice image 15T. When the OK button 112 is selected, the display of the drug development support screen 100C disappears.

[0106] As described above, in the third first embodiment, the display control unit 54 performs control to superimpose the delimited anatomical part on the target slice image 15T and display the superimposed image on the display 11. Therefore, the operator can confirm the validity of the delimited anatomical part.

[0107] [Implementation Method 3_2]

[0108] As an example, Fig. 22 As shown, in the 3-2nd embodiment, the operator who has demarcated the anatomical part in the drug development support screen 100C can make corrections. Specifically, a correction button 113 is provided at the bottom of the drug development support screen 100C, and while the correction button 113 is selected, the contour of the anatomical part on the target slice image 15T can be corrected using, for example, a mouse cursor 114. When the OK button 112 is selected, the correction of the anatomical part is confirmed.

[0109] As an example, Fig.23 As shown, in the 3-2nd embodiment, the instruction receiving unit 120 receives a correction instruction 121 of the anatomical part made by the operator through the input device 12. The correction instruction 121 includes the position coordinates of the pixels of the contour of the anatomical part corrected by the operator using the cursor 114. The instruction receiving unit 120 outputs the correction instruction 121 to the delineating unit 52. The delineating unit 52 corrects the delineation result 61 according to the correction instruction 121 to make it a corrected delineation result 61C. The delineating unit 52 outputs the corrected delineation result 61C to the generating unit 53.

[0110] The generation unit 53 generates the degree information 62 based on the revised delimitation result 61C. In a mode in which the drug development support screen 100C is displayed before the generation unit 53 generates the degree information 62 (before the drug development support screen 100B is displayed), the degree information 62 displayed on the drug development support screen 100B is the degree information generated from the beginning based on the revised delimitation result 61C. On the other hand, in a mode in which the drug development support screen 100C is displayed after the generation unit 53 generates the degree information 62 (after the drug development support screen 100B is displayed), the degree information 62 displayed on the drug development support screen 100B is switched from the degree information generated based on the delimitation result 61 before the revision to the degree information generated based on the revised delimitation result 61C.

[0111] Thus, in the third-second embodiment, the instruction receiving unit 120 receives the correction instruction 121 of the demarcated anatomical part. Therefore, the anatomical part demarcated by the demarcating unit 52 can be manually corrected by the operator.

[0112] Then, the display control unit 54 outputs the degree information 62 generated based on the anatomical part corrected according to the correction instruction 121. Therefore, the degree information 62 based on the anatomical part manually corrected by the operator can be browsed by the operator.

[0113] In each of the above-mentioned embodiments, a slide specimen 18 carrying only brain slices BS is illustrated, but it is not limited to this. Not only brain slices BS, but also slide specimens in which tissue slices of various organs such as heart slices and bone marrow slices are placed on one slide 16 can be processed. In this case, a recognition unit for identifying the tissue slices of each organ is constructed in the CPU 32, and after the recognition unit recognizes the tissue slices of each organ, subsequent processing such as detection of abnormal parts is performed. In this way, it is also possible to correspond to a slide specimen carrying tissue slices of various organs. As in the above-mentioned first embodiment, a slide specimen carrying tissue slices of various organs is more common than a slide specimen 18 carrying tissue slices of one organ. Therefore, drug development support that is more in line with general applications can be performed.

[0114] The encoder unit 71 of the autoencoder 70 is exemplified as the feature quantity extractor 41, but the present invention is not limited thereto. The encoder unit of a convolutional neural network may be used for the feature quantity extractor 41, and the convolutional neural network may learn to classify the input of the block image 65 into a certain category, for example, to output a determination result of whether an abnormality exists in the brain slice BS shown in the block image 65. In addition, other machine learning models such as a support vector machine may be used for the feature quantity extractor 41.

[0115] In the tissue slice shown in the learning block image 65L, the region that is easily detected as abnormal can be masked and used for learning. Examples of the region that is easily detected as abnormal include blood vessel regions in the brain slice BS and dust attached when the slide specimen 18 is prepared.

[0116] The feature amount 68 is not limited to the feature amount extracted by the feature amount extractor 41. It may be an average value, a maximum value, a minimum value, a mode value, or a dispersion of pixel values ​​of the block image 65.

[0117] The anatomical part is not limited to the cerebral cortex, hippocampus, and amygdala, etc. For example, the hippocampus can be further subdivided into the dentate gyrus, hippocampus, hippocampal peduncle, and entorhinal cortex, etc., and the hippocampus can be further classified into CA1, CA2, and CA3 regions. A plurality of reference images 43 with different classifications of anatomical parts can be stored in advance and used according to the purpose.

[0118] The organ is not limited to the brain B, etc., as exemplified. It may be the stomach, lung, small intestine, large intestine, etc. Furthermore, the subject S is not limited to rats. It may be mice, guinea pigs, gerbils, hamsters, ferrets, rabbits, dogs, cats, monkeys, etc.

[0119] The organ image is not limited to the exemplified slice image 15. It may be a tomographic image based on computed tomography (CT), a tomographic image based on magnetic resonance imaging (MRI), a tomographic image based on positron emission tomography (PET), or a tomographic image based on single photon emission computed tomography (SPECT).

[0120] The output method of the degree information 62 is not limited to the drug development support screen 100B illustrated in the first embodiment, and includes outputting the degree information 62 as a file by printing on a paper medium or attaching it to an e-mail.

[0121] like Figure 1 As shown, the drug development support device 10 may be a personal computer installed in a pharmaceutical facility, or may be a server computer installed in a data center independent of the pharmaceutical facility.

[0122] In the case where the drug development support device 10 is constituted by a server computer, the slice image 15 is sent from a personal computer installed in each pharmaceutical facility to the server computer via a network such as the Internet. The server computer distributes the drug development support screen 100A and the like to the personal computer in the form of network distribution screen data produced by a markup language such as XML (Extensible Markup Language). The personal computer reproduces the drug development support screen 100A and the like displayed on the web browser based on the screen data and displays it on the display. In addition, other data description languages ​​such as JSON (Javascript (registered trademark) Object Notation) can also be used instead of XML.

[0123] The drug development support device 10 according to the technology of the present invention can be widely used in all stages of drug development, from the initial stage, that is, setting of drug development targets, to the final stage, that is, clinical trials.

[0124] The hardware structure of the computer constituting the drug development support device 10 involved in the technology of the present invention can be modified in various ways. For example, in order to improve processing power and reliability, the drug development support device 10 can also be composed of a plurality of computers separated as hardware. For example, two computers are made to disperse the functions of the detection unit 51 and the demarcation unit 52. In this case, the drug development support device 10 is composed of two computers.

[0125] In this way, the hardware structure of the computer of the drug development support device 10 can be appropriately changed according to the required performance of processing power, safety, reliability, etc. In addition, it is not limited to hardware. For the purpose of ensuring safety and reliability, application programs such as the work program 40 can of course be duplicated or stored in multiple storage devices in a distributed manner.

[0126] In each of the above-mentioned embodiments, for example, as the hardware structure of the processing unit (Processing Unit) that performs various processes such as the RW control unit 50, the detection unit 51, the delimiting unit 52, the generating unit 53, the display control unit 54, and the instruction receiving unit 120, various processors (Processor) shown below can be used. As described above, among the various processors, in addition to the general-purpose processor CPU32 that executes software (work program 40) to function as various processing units, there are also processors such as FPGA (Field Programmable Gate Array) whose circuit structure can be changed after manufacturing, such as programmable logic devices (PLD), ASIC (Application Specific Integrated Circuit), and other processors with circuit structures specially designed for performing specific processes, such as dedicated circuits.

[0127] One processing unit may be composed of one of these various processors, or a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs and / or a combination of a CPU and an FPGA). Furthermore, multiple processing units may be composed of one processor.

[0128] As an example of a plurality of processing units constituted by one processor, there is the following method: as represented by computers such as client terminals and servers, one processor is constituted by a combination of one or more CPUs and software, and the processor functions as a plurality of processing units. Second, as represented by a system on chip (SoC), there is the following method: a processor that realizes the functions of the entire system including a plurality of processing units by one IC (Integrated Circuit) chip is used. In this way, various processing units are constituted as a hardware structure using one or more of the above-mentioned various processors.

[0129] Furthermore, as the hardware configuration of these various processors, more specifically, a circuit (circuitry) formed by combining circuit elements such as semiconductor elements can be used.

[0130] The technology of the present invention can also appropriately combine the above-mentioned various embodiments and / or various variations. Moreover, it is not limited to the above-mentioned embodiments, and various structures can of course be adopted as long as it does not deviate from the main purpose. In addition, the technology of the present invention also relates to a storage medium that stores the program non-temporarily in addition to the program.

[0131] The recorded contents and illustrated contents shown above are detailed descriptions of the parts involved in the technology of the present invention, which are only an example of the technology of the present invention. For example, the descriptions related to the above-mentioned structure, function, action and effect are descriptions related to an example of the structure, function, action and effect of the parts involved in the technology of the present invention. Therefore, without departing from the scope of the technical purpose of the present invention, it is of course possible to delete unnecessary parts of the recorded contents and illustrated contents shown above, or to add or replace new elements. In addition, in order to avoid complicated situations and facilitate understanding of the parts involved in the technology of the present invention, the descriptions of technical common sense that does not particularly need to be explained in terms of enabling the implementation of the technology of the present invention are omitted in the recorded contents and illustrated contents shown above.

[0132] In this specification, "A and / or B" has the same meaning as "at least one of A and B". That is, "A and / or B" may mean only A, only B, or a combination of A and B. Furthermore, in this specification, the same idea as "A and / or B" is also applicable to the case of expressing three or more items by connecting them with "and / or".

[0133] All documents, patent applications, and technical standards described in this specification are incorporated herein by reference to the same extent as if each document, patent application, and technical standard was specifically and individually described by reference.

Claims

1. An information processing device comprising a processor, The processor performs the following processing: acquiring an organ image obtained by photographing the organ of the subject; Detecting an abnormal part estimated to be abnormal based on the organ image; Delimiting the anatomical part in the organ image by aligning a reference image having position information of a plurality of anatomical parts of the organ with the organ image; and The degree information indicating the degree of occurrence of the abnormal portion in each of the plurality of anatomical parts is output.

2. The information processing device according to claim 1, wherein: The processor performs the following processing: extracting a feature quantity of a region of interest in the organ image; and Whether the region of interest is the abnormal portion is determined based on a difference between a feature amount extracted from the region of interest and a reference feature amount.

3. The information processing device according to claim 2, wherein: The reference feature quantity is a feature quantity extracted from a region surrounding the target region.

4. The information processing device according to claim 2, wherein: The organ image is an image used in an evaluation test of a candidate substance for evaluating a drug, The reference feature quantity is a feature quantity extracted from an organ image of a subject to which the candidate substance has not been administered.

5. The information processing device according to any one of claims 1 to 4, wherein: The processor performs the following control: The delineated anatomical part is superimposed on the organ image and displayed on a display.

6. The information processing device according to claim 5, wherein: The processor receives an indication of a revision of the delineated anatomical region.

7. The information processing device according to claim 6, wherein: The processor outputs the degree information generated based on the anatomical part corrected according to the correction instruction.

8. The information processing device according to any one of claims 1 to 7, wherein: The degree information includes the number of the abnormal parts in each of the plurality of anatomical parts and the area ratio of the abnormal parts in each of the plurality of anatomical parts.

9. The information processing device according to any one of claims 1 to 8, wherein: The organ image is a slice image obtained by photographing a tissue slice of the organ.

10. A working method of an information processing device, comprising the following steps: acquiring an organ image obtained by photographing the organ of the subject; Detecting an abnormal part estimated to be abnormal based on the organ image; demarcating the anatomical part in the organ image by aligning a reference image having position information of a plurality of anatomical parts of the organ with the organ image; and The degree information indicating the degree of occurrence of the abnormal portion in each of the plurality of anatomical parts is output.

11. A working program of an information processing device, which causes a computer to execute a process comprising the following steps: acquiring an organ image obtained by photographing the organ of the subject; Detecting an abnormal part estimated to be abnormal based on the organ image; demarcating the anatomical part in the organ image by aligning a reference image having position information of a plurality of anatomical parts of the organ with the organ image; and The degree information indicating the degree of occurrence of the abnormal portion in each of the plurality of anatomical parts is output.