Information processing device, information processing method, and program

Abstract

To improve detection accuracy of an abnormality candidate region such as a pancreas cancer.SOLUTION: An information processing device includes an image data acquisition unit, a travelling line acquisition unit, a feature amount acquisition unit, a feature amount change acquisition unit, and an abnormality candidate region identification unit. The image data acquisition unit acquires three-dimensional tomographic image data imaging the pancreas. The travelling line acquisition unit acquires travelling line data capable of identifying a travelling line travelling in the pancreas corresponding to the three-dimensional tomographic data. The feature amount acquisition unit acquires a feature amount along the travelling line corresponding to the three-dimensional tomographic image data. The feature amount change acquisition unit acquires an amount of change between the plurality of feature amounts along the travelling line. The abnormality candidate region identification unit identifies an abnormality candidate region in the vicinity of the travelling line in the pancreas on the basis of the amount of change.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The embodiments disclosed in this specification and the drawings relate to an information processing device, an information processing method, and a program. [Background technology]

[0002] In general, pancreatic cancer is difficult to detect and has a very poor prognosis. Therefore, there is a need to establish a diagnostic support technology for early detection. In recent years, with the development of deep learning, research on technology to detect areas suspected of pancreatic cancer from medical images is increasing. For example, as in Non-Patent Document 1, a method of segmenting areas suspected of pancreatic cancer using CNN, which is one type of deep learning, has been proposed.

[0003] However, pancreatic cancer is often unclear in medical images, and even methods using deep learning have the problem of failing to detect areas suspected of pancreatic cancer. On the other hand, indirect findings that are important for early detection of pancreatic cancer include dilation or obstruction of the main pancreatic duct and stenosis of the pancreas. Using this information may improve the accuracy of detection of unclear pancreatic cancer. [Prior art documents] [Non-patent literature]

[0004] [Non-Patent Document 1] Zhuotun Zhu et al., “Multi-Scale Coarse-to-Fine Segmentation for Screening Pancreatic Ductal Adenocarcinoma”, arXiv:1807.02941 [cs.CV], 2019. [Non-Patent Document 2] C. Hattori et al., “Centerline detection and estimation of pancreatic duct from abdominal CT images”, Proc. SPIE 12032, Medical Imaging 2022. Summary of the Invention [Problem to be solved by the invention]

[0005] One of the problems to be solved by the embodiments disclosed in this specification and the drawings is to improve the accuracy of detecting abnormal candidate regions such as pancreatic cancer. However, the problems to be solved by the embodiments disclosed in this specification and the drawings are not limited to the above problem. Problems corresponding to the effects of each configuration shown in the embodiments described later can also be positioned as other problems. [Means for solving the problem]

[0006] An information processing device according to an embodiment includes an image data acquisition unit, a driving line acquisition unit, a feature acquisition unit, a feature change acquisition unit, and an abnormal candidate area identification unit. The image data acquisition unit acquires three-dimensional tomographic image data obtained by capturing an image of the pancreas. The driving line acquisition unit acquires driving line data capable of identifying a driving line that runs through the pancreas corresponding to the three-dimensional tomographic image data. The feature acquisition unit acquires feature amounts along the driving line corresponding to the three-dimensional tomographic image data. The feature change acquisition unit acquires change amounts between a plurality of feature amounts along the driving line. The abnormal candidate area identification unit identifies an abnormal candidate area in the pancreas near the driving line based on the change amounts. [Brief description of the drawings]

[0007] [Figure 1] FIG. 1 is a diagram illustrating an example of a functional configuration of an information processing device according to the first embodiment. [Diagram 2] FIG. 2 is a diagram illustrating an example of a hardware configuration of the information processing device according to the first embodiment. [Diagram 3]FIG. 3 is a diagram illustrating CT image data according to the first embodiment. [Figure 4] FIG. 4 is a diagram illustrating the driving lines held by the driving line data according to the first embodiment. [Diagram 5] FIG. 5 is a flowchart illustrating an example of a processing procedure of the information processing device according to the first embodiment. [Figure 6] FIG. 6 is a flowchart illustrating an example of a procedure of the feature amount calculation process according to the first embodiment. [Figure 7] FIG. 7 is an example of a graph in which the amount of change in a feature quantity is plotted against the feature quantity along a driving line according to the first embodiment. [Figure 8] FIG. 8 is an example of a graph in which the amount of change in a feature quantity is plotted against the feature quantity along a driving line according to the first embodiment. [Figure 9] FIG. 9 is a diagram showing an example of a display form by the information processing device according to the first embodiment. [Figure 10] FIG. 10 is a diagram illustrating an example of a functional configuration of the information processing device according to the second embodiment. [Figure 11] FIG. 11 is a flowchart illustrating an example of a processing procedure of the information processing device according to the second embodiment. [Figure 12] FIG. 12 is a flowchart illustrating an example of a procedure for feature amount calculation processing according to the second embodiment. [Figure 13] FIG. 13 is a diagram illustrating an example of a functional configuration of the information processing device according to the third embodiment. [Figure 14] FIG. 14 is a flowchart illustrating an example of a processing procedure of the information processing device according to the third embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0008] Hereinafter, an embodiment of an information processing device, an information processing method, and a program will be described in detail with reference to the drawings. The same or equivalent components, members, and processes shown in each drawing will be given the same reference numerals, and duplicated descriptions will be omitted as appropriate. In addition, some of the components, members, and processes will be omitted as appropriate in each drawing.

[0009] In the following embodiments, CT image data captured by an X-ray computed tomography (X-ray CT (Computed Tomography)) device will be used as an example of three-dimensional tomographic image data. Note that the embodiments of the present invention are not limited to the following embodiments, and can be applied to images captured by, for example, a Magnetic Resonance Imaging (MRI) device, a Positron Emission Tomography (PET) device, and an ultrasound diagnostic device.

[0010] <First embodiment> (overview) In this embodiment, a method will be described in which image feature amounts are calculated along a line that runs through the pancreas in CT image data, and the feature amounts are used to detect abnormal candidate regions and abnormal regions.

[0011] More specifically, for each pixel of the CT image data on the travel line, the average brightness (CT value) calculated by a window of any size is acquired as a feature value along the travel line in the pancreas. Then, for the feature values ​​along the travel line, a first derivative between the feature values ​​is calculated, and when the absolute value of the first derivative is equal to or greater than a predetermined threshold value, the region is detected as an abnormal candidate region. Here, the absolute value of the first derivative is, in other words, the amount of change between the feature values. In this embodiment, the abnormal candidate region is further determined to be an abnormal region or not by a recognizer, and the abnormal region is identified. The average brightness is an example of a brightness statistic in this embodiment. The recognizer is an example of an inference model based on machine learning in this embodiment.

[0012] (Functional configuration) Hereinafter, the functional configuration of the information processing device 100 according to this embodiment will be described with reference to FIG. 1. As shown in the figure, the information processing device 100 includes an image data acquisition unit 110, a driving line acquisition unit 120, a feature acquisition unit 130, a feature change acquisition unit 140, an abnormality candidate area identification unit 150, an image recognition unit 160, and a display control unit 170. The information processing device 100 according to this embodiment also includes a storage device 70 and a display unit 80 externally. The storage device 70 and the display unit 80 may be included in the configuration of the information processing device 100. The storage device 70 may be provided within the information processing device 1000.

[0013] The storage device 70 is an example of a computer-readable storage medium, and is a non-transient large-capacity storage device such as a hard disk drive (HDD) or a solid-state drive (SSD). The storage device 70 stores CT image data and pancreatic travel line data corresponding to the CT image data.

[0014] Here, the CT image data and the running line data according to this embodiment will be described with reference to Figures 3 and 4. In Figures 3 and 4, the running line of the pancreas is shown on the drawing from the head to the tail of the pancreas, but this is only a simplified representation for the purpose of illustration, and the actual running line is a three-dimensional curve and does not necessarily exist on the same plane.

[0015] FIG. 3 is a diagram for explaining CT image data 300 according to the present embodiment. In the CT image data 300, a pancreas 301 is depicted, with the left side of the figure representing the head side of the pancreas and the right side representing the tail side of the pancreas. In addition, in the CT image data 300, a main pancreatic duct 303 and a pancreatic cancer 302 are depicted in the pancreas 301. In the main pancreatic duct 303, pancreatic juice flows from the tail side of the pancreas to the head side of the pancreas. In the case shown in FIG. 3, the pancreatic cancer 302 obstructs the main pancreatic duct 303, causing the flow of pancreatic juice in the main pancreatic duct 303 to deteriorate, and the main pancreatic duct 303 expands until it is clearly depicted at the tail side of the pancreas, and further, the main pancreatic duct 303 is interrupted just before the pancreatic cancer 302. In general, pancreatic cancer and the inside of the main pancreatic duct are depicted with a lower CT value than normal pancreatic parenchyma, so that in the CT image data 300 shown in FIG. 3, these tissues (the main pancreatic duct 303 and the pancreatic cancer 302) are depicted in a darker color than the pancreatic parenchyma. In addition, Fig. 3 shows localized stenosis of the pancreas 301 in the area around the pancreatic cancer 302 indicated by the arrow. Since dilation of the main pancreatic duct and localized stenosis of the pancreas are indirect findings of pancreatic cancer, it is expected that the detection performance of pancreatic cancer can be improved by utilizing this information.

[0016] FIG. 4 is a diagram for explaining the traveling line 401 held by the traveling line data according to this embodiment. In this embodiment, "the traveling line data holds the traveling line 401" means, in other words, that the traveling line data includes information capable of identifying the traveling line 401. FIG. 4 shows a state in which the traveling line 401 is superimposed on the CT image data 300. The traveling line 401 is a line that passes through approximately the center of the pancreas 301 so as to connect the tail and head of the pancreas 301. The traveling line 401 is also called a core line or a center line. As shown in FIG. 4, the traveling line 401 in this embodiment passes through the inside of the main pancreatic duct 303. In the traveling line data, data is held in a format capable of expressing the traveling line 401 as described above. For example, the traveling line data holds three-dimensional coordinate values ​​in an image coordinate system of a point group constituting the traveling line 401, and a control point group of a curve such as a spline curve. In addition, the traveling line data may be image data in which pixels belonging to the traveling line and pixels other than the traveling line are expressed by different pixel values. This driving line data is, for example, automatically created by another information processing device configured to perform a known image recognition method on CT image data, or semi-automatically created by manual operation by a doctor or the like.

[0017] Returning to FIG. 1, the image data acquisition unit 110 acquires CT image data of an image of the pancreas from the storage device 70 , and transmits the data to the feature amount acquisition unit 130 , the image recognition unit 160 , and the display control unit 170 .

[0018] The driving line acquisition unit 120 acquires driving line data of the pancreas corresponding to the CT image data from the storage device 70 and transmits it to the feature amount acquisition unit 130.

[0019] The feature amount acquiring unit 130 acquires feature amounts along the driving line corresponding to the CT image data. More specifically, the feature amount acquiring unit 130 first receives CT image data from the image data acquiring unit 110, and receives driving line data of the pancreas corresponding to the CT image data from the driving line acquiring unit 120. Next, the feature amount acquiring unit 130 calculates feature amounts along the driving line from the CT image data based on the driving line data. Then, the feature amount acquiring unit 130 transmits the feature amounts along the driving line to the feature amount change acquiring unit 140.

[0020] The feature amount change acquisition unit 140 acquires the amount of change between multiple feature amounts along the driving line. More specifically, the feature amount change acquisition unit 140 first receives the feature amounts along the driving line from the feature amount acquisition unit 130. Next, the feature amount change acquisition unit 140 acquires the amount of change in the feature amounts by calculating the amount of change between local feature amounts for the feature amounts along the driving line. Then, the feature amount change acquisition unit 140 transmits the amount of change in the feature amounts to the abnormality candidate area identification unit 150.

[0021] The abnormal candidate region identifying unit 150 identifies an abnormal candidate region near the driving line in the pancreas based on the feature amount. More specifically, the abnormal candidate region identifying unit 150 identifies an abnormal candidate region based on the change amount of the feature amount acquired by the feature amount change acquiring unit 140. The abnormal candidate region identifying unit 150 first receives the change amount of the feature amount from the feature amount change acquiring unit 140. Next, the abnormal candidate region identifying unit 150 identifies a change point where the change amount of the feature amount satisfies a predetermined condition, and identifies (detects) an area including the change point as an abnormal candidate region. Then, the abnormal candidate region identifying unit 150 generates image data (abnormal candidate region image data) representing the abnormal candidate region, and transmits the abnormal candidate region image data to the image recognition unit 160. The predetermined condition used for identifying the change point is, for example, that the change amount of the feature amount is equal to or greater than a threshold value. Details of identifying the change point will be described later in the flowchart of FIG. 5.

[0022] Here, the abnormal candidate region image data will be described. The abnormal candidate region image data is image data that indicates whether each pixel is included in the abnormal region candidate or not, and is, for example, a binary image of the same image size as the CT image data in which the pixel value of the pixel included in the abnormal region candidate is expressed as 1 and the pixel value of the pixel not included in the abnormal region candidate is expressed as 0. The abnormal candidate region image data may be in any format as long as the abnormal candidate region depicted on the CT image data can be identified, and may be likelihood image data in which the pixel value of each pixel indicates the likelihood of the abnormal candidate region. The abnormal candidate region image data may also be multi-value image data in which the pixel value of each pixel indicates the region of various tissues including the abnormal candidate region in multi-values. The abnormal candidate region image data may also be coordinate value data of the vertices when the abnormal candidate region is approximated to a polygon.

[0023] The image recognition unit 160 first receives CT image data from the image data acquisition unit 110, and receives abnormal candidate region image data from the abnormal candidate region identification unit 150. Next, the image recognition unit 160 uses a recognizer configured to be able to determine whether an abnormal candidate region on the CT image data is an abnormal region to determine whether the abnormal candidate region identified by the abnormal candidate region identification unit 150 is an abnormal region, and identifies the abnormal region. When the recognizer receives input of partial image data of the CT image data including an abnormal candidate region, the recognizer outputs a determination result as to whether the region depicted in the abnormal candidate region image data is an abnormal region. The image recognition unit 160 inputs the abnormal candidate region image data to the recognizer, and obtains the determination result output from the recognizer. The unit for determining whether or not there is an abnormality by the recognizer may be for each pixel, or may be for each region including a specified number of multiple pixels.

[0024] Then, the image recognition unit 160 generates image data (abnormal area image data) representing the identified abnormal area, and transmits the abnormal area image data to the display control unit 170. Note that the abnormal area image data is image data that holds the abnormal area in the same format as the above-mentioned abnormal candidate area image data. Here, "image data holds an abnormal area" means, in other words, that the image data includes information that can identify the abnormal area.

[0025] In this embodiment, the recognizer is, for example, a trained CNN (Convolutional Neural Network). The CNN is trained by a known method (backpropagation method, etc.) using, for example, partial image data (input data) including an abnormal candidate region and teacher data consisting of data (ground truth data) indicating whether the region is abnormal or not. The image recognition unit 160 stores, in the abnormal region image data, only the region that is determined to be an abnormal region by the above-mentioned recognizer with respect to the abnormal candidate region. The recognizer may be stored in the storage device 70 separately from the image recognition unit 160, or may be incorporated in the image recognition unit 160.

[0026] The display control unit 170 causes at least one of the abnormal candidate region and the image recognition result, which is the result of image recognition for the abnormal candidate region, to be identifiable on the display unit 80. For example, the display control unit 170 first receives CT image data from the image data acquisition unit 110, and receives abnormal region image data, which is the image recognition result, from the image recognition unit 160. Then, the display control unit 170 causes the CT image data and the abnormal region image data to be displayed on the display unit 80.

[0027] (Hardware configuration) Next, the hardware configuration of the information processing device 100 will be described with reference to Fig. 2. Fig. 2 is a diagram showing an example of the hardware configuration of the information processing device 100 according to the first embodiment. The information processing device 100 has the configuration of a known computer (information processing device). The information processing device 100 includes, as its hardware configuration, a CPU 201, a main memory 202, a magnetic disk 203, a display memory 204, a monitor 205, a mouse 206, and a keyboard 207.

[0028] A CPU (Central Processing Unit) 201 mainly controls the operation of each component. A main memory 202 stores a control program executed by the CPU 201 and provides a working area when the CPU 201 executes the program. A magnetic disk 203 stores programs for implementing various application software including an OS (Operating System), device drivers for peripheral devices, and programs for performing processes described below. The CPU 201 executes programs stored in the main memory 202, the magnetic disk 203, etc., thereby implementing the functions (software) of the information processing device 100 shown in FIG. 1 and processes in flowcharts described below. The magnetic disk 203 may be the same as the storage device 70.

[0029] The display memory 204 temporarily stores data for display. The monitor 205 is, for example, a CRT monitor or a liquid crystal monitor, and displays images, text, and the like based on data from the display memory 204. The mouse 206 and the keyboard 207 convert input operations received from the user into electrical signals and output the signals to the CPU 201, thereby allowing the user to perform pointing input and input of characters, and the like, respectively. The above components are connected to each other via a common bus 208 so as to be able to communicate with each other. The monitor 205 may be the same as the display unit 80. The mouse 206 and the keyboard 207 are examples of input interfaces of the information processing device 100, and may be replaced by other means. For example, the information processing device 100 may include input interfaces such as a trackball, a switch button, a touch pad that performs input operations by touching the operation surface, a touch screen in which a display screen and a touch pad are integrated, a non-contact input circuit using an optical sensor, and a voice input circuit. The input interface is not limited to one that includes physical operation parts. For example, the information processing device 100 may include, as an input interface, an electrical signal processing circuit that receives an electrical signal corresponding to an input operation from an external input device provided separately from the information processing device 100 and outputs this electrical signal to a control circuit.

[0030] The CPU 201 corresponds to an example of a processor or a control unit. The processor included in the information processing device 100 is not limited to the CPU 201. For example, in addition to or instead of the CPU 201, the information processing device 100 may have at least one of circuits such as a GPU (Graphics Processing Unit), an Application Specific Integrated Circuit (ASIC), a programmable logic device (for example, a Simple Programmable Logic Device (SPLD), a Complex Programmable Logic Device (CPLD), and a Field Programmable Gate Array (FPGA)). If the processor is an ASIC, instead of storing a program in the main memory 202, the magnetic disk 203, or the like, each function shown in FIG. 1 (image data acquisition unit 110, driving line acquisition unit 120, feature amount acquisition unit 130, feature amount change acquisition unit 140, abnormality candidate area identification unit 150, image recognition unit 160, and display control unit 170) is directly incorporated as a logic circuit in the circuit of the processor. Note that each processor in this embodiment is not limited to being configured as a single circuit for each processor, and may be configured as a single processor by combining multiple independent circuits to realize its function. Furthermore, multiple components may be integrated into a single processor to realize its function. The main memory 202 and the magnetic disk 203 correspond to examples of memory or storage device.

[0031] (Processing Procedure) Next, a processing procedure of the information processing device 100 according to this embodiment will be described with reference to Fig. 5 and Fig. 6. Fig. 5 is a flowchart showing an example of a processing procedure of the information processing device 100 according to the first embodiment. Fig. 6 is a flowchart showing an example of a procedure of a feature amount calculation process according to the first embodiment.

[0032] (Step S510) In step S510, the image data acquisition unit 110 acquires CT image data from the storage device 70. Then, the image data acquisition unit 110 transmits the CT image data to the feature amount acquisition unit 130 and the display control unit 170.

[0033] (Step S520) In step S520, the driving line acquisition unit 120 acquires driving line data from the storage device 70. Then, the driving line acquisition unit 120 transmits the driving line data to the feature acquisition unit 130. In this embodiment, the driving line data is three-dimensional coordinate values ​​of each of the points constituting the driving line 401. In addition, the order of the points in the driving line data does not matter, but in this embodiment, data arranged in order from the head to the tail of the pancreas is described as an example.

[0034] (Step S530) In step S530, the feature amount acquisition unit 130 receives the CT image data 300 from the image data acquisition unit 110, and receives the pancreas travel line data corresponding to the CT image data 300 from the travel line acquisition unit 120. Next, the feature amount acquisition unit 130 calculates feature amounts along the travel line 401 from the CT image data 300 based on the travel line data. The feature amount is a value based on the pixel value on the travel line 401. More specifically, in this embodiment, the feature amount along the travel line is an average value of brightness calculated for each pixel on the travel line by a window of any size. Although the window can be set arbitrarily, in this embodiment, a case where a three-dimensional rectangular window centered on a pixel on the travel line is used as an example will be described. Then, the feature amount acquisition unit 130 transmits the feature amount along the travel line to the feature amount change acquisition unit 140. Here, the feature amount along the travel line refers to a feature amount set in which the feature amounts calculated for each pixel on the travel line are arranged in order along the travel line.

[0035] Here, the feature amount calculation process in step S530 in this embodiment will be described in detail with reference to the flowchart in FIG.

[0036] (Step S531) In step S531, the feature acquisition unit 130 sets an arbitrary pixel on the driving line as a pixel of interest. In this embodiment, an example will be described in which feature amounts for pixels on the driving line are calculated in order from the head of the pancreas to the tail of the pancreas. Therefore, the feature acquisition unit 130 first sets a pixel on the end of the driving line on the head side of the pancreas as a pixel of interest. Thereafter, when this processing step is executed again after the processing of steps S532 and S533, a pixel adjacent to the pixel set above is set as a pixel of interest. That is, in step S530, the processing from step S531 to step S533 is executed repeatedly for the number of pixels on the driving line.

[0037] (Step S532) In step S532, the feature acquisition unit 130 calculates a feature for the pixel of interest set in step S531. In this embodiment, the feature is calculated based on pixels of the CT image included in a window of any size centered on the pixel of interest, and specifically, a three-dimensional rectangular window with 5 pixels on a side is used. As a method of calculating the feature, a case in which the average brightness value of the pixels included in the window is used as the feature will be described. The shape and size of the window may be set arbitrarily depending on the characteristics of the image. Using the average brightness value as the feature is merely one example of the embodiment of the present invention, and the feature may be calculated by any method.

[0038] (Step S533) In step S533, the feature acquisition unit 130 determines whether or not the next pixel on the driving line exists, and if the next pixel does not exist (S533 "No"), the step S530 is terminated, and if not (S533 "Yes"), the process returns to step S531 and the next pixel is set as the pixel of interest. That is, when the feature acquisition unit 130 has calculated the feature for all pixels on the driving line, the step S530 is terminated.

[0039] In this embodiment, the feature values ​​for the pixels on the running line are calculated in order from the head to the tail of the pancreas, so the next pixel is a pixel that is further in the direction toward the tail of the pancreas than the current pixel of interest. Also, the pixel of interest at the end of step S530 is the pixel on the running line that is at the end of the tail of the pancreas.

[0040] Here, the relationship between the pixel position relative to the driving line and the value of the feature amount will be described with reference to FIG. 7. FIG. 7 is an example of a graph in which the change in the feature amount is plotted against the feature amount along the driving line according to the first embodiment. In the above-mentioned step S530, the feature amount acquisition unit 130 obtains the feature amount along the driving line as shown in FIG. 7. In this embodiment, the value of the feature amount in the figure corresponds to the average brightness value of pixels within a window of a predetermined range centered on a pixel on the driving line. In particular, the section A and the section B are feature amounts (average brightness values) corresponding to the region of pancreatic cancer 302 and the region of the main pancreatic duct 303, respectively.

[0041] (Step S540) 5, in step S540, the feature amount change acquisition unit 140 first receives feature amounts along the driving line from the feature amount acquisition unit 130. Next, the feature amount change acquisition unit 140 acquires the amount of change in the feature amounts by calculating the amount of change between local feature amounts for the feature amounts along the driving line. Then, the feature amount change acquisition unit 140 transmits the amount of change in the feature amounts to the abnormal candidate area identification unit 150.

[0042] In this embodiment, the feature amount change acquisition unit 140 calculates the absolute value of the first derivative between feature amounts along the driving line, and acquires this as the amount of change in the feature amount. That is, the feature amount change acquisition unit 140 acquires the absolute value of the difference between feature amounts for adjacent pixels on the driving line as the amount of change in the feature amount. That is, the feature amount change acquisition unit 140 acquires the amount of change between local feature amounts for feature amounts along the driving line.

[0043] Here, the change in the feature amount will be described with reference to Fig. 8. Fig. 8 is an example of a graph in which the change in the feature amount is plotted against the feature amount (average brightness value) along the driving line calculated in step S530. The change in the feature amount is high at the left end of section B on the graph (the boundary between the main pancreatic duct 303 and the pancreatic parenchyma), and is then high at both ends of section A (the boundary between the pancreatic cancer 302 and the pancreatic parenchyma).

[0044] (Step S550) 5, in step S550, the abnormal candidate area identifying unit 150 first receives the amount of change in the feature amount from the feature amount change acquisition unit 140. Next, the abnormal candidate area identifying unit 150 identifies a change point where the amount of change in the feature amount satisfies a predetermined condition, and identifies an area including the change point as an abnormal candidate area. Then, the abnormal candidate area identifying unit 150 generates abnormal candidate area image data, and transmits the abnormal candidate area image data to the image recognition unit 160.

[0045] Here, a predetermined condition used for identifying the change point will be described. In general, pancreatic cancer that blocks the main pancreatic duct is present near the position where the main pancreatic duct becomes invisible (the blocked position of the main pancreatic duct) when the main pancreatic duct depicted on the CT image data is observed from the tail to the head of the pancreas. Furthermore, it is a typical example that it is present at a position closer to the head of the pancreas than the position. That is, the positional relationship is like that of the pancreatic cancer 302 on the CT image data 300 shown in FIG. 3 and the expanded main pancreatic duct 303. Therefore, the abnormal candidate region identifying unit 150 in this embodiment first identifies a pixel corresponding to a position (change point) where the change amount of the average brightness (feature amount) along the travel line is equal to or greater than a threshold value. This corresponds to identifying the "position where the main pancreatic duct becomes invisible" in the above description. In the example shown in FIG. 8, the end of section B is identified as the change point by the judgment based on the threshold value, but depending on the threshold value, the end of section A where pancreatic cancer exists may be identified as the change point. Next, the abnormal candidate region identifying unit 150 sets a pixel that is a predetermined distance away from the pixel corresponding to the change point (for example, a pixel that is 10 mm away from the pancreas head side along the driving line) as a reference pixel of the abnormal candidate region. Then, the abnormal candidate region identifying unit 150 identifies an ROI (Region of Interest) centered on the reference pixel of the abnormal candidate region as the abnormal candidate region. The ROI may have any shape or size, but in this embodiment, the shape of the ROI is a three-dimensional rectangular shape, and a preset value is used for the size. For example, the abnormal candidate region identifying unit 150 sets the size of the ROI based on the statistical value of the size of pancreatic cancer. Based on the abnormal candidate region identified by the above processing, the abnormal candidate region identifying unit 150 generates abnormal candidate region image data that can identify the corresponding region of the CT image data. In this embodiment, the abnormal candidate region image data is binary image data in which the abnormal candidate region is expressed by a pixel value of 1 and the other regions are expressed by a pixel value of 0.

[0046] (Step S560) In step S560, the image recognition unit 160 first receives CT image data from the image data acquisition unit 110, and receives abnormal candidate region image data from the abnormal candidate region identification unit 150. Next, the image recognition unit 160 uses a recognizer configured to be able to determine whether an abnormal candidate region is an abnormal region to determine whether the abnormal candidate region identified by the abnormal candidate region identification unit 150 is an abnormal region, and identifies the abnormal region. Then, the image recognition unit 160 generates abnormal region image data and transmits the abnormal region image data to the display control unit 170. The image recognition unit 160 stores only the regions that are determined to be abnormal by the recognizer in the abnormal region image data from among the candidate abnormal regions. In this embodiment, the abnormal region image data is binary image data in which the abnormal region is represented by a pixel value of 1 and the other regions are represented by a pixel value of 0.

[0047] (Step S570) In step S570, the display control unit 170 first receives the CT image data from the image data acquisition unit 110, and receives the abnormal area image data from the image recognition unit 160. Then, the display control unit 170 causes the display unit 80 to display the CT image data and the abnormal area image data.

[0048] In this embodiment, the display control unit 170 causes the display unit 80 to display a rectangular frame surrounding the abnormal region (ROI) held by the abnormal region image data so as to be superimposed on the CT image data 300. Here, FIG. 9 is a diagram showing an example of a display form by the information processing device 100 according to the first embodiment. FIG. 9 shows an example in which the display control unit 170 displays the CT image data 300 and the abnormal region 901 on the display unit 80. In this way, the display control unit 170 causes the abnormal region to be identifiable on the CT image data. Here, the processing of the flowchart in FIG. 5 ends.

[0049] Through the above process, the information processing device 100 calculates the amount of change in the luminance value of pixels on the driving line based on the CT image data and the driving line data of the pancreas, and identifies an abnormal candidate region in the pancreas based on the amount of change. Then, it is determined whether the identified abnormal candidate region is abnormal or not. In this way, it is possible to capture changes in the pancreas, such as indirect findings of pancreatic cancer, and it is expected that the detection performance of unclear pancreatic cancer can be improved.

[0050] That is, according to the information processing device 100 of this embodiment, feature amounts along a driving line running through the pancreas corresponding to CT image data, which is three-dimensional tomographic image data of the pancreas, are acquired, and candidate abnormal regions near the driving line in the pancreas are identified based on the feature amounts, thereby improving the accuracy of detecting candidate abnormal regions such as pancreatic cancer.

[0051] (Variation of the first embodiment) In the above description, the driving line acquisition unit 120 acquires a driving line corresponding to the CT image data from the storage device 70, but the present invention is not limited to this. For example, the driving line acquisition unit 120 may estimate and acquire the driving line from the CT image data using an image recognition method or the like disclosed in Non-Patent Document 2.

[0052] In the above description, the feature acquisition unit 130 acquires the feature for the pixel on the driving line using a three-dimensional window of any size as the feature, but is not limited thereto. For example, the feature may be acquired using a one-dimensional window or two-dimensional window of any size, or the pixel value of the pixel on the driving line itself may be acquired as the feature without using a window. When using a one-dimensional window, for example, a window may be set in the tangential direction of the driving line to acquire the feature in the window, or the feature may be acquired from the pixel in the window along the driving line (curve). When calculating the feature from the pixel in the window along the driving line, in step 532, the feature may be calculated based on the pixel values ​​of several pixels before and after the pixel of interest on the driving line, with the pixel of interest as the center. When using a two-dimensional window, for example, a window may be set for a cross section along the tangential direction of the driving line (a cross section stretched by a tangential axis and a direction axis perpendicular to the tangent) to acquire the feature. In addition, a window may be set for at least one cross section intersecting the driving line (or a cross section perpendicular to the tangential axis) to acquire the feature. Furthermore, for each pixel on the driving line, the result of integrating the respective feature amounts acquired in the above-mentioned multiple windows with different spatial dimensions may be used as the feature amount. The window set for the cross section intersecting the driving line is an example of cross-sectional image data intersecting the driving line.

[0053] In the above description, the feature amount acquiring unit 130 acquires the average brightness value as the feature amount, but any feature amount may be used as long as it is a statistical value of brightness in the vicinity of the driving line. For example, the minimum value, maximum value, median value, variance value, and weighted average value of brightness in the vicinity of pixels on the driving line may be acquired as the feature amount. Alternatively, the brightness distribution may be expressed by a histogram, and a vector of the same dimension as the number of bins of the histogram may be used as the feature amount. In this case, the feature amount change acquiring unit 140 may acquire the distance between the vectors as the change amount of the feature amount. In addition, a plurality of the above-mentioned feature amounts may be combined.

[0054] The feature amount acquiring unit 130 may calculate the feature amount using any filter. For example, in order to reduce the influence of noise near the driving line, the feature amount may be calculated using a smoothing filter such as a Gaussian filter. Alternatively, in order to obtain a feature amount in which image edges are emphasized, a filter that calculates a brightness gradient such as a Sobel filter or a Laplacian filter may be used, or in order to extract a specific pattern, the feature amount may be calculated using a Gabor filter or the like. In addition, in order to obtain a feature amount in which a local structure (e.g., the likeliness of a linear structure) within a window is emphasized, the feature amount may be calculated using a Hessian.

[0055] In addition to the above example, the feature acquisition unit 130 may acquire feature amounts related to the shape of the driving line (for example, the curvature and driving direction of the driving line) in order to capture changes in the shape of the pancreas. The calculation of feature amounts using the brightness gradient and local structure described above may be calculated based on the traveling direction of the driving line. For example, since it is expected that the texture of the pancreas will change significantly at the point where the driving line intersects with the image edge, the relationship between the traveling direction of the driving line and the direction of the image edge may be used as the feature amount. In this case, the value of the feature amount is increased when there is an image edge close to the orthogonal direction to the traveling direction of the driving line, and the value of the feature amount is decreased when there is an image edge close to the traveling direction of the driving line. In this way, it becomes possible to identify a place on the driving line where there is a large texture change (anomaly candidate area) in steps S540 to S550.

[0056] In addition, the feature acquisition unit 130 may calculate features for excluding typical false positive patterns, in addition to features for identifying (detecting) abnormal candidate regions, and perform a process of excluding false positive patterns based on the features.

[0057] In the above description, the feature amount change acquisition unit 140 acquires the change amount of the feature amount from the pixel of interest on the driving line and the pixels adjacent thereto, but this is not limited thereto. For example, the change amount of the feature amount may be acquired from the pixel of interest and a nearby pixel that is not adjacent to the pixel of interest, or the change amount of the feature amount may be acquired using the feature amount of each of a plurality of nearby pixels centered on the pixel of interest on the driving line, in addition to the pixel of interest and the pixels adjacent thereto. For the latter, for example, a method is considered in which the weighted average value of the feature amount of each of the neighboring pixels and the difference between the feature amount of the pixel of interest are used as the change amount. In this way, the change amount of the feature amount becomes synonymous with the shift amount of the average value of the feature amount, and it becomes possible to identify an abnormal candidate region by the average value shift. In addition, any method may be used as long as it is a method for calculating the local change amount between the feature amounts of pixels on the driving line.

[0058] In the above description, the abnormal candidate region identifying unit 150 identifies an ROI centered on a pixel that is a predetermined distance away from a pixel corresponding to a change point as an abnormal candidate region, but for example, an ROI centered on a pixel corresponding to a change point may be identified as an abnormal candidate region. The abnormal candidate region identifying unit 150 identifies, as an abnormal candidate region, a region that includes at least one of a pixel corresponding to a change point whose amount of change satisfies a predetermined condition and a pixel corresponding to a position that is a predetermined distance away from the change point.

[0059] In the above description, the abnormal candidate region identifying unit 150 identifies the abnormal candidate region as a three-dimensional rectangular region, but the ROI may have any shape. For example, the ROI may be an ellipsoid or an arbitrary polygon. The ROI may also be a two-dimensional ROI. Alternatively, the abnormal candidate region may be identified by a likelihood map that indicates the likelihood of the abnormal candidate region spreading concentrically around the reference pixel described in step S550. In this case, the likelihood map is set so that the likelihood is high near the reference pixel and decreases as the region moves away from the reference pixel.

[0060] In the above description, the abnormal candidate area identifying unit 150 identifies the change point by performing a threshold judgment on the amount of change in the feature amount in step S550. However, the method of identifying the change point is not limited to this. For example, the change point may be identified by a recognizer configured to identify the change point by using the amount of change in the feature amount along the driving line as an input. The recognizer may be anything based on machine learning, and an example of the recognizer is a neural network trained using teacher data consisting of the amount of change in the feature amount along the driving line (input data) and the change point (correct answer data). In addition, the abnormal candidate area may be identified based on the result of a recognizer configured to identify the abnormal candidate area (or change point) by using the amount of change in the feature amount for the pixel on the driving line and partial image data including the pixel as an input. An example of the recognizer is a CNN trained using teacher data consisting of partial image data (input data) including the pixel on the driving line, the amount of change (input data), and the abnormal candidate area (correct answer data).

[0061] In the above description, the image recognition unit 160 uses CNN as a recognizer configured to distinguish an abnormal region, but any recognizer that performs image recognition may be used, such as a recognizer based on a deep learning method other than CNN (e.g., a vision transformer, etc.), a recognizer based on machine learning other than deep learning (e.g., SVM (Support Vector Machine) or Random Forest, etc.), or a recognizer based on a pattern matching process not based on learning. The recognizer may be a self-learning model that further updates the internal algorithm by a user giving feedback on the output product. Instead of a recognizer, a method such as a mathematical model may be applied.

[0062] In the above description, the image recognition unit 160 uses a recognizer configured to determine whether an abnormal candidate region is an abnormal region or not, but may be a recognizer configured to extract an abnormal region from partial image data including an abnormal candidate region. For example, the image recognition unit 160 identifies an abnormal region from partial image data including an abnormal candidate region by a recognizer trained using teacher data consisting of partial image data (input data) of CT image data including an abnormal candidate region and image data (correct answer data) representing the abnormal region. The recognizer is not limited to a configuration that uses the partial image data described above as input data, and may be any configuration that can recognize an abnormal region based on an abnormal candidate region. For example, the recognizer may be a configuration that uses CT image data and abnormal candidate region image data as input, or a configuration that uses image data obtained by masking the CT image data with the abnormal candidate region as input.

[0063] In the above description, the display control unit 170 controls the display unit 80 to display the abnormal region with a rectangular frame line, but the display method of the abnormal region may be any display method as long as the abnormal region is identifiable on the display unit 80. For example, the abnormal region may be displayed on the display unit 80 by filling it with a predetermined color or using an arrow, or partial image data including the abnormal region in the CT image data may be displayed on the display unit 80. In addition, the display color representing the abnormal region may be changed according to the magnitude of the change in the feature amount. Furthermore, as shown in FIG. 4, the driving line may be superimposed on the CT image data. Alternatively, the region that is not identified as an abnormal region by the image recognition unit 160 among the abnormal candidate regions identified by the abnormal candidate region identification unit 150 may be superimposed on the CT image data. In this case, for example, the region identified as an abnormal region and the region not identified as an abnormal region may be displayed in different colors or in different display forms.

[0064] In the above description, the information processing device 100 calculates a local feature (average brightness value of pixels in a window) in step S530 and obtains a change (differential value) in the feature in step S540, but the effects of the present invention can be obtained even if the processing procedure is reversed. That is, the processing procedure may be such that a differential value of pixel values ​​in the driving line direction is calculated and an average value of the differential values ​​for pixels in the window is obtained. In addition, it is not necessary to calculate the feature from image data and the change therein separately, and the feature change acquisition unit 140 may be configured to directly calculate the change in the feature from image data.

[0065] In this embodiment, the information processing device 100 displays the CT image data and the abnormal region image data on the display unit 80 via the display control unit 170. However, such display control is not essential, and for example, the abnormal candidate region may be stored in the storage device 70 or transmitted to another information processing device.

[0066] In this embodiment, the information processing device 100 determines whether or not an abnormal candidate region is an abnormal region via the image recognition unit 160, but the image recognition unit 160 is not essential. As a configuration without the image recognition unit 160, for example, the information processing device 100 controls the display control unit 170 to superimpose and display the abnormal candidate region identified by the abnormal candidate region identifying unit 150 on the CT image data. Note that the abnormal candidate region may be displayed in the same manner as the above-mentioned method for displaying the abnormal region.

[0067] <Second embodiment> (overview) In this embodiment, a method is described in which feature amounts are calculated for cross sections that intersect with a line running through the pancreas, feature amounts are calculated along the line, and a candidate abnormal region is detected using the change in feature amount.

[0068] (Functional configuration) Hereinafter, the functional configuration of the information processing device 1000 according to this embodiment will be described with reference to Fig. 10. As shown in the figure, the information processing device 1000 is configured by an image data acquisition unit 1010, a driving line acquisition unit 1020, an area acquisition unit 1030, a feature amount acquisition unit 1040, a feature amount change acquisition unit 1050, and an abnormality candidate area identification unit 1060. In addition, the information processing device 1000 according to this embodiment is equipped with an external storage device 70.

[0069] The storage device 70 and the display unit 80 are similar to those in the first embodiment, and therefore the description thereof will be omitted. The storage device 70 and the display unit 80 may be included in the configuration of the information processing device 1000. The storage device 70 may be provided within the information processing device 1000.

[0070] The image data acquisition unit 1010 acquires CT image data from the storage device 70 , and transmits the data to the region acquisition unit 1030 , the feature amount acquisition unit 1040 , and the display control unit 1070 .

[0071] The driving line acquisition unit 1020 acquires driving line data of the pancreas corresponding to the CT image data from the storage device 70 and transmits it to the feature amount acquisition unit 1040.

[0072] The region acquiring unit 1030 first receives CT image data from the image data acquiring unit 1010. Next, the region acquiring unit 1030 extracts a region of the pancreas on the CT image data to generate data representing the region of the pancreas (hereinafter, pancreas region data). Then, the region acquiring unit 1030 transmits the pancreas region data to the feature acquiring unit 1040. The region of the pancreas on the CT image data is an example of a target region from which features are acquired in this embodiment.

[0073] Here, the pancreas region data will be described. The pancreas region data is image data indicating whether each pixel is included in the pancreas or not, and is, for example, a binary image of the same image size as the CT image data, in which the pixel value of a pixel included in the pancreas is expressed as 1 and the pixel value of a pixel not included in the pancreas is expressed as 0. The pancreas region data may be in any format as long as the region of the pancreas depicted on the CT image data can be identified, and may be likelihood image data in which the pixel value of each pixel indicates the likelihood of the region of the pancreas. In addition, the pancreas region data may be multi-value image data in which the pixel value of each pixel indicates the region of various tissues including the pancreas in multi-values. In addition, the pancreas region data may be text data that approximates the shape of the pancreas to a polygon and holds the coordinate values ​​of the vertices. In other words, the pancreas region data is data that can specify a target region.

[0074] The feature amount acquiring unit 1040 acquires the feature amount based on the target area in the cross-sectional image data intersecting the driving line. More specifically, the feature amount acquiring unit 1040 first receives CT image data from the image data acquiring unit 1010, receives pancreas driving line data corresponding to the CT image data from the driving line acquiring unit 1020, and receives pancreas area data corresponding to the CT image data from the area acquiring unit 1030. Next, the feature amount acquiring unit 1040 calculates the feature amount for each cross section intersecting the driving line in the CT image data, and acquires the feature amount along the driving line. For example, the feature amount acquiring unit 1040 of this embodiment acquires the feature amount for at least one cross-sectional image data intersecting the driving line among the CT image data that is the three-dimensional tomographic image data. The CT image data includes a plurality of cross-sectional image data. Then, the feature amount acquiring unit 1040 transmits the feature amount along the driving line to the feature amount change acquiring unit 1050.

[0075] The feature amount change acquisition unit 1050 first receives feature amounts along the driving line from the feature amount acquisition unit 1040. Next, similar to the first embodiment, the feature amount change acquisition unit 1050 calculates the amount of change between local feature amounts for the feature amounts along the driving line to acquire the amount of change in the feature amounts. Then, the feature amount change acquisition unit 1050 transmits the amount of change in the feature amounts to the abnormality candidate area identification unit 1060.

[0076] The abnormal candidate area identifying unit 1060 first receives the amount of change in the feature amount from the feature amount change acquisition unit 1050. Next, similar to the first embodiment, the abnormal candidate area identifying unit 1060 identifies a change point where the amount of change in the feature amount satisfies a predetermined condition, and identifies (detects) an area including the change point as an abnormal candidate area. Then, the abnormal candidate area identifying unit 1060 generates abnormal candidate area image data, and transmits the abnormal candidate area image data to the display control unit 1070.

[0077] The display control unit 1070 first receives the CT image data from the image data acquisition unit 1010, and receives the abnormal candidate region image data from the abnormal candidate region identification unit 1060. Then, the display control unit 1070 causes the display unit 80 to display the CT image data and the abnormal candidate region image data.

[0078] (Hardware configuration) The hardware configuration of the information processing device 1000 according to this embodiment is the same as that of the first embodiment, and therefore a description thereof will be omitted.

[0079] (Processing Procedure) Next, a processing procedure of the information processing device 1000 in this embodiment will be described with reference to Fig. 11 and Fig. 12. Fig. 11 is a flowchart showing an example of a processing procedure of the information processing device 1000 according to the second embodiment. Fig. 12 is a flowchart showing an example of a procedure of a feature amount calculation process according to the second embodiment. In the following, a description will be given by omitting parts that overlap with the description of the information processing device 100 according to the first embodiment.

[0080] (Step S1110) Step S1110 is the same as step S510 in the first embodiment, and therefore a description thereof will be omitted.

[0081] (Step S1120) Step S1120 is the same as step S520 in the first embodiment, and therefore a description thereof will be omitted.

[0082] (Step S1130) In step S1130, the region acquiring unit 1030 first receives CT image data from the image data acquiring unit 1010. Next, the region acquiring unit 1030 extracts a region of the pancreas on the CT image data to generate pancreas region data. Then, the region acquiring unit 1030 transmits the pancreas region data to the feature amount acquiring unit 1040.

[0083] In this embodiment, the region acquisition unit 1030 extracts the region of the pancreas on the CT image data using a recognizer based on machine learning configured to extract the region of the pancreas using CT image data as input. The above-mentioned recognizer is, for example, a trained CNN, which is trained by a known method such as an error backpropagation method using teacher data composed of CT image data and correct pancreas region data corresponding to the CT image data. The recognizer may be any recognizer that extracts the region of the pancreas on the CT image data. For example, the recognizer may be a recognizer based on machine learning such as SVM or Random Forest, or a recognizer based on image processing such as a region growing method, graph cut segmentation, or threshold processing. The recognizer may be a self-learning model that further updates the internal algorithm by a user giving feedback to the output product. The recognizer is an example of an inference model based on machine learning in this embodiment. In place of the recognizer, a method such as a mathematical expression model may be applied.

[0084] (Step S1140) In step S1140, the feature amount acquisition unit 1040 first receives CT image data from the image data acquisition unit 1010, and receives pancreas driving line data and pancreas region data corresponding to the CT image data from the driving line acquisition unit 1020 and the region acquisition unit 1030. Next, the feature amount acquisition unit 1040 calculates feature amounts for each cross section intersecting with the driving line in the CT image data, and acquires feature amounts along the driving line. Then, the feature amount acquisition unit 1040 transmits the feature amounts along the driving line to the feature amount change acquisition unit 1050.

[0085] Here, the feature amount calculation process in step S1140 in this embodiment will be described in detail with reference to the flowchart in FIG.

[0086] (Step S1141) In step S1141, the feature acquisition unit 1040 sets an arbitrary pixel on the running line as a pixel of interest. In this embodiment, a case where feature amounts for pixels on the running line are calculated in order from the head of the pancreas to the tail of the pancreas will be described. The feature acquisition unit 1040 first sets a pixel on the running line at the end on the head side of the pancreas as a pixel of interest.

[0087] (Step S1142) In step S1142, the feature amount acquiring unit 1040 generates cross section image data that intersects with the driving line at the pixel of interest (hereinafter, cross section image data). In this embodiment, the tangent vector of the driving line at the pixel of interest is used as the normal vector, and cross section image data is generated from each of the CT image data and the pancreas region data. Hereinafter, the cross section image data generated from the CT image data is referred to as CT cross section image data, and the cross section image data generated from the pancreas region data is referred to as pancreas cross section image data.

[0088] (Step S1143) In step S1143, the feature amount acquiring unit 1040 calculates a feature amount for the pixel of interest based on the CT cross section image data and the pancreas cross section image data. More specifically, the average value of the luminance of the pancreas region in the CT cross section image data is calculated, and this is set as the feature amount for the pixel of interest. The pancreas region in the CT cross section image data can be specified by referring to the pancreas cross section image data. The average value of the luminance of the pancreas region is an example of a statistic (statistical value) of the luminance of the pancreas region, and the feature amount acquired by the feature amount acquiring unit 1040 may be a value based on the statistic (statistical value) of the luminance of the pancreas region other than the average value (for example, at least one of the minimum value, maximum value, median value, variance value, and weighted average value). The statistic of the luminance of the pancreas region on each cross section in the CT cross section image data can be rephrased as the statistic of the luminance in the vicinity of the driving line.

[0089] (Step S1144) In step S1144, the feature amount acquisition unit 1040 determines whether or not the next pixel on the driving line exists, and if the next pixel does not exist, the step S1140 is terminated, and if not, the process returns to step S1141 and the next pixel is set as the pixel of interest. In other words, when the feature amount is calculated for all pixels on the driving line based on the cross section intersecting the driving line, the step S1140 is terminated.

[0090] In this embodiment, the feature values ​​for the pixels on the running line are calculated in order from the head to the tail of the pancreas, so the next pixel is a pixel that is further in the direction toward the tail of the pancreas than the current pixel of interest. Also, the pixel of interest at the end of step S1140 is the pixel on the running line that is at the end of the tail of the pancreas.

[0091] In step S1140 described above, the feature amount acquiring unit 1040 acquires feature amounts along the driving line as shown in Fig. 7. In this embodiment, the value of the feature amount in Fig. 7 corresponds to the average brightness value of the pancreas region in the cross section intersecting the driving line.

[0092] (Step S1150) Returning to FIG. 11, step S1150 is the same as step S540 in the first embodiment, and therefore a description thereof will be omitted.

[0093] (Step S1160) In step S1160, the abnormal candidate area identifying unit 1060 first receives the amount of change in the feature amount from the feature amount change acquisition unit 1050. Next, similar to the first embodiment, the abnormal candidate area identifying unit 1060 identifies a change point where the amount of change in the feature amount satisfies a predetermined condition, and identifies (detects) an area including the change point as an abnormal candidate area. Then, the abnormal candidate area identifying unit 1060 generates abnormal candidate area image data, and transmits the abnormal candidate area image data to the display control unit 1070.

[0094] (Step S1170) In step S1170, the display control unit 1070 first receives the CT image data from the image data acquisition unit 1010, and receives the abnormal candidate region image data from the abnormal candidate region identification unit 1060. Then, the display control unit 1070 causes the display unit 80 to display the CT image data and the abnormal candidate region image data.

[0095] In this embodiment, the display control unit 1070 causes the display unit 80 to display a rectangular frame surrounding the abnormal candidate region (ROI) held in the abnormal candidate region image data so as to be superimposed on the CT image data 300. Other display methods for the abnormal candidate region are similar to the display method for the abnormal region in the first embodiment.

[0096] Through the above process, the information processing device 1000 calculates the amount of change in the brightness value of the pancreas region in the cross section intersecting the driving line based on the CT image data and the driving line data of the pancreas, and identifies an abnormal candidate region in the pancreas based on the amount of change. Then, it is determined whether the identified abnormal candidate region is abnormal or not. In this way, it is possible to capture changes in the pancreas, such as indirect findings of pancreatic cancer, and it is expected to improve the detection performance of unclear pancreatic cancer.

[0097] In the above description, the region acquisition unit 1030 acquires pancreas region data representing the region of the pancreas on the CT image data by using a recognizer, but is not limited thereto. For example, pancreas region data corresponding to the CT image data generated in advance may be acquired from the storage device 70, or pancreas region data corresponding to the CT image data may be acquired from another information processing device. The region acquisition unit 1030 may acquire not only pancreas region data but also pancreatic duct region data representing the region of the pancreatic duct. That is, the region of the pancreatic duct may be an example of a target region related to the pancreas. In this case, the feature amount acquisition unit 1040 can calculate the feature amount only from the region of the pancreas (region of the pancreatic parenchyma) excluding the region of the pancreatic duct, for example, when calculating the feature amount from a cross section intersecting with the running line. In this way, since minute changes in the pancreatic parenchyma can be captured, it is expected that the detection accuracy of pancreatic cancer can be improved. The region acquisition unit 1030 acquires, for example, at least one of the region of the pancreas and the region of the pancreatic duct as the target region related to the pancreas.

[0098] In the above description, the feature amount acquiring unit 1040 calculates the feature amount related to the brightness of the pancreas, but the feature amount related to the shape of the pancreas may be acquired. For example, the feature amount acquiring unit 1040 may acquire the area, circumference, major axis, minor axis, average diameter, etc. of the region of the pancreas on the cross section image data as the feature amount. The area and circumference of the region of the pancreas on the cross section image data are examples of the area and circumference of the target region in this embodiment, respectively. The major axis, minor axis, and average diameter of the region of the pancreas on the cross section image data are examples of the diameter of the target region in this embodiment. By using such feature amount, the feature amount acquiring unit 1040 can capture localized stenosis of the pancreas, which is one of the indirect findings of pancreatic cancer. In addition, the feature amount related to the shape of the pancreas may be a feature amount representing the characteristics of the shape of the cross section, such as flattening ratio or circularity. In addition, the feature amount acquiring unit 1040 may acquire the feature amount by using any filter (such as a Sobel filter or a Gabor filter) or Hessian as described in the first embodiment for the cross section image data. Furthermore, the feature amount acquiring unit 1040 may use a combination of multiple feature amounts obtained from the above-mentioned brightness and shape of the pancreas, any filter, etc. In this case, for example, the information processing device 1000 calculates the amount of change for each of the feature amounts in step S1150, and identifies the abnormal candidate region by using a corresponding threshold value for each amount of change in each feature amount in step S1160. Alternatively, the feature amount acquiring unit 1040 may use the result of integrating multiple feature amounts as the feature amount for the pixel on the driving line.

[0099] In the above description, the feature amount acquiring unit 1040 calculates the feature amount for each of the cross section image data for the travel line, but the feature amount may be calculated for a plurality of cross section image data for the travel line. In this case, the plurality of cross section image data may be continuous (adjacent) cross section image data, or may be discontinuous (not adjacent) cross section image data. In addition, when calculating the feature amount from the plurality of cross section image data, the step width for the travel line when calculating the feature amount along the travel line may be smaller or larger than the number of the plurality of cross section image data. In other words, the plurality of cross section image data when calculating the feature amount may or may not overlap with the plurality of cross section image data when calculating the adjacent feature amount. In addition, the feature amount may be calculated for a slab image having a certain thickness in the cross section for the travel line.

[0100] In the above description, the feature acquisition unit 1040 generates cross section image data for the driving line when calculating the feature, but the feature may be calculated by selectively using pixels in the target area for calculating the feature without explicitly generating the cross section image data.

[0101] In the above description, the display control unit 1070 displays the abnormal candidate region on the CT image data on the display unit 80 in the same manner as in the first embodiment, but partial image data including CT cross section image data corresponding to the abnormal candidate region may be displayed. At this time, pancreatic cross section image data may be superimposed on the partial image data including CT cross section image data corresponding to the abnormal candidate region. Alternatively, a CPR (Curved Planar Reconstruction) image in which the running line of the pancreas is depicted in one cross section may be generated, and the abnormal candidate region may be superimposed on the CPR image.

[0102] In the above description, the information processing device 1000 extracts the pancreas region from the CT image data by the region acquiring unit 1030, but may be configured to extract the pancreas region from the CT cross sectional image data generated in step S1142. In this case, for example, the feature acquiring unit 1040 extracts the pancreas region from the CT cross sectional image data by processing such as the region acquiring unit 1030 according to the present embodiment. Alternatively, the feature acquiring unit 1040 may be configured to calculate the feature from only pixels having a luminance value that the pancreas can have, without explicitly extracting the pancreas region.

[0103] In the above description, the information processing device 1000 calculates the feature amount along the driving line in step S1140, and calculates the change amount of the feature amount along the driving line in step S1150, but the processing procedure may be reversed. That is, the change amount (difference, etc.) with respect to the CT cross sectional image data along the driving line may be calculated first, and then the feature amount may be calculated for the change amount. Even with such a processing procedure, the effect of the present invention can be enjoyed.

[0104] <Third embodiment> (overview) In the first embodiment, a method for detecting an abnormal candidate region by calculating feature amounts along a line running through the pancreas and using the change in the feature amount was described. In the present embodiment, a method for detecting an abnormal candidate region by using the feature amount values ​​themselves will be described.

[0105] More specifically, the information processing device of the present embodiment acquires the degree of stenosis of the pancreas around each pixel on the driving line as a feature value along the driving line in the pancreas. Then, when the feature value along the driving line is equal to or greater than a predetermined threshold, the information processing device detects the region as an abnormal candidate region.

[0106] (Functional configuration) Hereinafter, the functional configuration of the information processing device 1300 according to this embodiment will be described with reference to Fig. 13. As shown in the figure, the information processing device 1300 is configured by an image data acquisition unit 1310, a driving line acquisition unit 1320, an area acquisition unit 1325, a feature amount acquisition unit 1330, an abnormality candidate area identification unit 1340, and a display control unit 1350. In addition, the information processing device 1300 according to this embodiment is externally equipped with a storage device 70 and a display unit 80.

[0107] The storage device 70 and the display unit 80 are similar to the storage device 70 in the first embodiment, and therefore the description thereof will be omitted. The storage device 70 and the display unit 80 may be included in the configuration of the information processing device 1300. The storage device 70 may be provided within the information processing device 1300.

[0108] The image data acquisition unit 1310 acquires CT image data from the storage device 70 , and transmits the data to the feature amount acquisition unit 1330 and the display control unit 1350 .

[0109] The driving line acquisition unit 1320 acquires driving line data of the pancreas corresponding to the CT image data from the storage device 70 and transmits the data to the feature amount acquisition unit 1330 .

[0110] The region acquiring unit 1325 first receives CT image data from the image data acquiring unit 1310. Next, the region acquiring unit 1325 extracts a region of the pancreas on the CT image data to generate data representing the region of the pancreas (hereinafter, pancreas region data). Then, the region acquiring unit 1325 transmits the pancreas region data to the feature acquiring unit 1330 and the display control unit 1350.

[0111] The feature amount acquiring unit 1330 first receives CT image data from the image data acquiring unit 1310, receives pancreas driving line data corresponding to the CT image data from the driving line acquiring unit 1320, and receives pancreas region data corresponding to the CT image data from the region acquiring unit 1325. Next, the feature amount acquiring unit 1330 calculates feature amounts along the driving line from the CT image data based on the driving line data. Then, the feature amount acquiring unit 1330 transmits the feature amounts along the driving line to the abnormal candidate region identifying unit 1340.

[0112] The abnormal candidate area identifying unit 1340 first receives feature amounts along the driving line from the feature amount acquiring unit 1330. Next, the abnormal candidate area identifying unit 1340 identifies pixels on the driving line whose feature amount values ​​satisfy a predetermined condition, and identifies (detects) an area including the pixels on the driving line as an abnormal candidate area. Then, the abnormal candidate area identifying unit 1340 generates abnormal candidate area image data, and transmits the abnormal candidate area image data to the display control unit 1350.

[0113] The display control unit 1350 first receives the CT image data from the image data acquisition unit 1310, and receives the abnormal candidate region from the abnormal candidate region identification unit 1340. Then, the display control unit 1350 causes the display unit 80 to display the CT image data and the abnormal candidate region.

[0114] (Hardware configuration) The hardware configuration of the information processing device 1300 according to this embodiment is the same as that of the first embodiment, and therefore a description thereof will be omitted.

[0115] (Processing Procedure) Next, a processing procedure of the information processing device 1300 in this embodiment will be described with reference to Fig. 14. Fig. 14 is a flowchart showing an example of a processing procedure of the information processing device 1300 according to the third embodiment. In the following, the description will be made by omitting parts that overlap with the description of the information processing device 100 according to the first embodiment.

[0116] (Step S1410) In step S1410, the image data acquisition unit 1310 acquires CT image data from the storage device 70, and transmits the data to the feature amount acquisition unit 1330 and the display control unit 1350.

[0117] (Step S1420) In step S1420, the driving line acquisition unit 1320 acquires driving line data of the pancreas corresponding to the CT image data from the storage device 70, and transmits the data to the feature amount acquisition unit 1330.

[0118] (Step S1425) In step S1425, the region acquiring unit 1325 first receives CT image data from the image data acquiring unit 1310. Next, the region acquiring unit 1325 extracts a region of the pancreas on the CT image data to generate pancreas region data. Then, the region acquiring unit 1325 transmits the pancreas region data to the feature acquiring unit 1330 and the display control unit 1350. In this processing step, the processing executed by the area obtaining unit 1325 is similar to that in step S1130 executed by the area obtaining unit 1030 in the second embodiment, and a detailed description thereof will be omitted here.

[0119] (Step S1430) In step S1430, the feature amount acquiring unit 1330 first receives CT image data from the image data acquiring unit 1310, and receives pancreas driving line data and pancreas region data corresponding to the CT image data from the driving line acquiring unit 1320 and the region acquiring unit 1325, respectively. Next, the feature amount acquiring unit 1330 calculates a feature amount along the driving line from the CT image data based on the driving line data. In this embodiment, a feature amount related to the shape of the pancreas is calculated as the feature amount along the driving line. Specifically, the degree to which the shape of the pancreas on the cross section image data is narrowed in the driving direction of the driving line is calculated as the feature amount. A more specific processing procedure will be described below.

[0120] First, the feature acquisition unit 1330 sets an arbitrary pixel on the driving line as a pixel of interest, and generates cross-sectional image data intersecting the driving line at the pixel. Here, the cross-sectional image data generated by the feature acquisition unit 1330 is a plurality of pieces of cross-sectional image data along the direction of the tangent vector of the driving line. As an example, the feature acquisition unit 1330 acquires a total of three pieces of tomographic image data, including a tomographic image data centered on the pixel position (center tomographic image) and two front and rear tomographic image data (front and rear tomographic images) spaced a predetermined distance apart in the tangent direction of the driving line.

[0121] Next, for each of the three generated tomographic image data, the average diameter of the pancreas region on the tomographic image data is calculated. If the average diameter in the central tomographic image data is smaller than the average diameter in the anterior-posterior tomographic image data, the degree of stenosis is deemed to be large and a large feature value is calculated. Conversely, if the average diameter in the central tomographic image is equal to or larger than the average diameter in the anterior-posterior tomographic image, the degree of stenosis is deemed to be small and a small feature value is calculated.

[0122] (Step S1440) In step S1440, the abnormal candidate area identifying unit 1340 first receives feature amounts along the driving line from the feature amount acquiring unit 1330. Next, the abnormal candidate area identifying unit 1340 identifies pixels on the driving line whose feature amount values ​​satisfy a predetermined condition, and identifies (detects) an area including the pixels on the driving line as an abnormal candidate area. Then, the abnormal candidate area identifying unit 1340 generates abnormal candidate area image data, and transmits the abnormal candidate area image data to the display control unit 1350.

[0123] As shown in FIG. 3, the shape of the pancreas around the area where pancreatic cancer exists may have a stenosis. Therefore, the abnormal candidate region identifying unit 1340 in this embodiment identifies an abnormal candidate region using a condition that the feature amount (degree of stenosis) calculated in step S1430 is equal to or greater than a predetermined value as a predetermined condition. More specifically, the abnormal candidate region identifying unit 1340 in this embodiment first identifies pixels corresponding to section A as pixels on a driving line that satisfy the above-mentioned condition. That is, the abnormal candidate region identifying unit 1340 identifies pixels whose feature amount is equal to or greater than a predetermined value. Then, the abnormal candidate region identifying unit 1340 identifies an abnormal candidate region by using an ROI of any size that includes pixels on the driving line. In this embodiment, a three-dimensional rectangular ROI with the length of one side being the length of consecutive pixels (section A) on the driving line that satisfy the predetermined condition is used as the ROI of any size. In addition, since the feature value in this embodiment is the degree of pancreatic stenosis, the abnormal candidate region identifying unit 1340 may identify, as an abnormal candidate region, a localized stenosis of the pancreas, which is an indirect finding of pancreatic cancer, or may identify a region suspected of the presence of pancreatic cancer, as in the first embodiment.

[0124] (Step S1450) In step S1450, the display control unit 1350 first receives CT image data from the image data acquisition unit 1310, and receives abnormal candidate region image data from the abnormal candidate region identification unit 1340. Then, the display control unit 1350 causes the CT image data and the abnormal candidate region image data to be displayed on the display unit 80. Details of the process are omitted since they are similar to those of step S1170 in the second embodiment.

[0125] Through the above processing procedure, the information processing device 1300 calculates the feature amount along the driving line based on the CT image data and the driving line data of the pancreas, and identifies the abnormal candidate region in the pancreas based on the feature amount along the driving line. In this way, it is possible to capture the change in the pancreas, and therefore it is possible to identify (detect) even an unclear pancreatic cancer.

[0126] (Variation of the third embodiment) The feature acquisition unit 1330 may obtain the feature representing the degree of stenosis of the pancreas by a method other than that described above. For example, the diameter or area of ​​the pancreas in each cross section image intersecting with the driving line may be used as the degree of stenosis. Also, each cross section image intersecting with the driving line may be input, and a classifier that infers the degree of stenosis may be constructed by deep learning, and the degree of stenosis may be estimated based on the output. Also, a value indicating an abnormality in the shape of the pancreas other than the degree of stenosis (for example, flattening ratio or deviation from a circular shape) may be used as the feature.

[0127] In addition, a feature based on the shape of a target region related to the pancreas other than the pancreatic region may be used. For example, the pancreatic duct may be used as a target region and a feature based on the shape of the pancreatic duct may be used. For example, a configuration may be used in which the diameter or area of ​​the pancreatic duct in each cross-sectional image that crosses the running line is used as a feature to identify the vicinity of the position where the pancreatic duct is interrupted as an abnormal candidate region.

[0128] In the above description, the feature acquisition unit 1330 calculates the feature based on the shape of the pancreas, but this is not limited to the above. For example, the feature may be calculated based on the brightness value of a pixel on the driving line or the brightness statistics of pixels in the vicinity of the pixel on the driving line. In this case, the abnormal candidate region identification unit 1340 may identify a pixel having a feature outside a predetermined range. In addition to this, for example, a difference value between the average value of the feature in the CT image data and the value of the feature for a pixel on the driving line may be calculated, and the difference value may be determined by a threshold value. Any other method may be used as long as it is a method for identifying a pixel having a feature that is an outlier.

[0129] In each of the above embodiments, the pancreas has been described as an example, but the method realized by each functional unit of the information processing device 100, 1000 of the above embodiments may be applied to anatomical tissue other than the pancreas, such as other solid organs, to detect abnormal candidate regions.

[0130] The various data handled in this specification are typically digital data.

[0131] According to at least one of the embodiments described above, it is possible to improve the accuracy of detecting an abnormality candidate region such as pancreatic cancer.

[0132] Although some embodiments have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These embodiments can be implemented in various other forms, and various omissions, substitutions, modifications, and combinations of embodiments can be made without departing from the spirit of the invention. These embodiments and their modifications are included in the scope of the invention and its equivalents as described in the claims, as well as in the scope and spirit of the invention. [Explanation of symbols]

[0133] 70 Storage device 80 Display section 100, 1000, 1300 Information processing equipment 110,1010,1310 Image data acquisition section 120, 1020, 1320 Track acquisition section 130,1040,1330 Feature acquisition section 140,1050 Feature change acquisition section 150,1060,1340 Abnormal candidate area identification part 160 Image Recognition Unit 170,1070,1350 Display control unit 202 Main Memory 203 Magnetic Disk 204 Display Memory 205 Monitor 206 Mouse 207 Keyboard 208 Common Bus 300 CT image data 301 Pancreas 302 Pancreatic cancer 303 Main pancreatic duct 401 Running Line 901 Abnormal area 1030,1325 Area acquisition part

Claims

1. an image data acquisition unit that acquires three-dimensional tomographic image data of the pancreas; a travel line acquisition unit that acquires travel line data capable of identifying a travel line that runs through the pancreas corresponding to the three-dimensional tomographic image data; a feature amount acquiring unit for acquiring feature amounts along the driving line corresponding to the three-dimensional tomographic image data; a feature amount change acquisition unit that acquires a change amount between the plurality of feature amounts along the traveling line; an abnormal candidate region identifying unit that identifies an abnormal candidate region based on the amount of change; An information processing device comprising:

2. The three-dimensional tomographic image data includes a plurality of cross-sectional image data; the feature amount acquiring unit acquires the feature amount for at least one cross-sectional image data intersecting the driving line among the three-dimensional tomographic image data; The information processing device according to claim 1 .

3. the feature amount acquiring unit acquires the feature amounts for a plurality of cross-sectional image data intersecting the driving line; The information processing device according to claim 2 .

4. The feature amount acquisition unit acquires a value based on a pixel value on the driving line as the feature amount. The information processing device according to claim 1 .

5. the feature amount change acquisition unit acquires the amount of change based on a value obtained by first-order differentiation along the driving line for the plurality of feature amounts; The information processing device according to claim 1 .

6. the abnormal candidate region identifying unit identifies, as the abnormal candidate region, a region including at least one of a pixel corresponding to a change point where the amount of change satisfies a predetermined condition and a pixel corresponding to a position away from the change point by a predetermined distance; The information processing device according to claim 1 .

7. a region acquiring unit that acquires at least one of a region of the pancreas and a region of the pancreatic duct as a target region for acquiring feature amounts in the three-dimensional tomographic image data; The feature amount acquiring unit acquires the feature amount based on the target area in the cross-sectional image data that intersects with the driving line. The information processing device according to claim 1 .

8. the feature amount acquiring unit acquires, as the feature amount, a feature amount based on a statistical value of brightness of the target region in the cross-sectional image data; The information processing device according to claim 7.

9. the feature amount acquiring unit acquires a feature amount based on at least one of an average value, a maximum value, a minimum value, a median value, and a variance value as a statistical value of the brightness of the target region in the cross-sectional image data; The information processing device according to claim 8.

10. The feature amount acquiring unit acquires, as the feature amount, a feature amount related to a shape of the target region in the cross-sectional image data. The information processing device according to claim 7.

11. the feature amount acquiring unit acquires at least one of an area, a perimeter, and a diameter of the target region in the cross-sectional image data as the feature amount; The information processing device according to claim 10.

12. Further comprising an image recognition unit that performs image recognition on the anomaly candidate region using an inference model based on machine learning. The information processing device according to claim 1 .

13. a display control unit that causes at least one of the abnormal candidate region and an image recognition result that is a result of image recognition on the abnormal candidate region to be identifiable on a display unit; The information processing device according to claim 1 .

14. the display control unit causes the abnormality candidate region for the three-dimensional tomographic image data to be displayed on the display unit. The information processing device according to claim 13.

15. The display control unit causes the display unit to display the cross-sectional image data including the abnormality candidate region. The information processing device according to claim 13.

16. an image data acquisition unit that acquires three-dimensional tomographic image data of the pancreas; a travel line acquisition unit that acquires travel line data capable of identifying a travel line that runs through the pancreas corresponding to the three-dimensional tomographic image data; a feature amount acquiring unit that acquires, as a feature amount, a statistical value of brightness in the vicinity of the driving line corresponding to the three-dimensional tomographic image data; an abnormality candidate region identifying unit that identifies an abnormality candidate region based on the feature amount; An information processing device comprising:

17. An image data acquisition step of acquiring three-dimensional tomographic image data of the pancreas; a travel line acquisition step of acquiring travel line data capable of identifying a travel line running through the pancreas corresponding to the three-dimensional tomographic image data; a feature amount acquiring step of acquiring feature amounts along the driving line corresponding to the three-dimensional tomographic image data; a feature amount change acquisition step of acquiring a change amount between the plurality of feature amounts along the driving line; an abnormality candidate region specifying step of specifying an abnormality candidate region based on the amount of change; An information processing method comprising:

18. An image data acquisition step of acquiring three-dimensional tomographic image data of the pancreas; a travel line acquisition step of acquiring travel line data capable of identifying a travel line running through the pancreas corresponding to the three-dimensional tomographic image data; a feature amount acquiring step of acquiring, as a feature amount, a statistical value of brightness in the vicinity of the driving line corresponding to the three-dimensional tomographic image data; an abnormality candidate region specifying step of specifying an abnormality candidate region based on the feature amount; An information processing method comprising:

19. An image data acquisition step of acquiring three-dimensional tomographic image data of the pancreas; a travel line acquisition step of acquiring travel line data capable of identifying a travel line running through the pancreas corresponding to the three-dimensional tomographic image data; a feature amount acquiring step of acquiring feature amounts along the driving line corresponding to the three-dimensional tomographic image data; a feature amount change acquisition step of acquiring a change amount between the plurality of feature amounts along the driving line; an abnormal candidate region specifying step of specifying an abnormal candidate region in the pancreas near the driving line based on the amount of change; A program for causing a computer to execute the following.

20. An image data acquisition step of acquiring three-dimensional tomographic image data of the pancreas; a travel line acquisition step of acquiring travel line data capable of identifying a travel line running through the pancreas corresponding to the three-dimensional tomographic image data; a feature amount acquiring step of acquiring, as a feature amount, a statistical value of brightness in the vicinity of the driving line corresponding to the three-dimensional tomographic image data; an abnormality candidate region specifying step of specifying an abnormality candidate region based on the feature amount; A program for causing a computer to execute the following.

Images