Image processing device, image processing method, and program

Abstract

To extract (segment) a target area captured in medical image data accurately at high speed.SOLUTION: An image processing apparatus according to an embodiment extracts a target area including a specified point on medical image data, and comprises a first extraction unit and a second extraction unit. The first extraction unit extracts a first extraction area estimated to be the target area from first partial image data included in a first processing range including the specified point on the medical image data. When the first extraction area is assumed to be a part of the target area, the second extraction unit extracts a second extraction area estimated to be the target area from second partial image data that is in contact with a peripheral part of the first processing range or is included in a second processing range including at least part of the peripheral part from the medical image data.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The embodiments disclosed in this specification and the drawings relate to an image processing apparatus, an image processing method, and a program.

Background Art

[0002] There is a technique for extracting a target region having various sizes shown in medical image data. For example, the target region is a lesion region such as a tumor. The volume of the tumor may be 0.01 cm 3 or less, or may be 100 cm 3 or more, and the size of the tumor becomes various sizes that are orders of magnitude different. For example, region extraction using a Deep Neural Network (DNN) is generally highly accurate, but can only input images of a fixed size. If the fixed size is too large for the target region, the extraction accuracy of the target region that is too small for the fixed size decreases. On the other hand, if the fixed size is too small for the target region, only a part of the target region that is too large for the fixed size can be extracted.

[0003] Also, there is a technique for extracting a target region by dividing the entire image data into a plurality of partial regions of a fixed size and processing all the partial regions. In such a technique, since the number of partial regions to be processed becomes very large, the calculation time becomes very long.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Patent Document 2

Patent Document 3

Patent Document 4

Patent Document 5

Patent Document 6

[0005] One of the problems that the embodiments disclosed herein and in the drawings aim to solve is to extract (segment) a region of interest captured in medical image data with high accuracy and speed. However, the problems that the embodiments disclosed herein and in the drawings aim to solve are not limited to the above problem. Problems corresponding to the effects of each configuration shown in the embodiments described later can also be positioned as other problems. [Means for solving the problem]

[0006] The image processing apparatus according to the embodiment extracts a region of interest including a designated point on medical image data. The image processing apparatus comprises a first extraction unit and a second extraction unit. The first extraction unit extracts a first extraction region, which is presumed to be the region of interest, from a first partial image data included within a first processing range of the medical image data that includes the designated point. The second extraction unit, assuming that the first extraction region is part of the region of interest, extracts a second extraction region, which is presumed to be the region of interest, from a second partial image data included within a second processing range that is adjacent to the edge of the first processing range or includes at least a part of the edge of the first processing range, from the medical image data. [Brief explanation of the drawing]

[0007] [Figure 1] Figure 1 shows an example of the configuration of an image processing apparatus according to the first embodiment. [Figure 2A] Figure 2A is a flowchart showing an example of the flow of region extraction processing performed by the image processing device according to the first embodiment. [Figure 2B]Figure 2B is a flowchart showing an example of the flow of region extraction processing performed by the image processing apparatus according to the first embodiment. [Figure 3] Figure 3 is a diagram illustrating an example of processing performed by the image processing apparatus according to the first embodiment. [Figure 4] Figure 4 is a diagram illustrating an example of processing performed by the image processing device according to the first embodiment. [Figure 5] Figure 5 is a diagram illustrating an example of processing performed by the image processing device according to the first embodiment. [Figure 6] Figure 6 is a diagram illustrating an example of processing performed by the image processing apparatus according to the first embodiment. [Figure 7] Figure 7 is a diagram illustrating an example of processing performed by the image processing apparatus according to the first embodiment. [Figure 8] Figure 8 is a diagram illustrating an example of processing performed by the image processing apparatus according to the first embodiment. [Figure 9] Figure 9 is a diagram illustrating an example of processing performed by the image processing device according to the first embodiment. [Figure 10] Figure 10 is a diagram illustrating an example of processing performed by the image processing apparatus according to the first embodiment. [Figure 11] Figure 11 is a diagram illustrating an example of processing performed by the image processing apparatus according to the first embodiment. [Figure 12] Figure 12 is a diagram illustrating an example of processing performed by the image processing apparatus according to the first embodiment. [Figure 13] Figure 13 is a diagram illustrating an example of processing performed by the image processing apparatus according to the first embodiment. [Figure 14] Figure 14 is a diagram illustrating an example of processing performed by an image processing device according to a first modification of the first embodiment. [Figure 15] Figure 15 shows an example of a fixed size 1 used in the first embodiment. [Figure 16]FIG. 16 is a diagram showing an example of the fixed size 2 used in the second embodiment. [Figure 17A] FIG. 17A is a flowchart showing an example of the flow of the region extraction process executed by the image processing apparatus according to the second embodiment. [Figure 17B] FIG. 17B is a flowchart showing an example of the flow of the region extraction process executed by the image processing apparatus according to the second embodiment. [Figure 18] FIG. 18 is a diagram for explaining an example of the process executed by the image processing apparatus according to the second embodiment. [Figure 19A] FIG. 19A is a flowchart showing an example of the flow of the region extraction process executed by the image processing apparatus according to the first modification of the second embodiment. [Figure 19B] FIG. 19B is a flowchart showing an example of the flow of the region extraction process executed by the image processing apparatus according to the first modification of the second embodiment.

Embodiments for Carrying Out the Invention

[0008] Hereinafter, each embodiment and each modification of the image processing apparatus, the image processing method, and the program will be described in detail with reference to the drawings. Note that the embodiments can be combined with the prior art, other embodiments, or modifications as long as there is no contradiction in the content. Similarly, the modifications can be combined with the prior art, embodiments, or other modifications as long as there is no contradiction in the content. In the following description, the same components may be given common reference numerals, and redundant descriptions may be omitted.

[0009] (First Embodiment) FIG. 1 is a diagram showing an example of the configuration of the image processing apparatus 100 according to the first embodiment. For example, the image processing apparatus 100 is installed in a medical facility such as a hospital or a clinic. And the image processing apparatus 100 is communicably connected to the modality via a network.

[0010] Such modalities are, for example, medical image generation devices that generate medical image data such as X-ray CT (Computed Tomography) devices, ultrasound diagnostic devices, magnetic resonance imaging (MRI) devices, PET (Positron Emission Tomography) devices, or SPECT (Single Photon Emission Computed Tomography) devices. For example, a modality generates medical image data (medical image data depicting the area of ​​interest) of a subject (patient). The area of ​​interest is, for example, the area of ​​a lesion such as a tumor. Such medical image data is either three-dimensional or two-dimensional. Examples of medical image data include CT image data, ultrasound image data, MR image data, PET image data, and SPECT image data. The modality then transmits the generated medical image data to the image processing device 100 via a network.

[0011] The image processing device 100 acquires medical image data from a modality connected via a network, performs image processing on the medical image data, and displays the results of the image processing. The image processing device 100 may also analyze the results of the image processing (for example, by measuring the volume of the region of interest). Furthermore, the image processing device 100 may save the results of the image processing in association with the medical image data, or output the results of the image processing to an external server. For example, the image processing device 100 may extract the region of interest captured in the medical image data and display the extraction results. The image processing device 100 can be implemented, for example, by computer equipment such as a server or workstation.

[0012] As shown in Figure 1, the image processing device 100 includes a network (NW) interface 101, a storage circuit 102, an input interface 103, a display 104, and a processing circuit 105.

[0013] The NW interface 101 controls the transmission and communication of various data sent and received between the image processing device 100 and other devices (modalities, etc.) connected to the image processing device 100 via a network. For example, the NW interface 101 is connected to the processing circuit 105 and receives data transmitted by other devices and transmits the received data to the processing circuit 105. Specifically, it receives medical image data transmitted by a modality and transmits the received medical image data to the processing circuit 105. The NW interface 101 also receives data transmitted by the processing circuit 105 and transmits the received data to other devices. For example, the NW interface 101 can be implemented by a network card, network adapter, NIC (Network Interface Controller), etc.

[0014] The memory circuit 102 stores various data and programs. Specifically, the memory circuit 102 is connected to the processing circuit 105 and stores various data under the control of the processing circuit 105. For example, the memory circuit 102 stores medical image data under the control of the processing circuit 105. The memory circuit 102 also functions as a work memory that temporarily stores various data used in processing performed by the processing circuit 105. For example, the memory circuit 102 can be implemented using semiconductor memory elements such as RAM (Random Access Memory) or flash memory, or by hard disks, optical discs, etc.

[0015] The input interface 103 receives various instructions and input operations for various information from the user of the image processing device 100. Specifically, the input interface 103 is connected to the processing circuit 105 and converts the input operations received from the user into electrical signals and transmits them to the processing circuit 105. For example, the input interface 103 can be implemented by a trackball, switch buttons, mouse, keyboard, touchpad that performs input operations by touching the operating surface, touchscreen that integrates a display screen and a touchpad, a non-contact input interface using an optical sensor, and an audio input interface. In this specification, the input interface 103 is not limited to those equipped with physical operating components such as a mouse or keyboard. For example, an electrical signal processing circuit that receives electrical signals corresponding to input operations from an external input device provided separately from the image processing device 100 and transmits these electrical signals to the processing circuit 105 is also included as an example of the input interface 103. Such a processing circuit can be implemented by a processor, for example. The input interface 103 is an example of a reception unit.

[0016] The display 104 displays various images, various information, and various data. Specifically, the display 104 is connected to the processing circuit 105 and displays images, various information, and various data based on various image data received from the processing circuit 105. For example, the display 104 displays medical images based on medical image data. For example, the display 104 can be implemented as an LCD monitor, a CRT (Cathode Ray Tube) monitor, a touch panel, etc. The display 104 is an example of a display unit.

[0017] The processing circuit 105 controls the entire image processing device 100. For example, the processing circuit 105 performs various processes in response to input operations received from the user via the input interface 103. The processing circuit 105 is implemented, for example, by a processor.

[0018] Furthermore, when the processing circuit 105 receives medical image data transmitted via the NW interface 101, it stores the received medical image data in the storage circuit 102.

[0019] For example, as shown in Figure 1, the processing circuit 105 includes a first extraction function 105a, a deletion function 105b, a determination function 105c, a direction acquisition function 105d, a second extraction function 105e, an integration function 105f, and a display control function 105g. The first extraction function 105a is an example of the first extraction unit. The deletion function 105b is an example of the deletion unit. The determination function 105c is an example of the determination unit. The direction acquisition function 105d is an example of the direction acquisition unit. The second extraction function 105e is an example of the second extraction unit. The integration function 105f is an example of the integration unit and also an example of the second extraction unit. The display control function 105g is an example of the display control unit.

[0020] Here, for example, the processing functions of the processing circuit 105 shown in Figure 1, namely the first extraction function 105a, deletion function 105b, determination function 105c, direction acquisition function 105d, second extraction function 105e, integration function 105f, and display control function 105g, are stored in the memory circuit 102 in the form of programs that can be executed by a computer. The processing circuit 105 reads each program from the memory circuit 102 and executes each program it has read to realize the function corresponding to each program. In other words, the processing circuit 105 in the state in which each program has been read has the functions shown in the processing circuit 105 of Figure 1.

[0021] The configuration example of the image processing apparatus 100 according to this embodiment has been described above. According to this embodiment, the image processing apparatus 100 performs various processes described below so that the region of interest captured in the medical image data can be extracted (segmented) with high accuracy and speed. The following description will focus on the case where the image processing apparatus 100 performs various processes on three-dimensional medical image data. However, the same processes that the image processing apparatus 100 performs on three-dimensional medical image data may also be performed on two-dimensional medical image data. In addition, the drawings referenced in the following description may appear to correspond to two-dimensional medical image data rather than three-dimensional medical image data. However, in reality, each drawing corresponds to three-dimensional medical image data.

[0022] An example of region extraction processing performed by the image processing device 100 will be described. Region extraction processing is the process of extracting a region of interest from medical image data. Here, the region of interest is the region to be extracted (the region to be extracted). Figures 2A and 2B are flowcharts illustrating an example of the flow of region extraction processing performed by the image processing device 100 according to the first embodiment. The region extraction processing shown in Figures 2A and 2B is executed when the user inputs an instruction to the processing circuit 105 via the input interface 103 to execute region extraction processing on the medical image data to be processed, while the medical image data to be processed is stored in the storage circuit 102.

[0023] As shown in Figure 2A, the first extraction function 105a sets one point within the region of interest (the region of interest depicted in the 3D medical image data) that is captured in the 3D medical image data to be processed as a designated point (step S101).

[0024] An example of the processing in step S101 will be described. Figures 3 to 13 are diagrams illustrating an example of the processing performed by the image processing apparatus 100 according to the first embodiment. In step S101, first, the first extraction function 105a acquires the medical image data to be processed stored in the memory circuit 102. Then, as shown in Figure 3, the first extraction function 105a sets one point within the area of ​​interest 11 captured in the medical image data as the designated point 12.

[0025] Various methods can be used to set the designated point 12. For example, the first extraction function 105a displays a medical image based on medical image data on the display 104. As a result, the display 104 displays the area of ​​interest 11 captured in the medical image data (medical image). A user, such as a doctor, confirms the area of ​​interest 11 and specifies the position of the designated point 12 within the area of ​​interest 11 by operating the input interface 103. The first extraction function 105a then accepts the position specified by the user as the position of the designated point 12. The first extraction function 105a then sets the accepted position of the designated point 12. In this way, the designated point 12 is set. In this embodiment, the image processing device 100 extracts the area of ​​interest 11 including the designated point 12 from the medical image data.

[0026] The first extraction function 105a may also automatically detect candidate lesion points from medical image data using conventional technology and set the automatically detected candidate lesion points as designated points 12. In this context, candidate lesion points are, for example, points within the area of ​​a lesion.

[0027] Next, as shown in Figure 3, the first extraction function 105a defines a predetermined fixed-size (fixed-size 1) three-dimensional image range containing the specified point 12 in the medical image data as the processing range 1, obtains the coordinates of the processing range 1, and obtains three-dimensional partial image data 1 within the processing range 1 (step S102). The processing range 1 is, for example, an area indicated by a cube of a predetermined size with the specified point 12 set as its center position. That is, the processing range 1 is an area defined by the six faces of a cube of a predetermined size. In step S102, since the length of one side of the processing range 1 is predetermined and the orientation of each of the 12 sides of the processing range 1 in the image coordinate system, which is a three-dimensional orthogonal (X,Y,Z) coordinate system, the first extraction function 105a may obtain the coordinates of one vertex of the processing range 1 as coordinates that can define the processing range 1. Note that the shape of the processing range 1 may be a shape other than a cube. For example, the processing range 1 may be an area indicated by a rectangular parallelepiped.

[0028] Next, the first extraction function 105a, as a region extraction means for extracting a region of interest from medical image data, uses a pre-trained segmentation inference model to obtain an inference probability map 1 corresponding to the partial image data 1 (S103). An example of the processing in step S103 will be described. For example, "3D U-Net" is used as the inference model described above. Here, "U-Net" is a known segmentation inference model (consisting of an encoder and a decoder) that uses deep neural network technology. In this embodiment, a "U-Net" created to take a 3D image as input and output a 3D probability map corresponding to the input is called a "3D U-Net". The inference model is stored in the memory circuit 102 in advance. The inference model is a neural network that, when a predetermined fixed-size partial image data 1 is input, outputs an inference probability map 1, which is a map showing the probability (inference probability) that each pixel of the partial image data 1 is a pixel included in the region of interest 11, as shown in Figure 4. Such probabilities are between 0.0 and 1.0.

[0029] In step S103, the first extraction function 105a retrieves the inference model stored in the memory circuit 102. Then, the first extraction function 105a inputs the partial image data 1 to the retrieved inference model and retrieves the inference probability map 1 output from the inference model.

[0030] In step S103, the first extraction function 105a may resize (isotropize) the partial image data 1 so that the pixel spacing of the partial image data 1 becomes 1 × 1 × 1 [mm], and input the isotropized image data obtained as a result of the resizing into the inference model. In this case, the coordinates of the specified point 12 are converted to the coordinates corresponding to the isotropized image data. Also in this case, the size of the partial image data 1 is set to a predetermined size in the isotropized image data. For example, when the size of the partial image data 1 in the isotropized image data is m × m × m [pixels], if the pixel spacing of the original image is 0.5 × 0.5 × 2.0 [mm], then a partial image data 1 of size 2m × 2m × (m / 2) [pixels] is obtained from the original image data.

[0031] Next, the first extraction function 105a sets the threshold (binarization threshold) T1, which will be used in step S105 described later, to an initial value of "0.5" (step S104). Note that the initial value is not limited to "0.5". Any value greater than 0.0 and less than 1.0 may be used as the initial value.

[0032] Next, the first extraction function 105a uses a binarization threshold T1 to perform a binarization process on the inference probability map 1 obtained in step S103 to obtain the extracted region 1 (step S105).

[0033] An example of the processing in step S105 is described below. In step S105, the first extraction function 105a performs a binarization process on the inference probability map 1 using a binarization threshold T1 to obtain the three-dimensional binarized image data 13 shown in Figure 5. That is, pixels in the binarized image data 13 corresponding to pixels in the inference probability map 1 that have a value greater than or equal to the binarization threshold T1 are set to a value indicating that they are within the extraction region 1 (the value corresponding to black in the example in Figure 5). Also, pixels in the binarized image data 13 corresponding to pixels in the inference probability map 1 that have a value less than the binarization threshold T1 are set to a value indicating that they are outside the extraction region 1 (the value corresponding to white in the example in Figure 5). Then, as shown in Figure 5, the first extraction function 105a extracts the region consisting of pixels in the binarized image data 13 that have a value indicating that they are within the extraction region 1 as the extraction region 1. Note that the example in Figure 5 shows the case where the first extraction function 105a extracts the extraction region 1, which includes two subregions, from the binarized image data 13. Furthermore, the method for obtaining the extracted region 1 from the inference probability map 1 is not limited to the method described above. The first extraction function 105a may obtain the extracted region 1 using any method.

[0034] Next, the first extraction function 105a calculates the size of the extraction region 1 extracted in step S105 and determines whether the calculated size is greater than or equal to a predetermined size (minimum size) (step S106). For example, the size of the extraction region 1 can be the number of pixels or the volume of the extraction region 1. The minimum size may also be set by the user. If volume is used as the size of the extraction region 1, for example, the minimum size may be 10 mm. 3 This is the result.

[0035] If the size of the extraction region 1 is greater than or equal to the minimum size (step S106: Yes), the first extraction function 105a proceeds to step S108. On the other hand, if the size of the extraction region 1 is less than the minimum size (step S106: No), the first extraction function 105a reduces the binarization threshold T1 according to a predetermined rule (step S107). Here, the predetermined rule is, for example, the rule to multiply the binarization threshold T1 by "0.2". The first extraction function 105a may also readjust the binarization threshold T1 based on other methods. For example, it may perform the same process when the specified point 12 is not included in the extraction region 1. This makes it possible to include the specified point 12 in the region of interest. Alternatively, the first extraction function 105a may determine the binarization threshold T1 based on the specified point 12 so that the specified point 12 is included in the region of interest. For example, the first extraction function 105a may determine the binarization threshold T1 based on the values ​​of the inference probability map 1 at the designated point 12 or its neighboring locations. As an example, the first extraction function 105a may set the binarization threshold T1 to a value lower than the values ​​of the inference probability map 1 at the designated point 12 or its neighboring locations.

[0036] Then, the first extraction function 105a returns to step S105. In this case, in step S105, the first extraction function 105a uses the binarization threshold T1 reduced in step S107 to perform a binarization process on the inference probability map 1 obtained in step S103 to obtain the extracted region 1. Then, the first extraction function 105a executes the processing of the steps from step S106 onwards again. Note that if the binarization threshold T1 reduced in step S107 becomes smaller than a predetermined minimum threshold (for example, 0.02), the first extraction function 105a sets the minimum threshold to the binarization threshold T1 and proceeds from step S107 to step S108 without returning from step S105.

[0037] The processing in steps S106 and S107 increases the likelihood of extracting an extraction region 1 that is larger than or equal to the minimum size. Note that the processing in steps S106 and S107 may be omitted. That is, the first extraction function 105a may execute the processing in step S108, which will be described later, after executing the processing in step S105, without executing the processing in steps S106 and S107.

[0038] As described above, the first extraction function 105a extracts an extraction region 1, which is presumed to be the region of interest 11, from the partial image data 1 included within the processing range 1 that includes the specified point 12 of the medical image data. Processing range 1 is an example of the first processing range. Partial image data 1 is an example of the first partial image data. Extraction region 1 is an example of the first extraction region. Furthermore, the first extraction function 105a obtains an inference probability map 1 by calculating the probability that each pixel of the partial image data 1 is a pixel included in the region of interest 11, and extracts the extraction region 1 by binarizing the inference probability map 1. Inference probability map 1 is an example of the first inference probability map.

[0039] Furthermore, as described above, if the size of the extraction region 1 is smaller than a predetermined size (minimum size), the first extraction function 105a reduces the binarization threshold T1 used when binarizing the inference probability map 1 based on a predetermined rule, and then extracts the extraction region 1 by binarizing the inference probability map 1 again using the reduced binarization threshold T1.

[0040] Next, the deletion function 105b determines whether the extracted region 1 contains multiple subregions that are not connected to each other (step S108). For example, in the case shown in Figure 5, the deletion function 105b determines that the extracted region 1 contains two subregions that are not connected to each other (step S108: Yes).

[0041] If the extracted region 1 does not contain multiple subregions that are not connected to each other (step S108: No), that is, if the extracted region 1 is a single region, the deletion function 105b proceeds to step S111.

[0042] On the other hand, if the extracted region 1 contains multiple subregions that are not connected to each other (step S108: Yes), the deletion function 105b deletes the other subregions, leaving only a specific subregion (step S109).

[0043] An example of the processing in step S109 will be described. For example, in step S109, the deletion function 105b deletes all but the sub-region containing the specified point 12 from among multiple sub-regions. As a result, the deletion function 105b obtains the sub-region containing the specified point 12 as the new extracted region 1. For example, as shown in Figure 6, in step S109, the deletion function 105b deletes all but the sub-region 14a containing the specified point 12 from among multiple sub-regions 14a, 14b. That is, if the extracted region 1 contains multiple sub-regions 14a, 14b that are not connected to each other, the deletion function 105b deletes the sub-region 14b that does not contain the specified point 12 from among the multiple sub-regions 14a, 14b. As a result, the deletion function 105b obtains sub-region 14a as the new extracted region 1.

[0044] In step S109, the deletion function 105b may delete all subregions except for the subregion whose centroid is closest to the designated point 12. That is, if the extracted region 1 includes multiple subregions that are not connected to each other, the deletion function 105b calculates the centroid of each of the multiple subregions and deletes all subregions except the one whose centroid is closest to the designated point 12. As a result, the deletion function 105b obtains the subregion whose centroid is closest to the designated point 12 as the new extracted region 1. Alternatively, if the extracted region 1 includes multiple subregions that are not connected to each other, the deletion function 105b may delete all subregions except the one whose center of the bounding rectangle is closest to the designated point 12. As a result, the deletion function 105b obtains the subregion closest to the center of the bounding rectangle of the multiple subregions, or the subregion containing the center of the bounding rectangle, as the new extracted region 1.

[0045] Furthermore, in step S109, the deletion function 105b may delete subregions other than the subregion with the largest size among the multiple subregions. As a result, the deletion function 105b obtains the subregion with the largest size as the new extracted region 1.

[0046] The process in step S109 removes unnecessary subregions that are highly likely to be mis-extracted, such as regions other than the region of interest 11. Furthermore, by removing these unnecessary subregions, the increase in computation time caused by expanding the processing range based on the mis-extracted regions can be suppressed.

[0047] Next, the first extraction function 105a sets the probability value set for all pixels of the inference probability map 1 acquired in step S103, excluding those corresponding to the pixels of the extraction region 1 (new extraction region 1) obtained in step S109, to "0.0" (step S110). Note that in step S110, the first extraction function 105a does not change the probability set for the pixels of the extraction region 1 obtained in step S109, among all pixels of the inference probability map 1 acquired in step S103. As a result, the first extraction function 105a obtains a new inference probability map 1 as shown in Figure 7. If an inference probability map 1 is obtained in step S110, the inference probability map 1 obtained in step S110 is used in place of the inference probability map 1 acquired in step S103 in the processing of steps S111 and later. The processing in steps S108 to S110 may be omitted.

[0048] Next, the first extraction function 105a substitutes the value of the binarization threshold T1 used in step S105 as an initial value for the binarization threshold T2 used in step S118, which will be described later (step S111). In step S111, the first extraction function 105a may substitute a predetermined initial value "0.5" for the binarization threshold T2. Note that the initial value is not limited to "0.5". Any value greater than 0.0 and less than 1.0 may be used as the initial value.

[0049] Here, the extracted region 1 may be only a part of the region of interest 11. In other words, the first extraction function 105a may not have extracted the entire region of interest 11. Therefore, as shown in Figure 2B, the determination function 105c determines whether or not the region of interest 11 extends beyond the processing range 1 (step S112).

[0050] Two methods can be used to determine step S112. The first method will be explained below. For example, the determination function 105c determines whether the number of pixels in the extracted region 1 that are located at any of the multiple edges (specifically, six in three dimensions (four in two dimensions)) of the processing range 1 shown in Figure 8 is greater than or equal to a predetermined threshold. More specifically, if the number of pixels in the extracted region 1 that are located at any of the multiple edges is greater than or equal to a predetermined threshold, the determination function 105c determines that the region of interest 11 extends beyond the processing range 1 in the direction of that edge. On the other hand, if the number of pixels located at any of the edges is less than a predetermined threshold for all edges, the determination function 105c determines that the region of interest 11 does not extend beyond the processing range 1.

[0051] Furthermore, in the first determination method, the determination function 105c may compare the ratio of the number of pixels in the extracted region 1 present in the edge to the number of pixels constituting the edge, with a predetermined threshold, rather than comparing the number of pixels with a threshold. Specifically, the determination function 105c determines whether the ratio of the number of pixels in the extracted region 1 present in any of the multiple edges of the processing range 1 to the number of pixels constituting that edge is greater than or equal to a predetermined threshold. For example, if the ratio of the number of pixels in the extracted region 1 present in any of the multiple edges of the edge is greater than or equal to a predetermined threshold, the determination function 105c determines that the area of ​​interest 11 extends beyond the processing range 1 in the direction of that edge. On the other hand, if, for all edges, the ratio of the number of pixels in the extracted region 1 present in the edge to the number of pixels constituting that edge is less than a predetermined threshold, the determination function 105c determines that the area of ​​interest 11 does not extend beyond the processing range 1.

[0052] Now, let's explain the edges. In this embodiment, as mentioned above, there are six edges in three dimensions. Various definitions of edges are possible. Let's explain the first definition of an edge. For example, in the first definition of an edge, each of the six edges is a region that includes only all the pixels that are adjacent to each of the six faces of the cube that defines the processing range 1, out of the multiple pixels within the processing range 1.

[0053] Let's explain the definition of the second edge region. For example, in the second definition of the edge region, each of the six edge regions is a region that includes only all pixels that are within a certain distance from each of the six faces of the cube that defines the processing range 1, out of the multiple pixels within the processing range 1.

[0054] Let's explain the definition of the third edge region. For example, in the third definition of the edge region, each of the six edge regions is a region that includes only all pixels that are located at a certain distance or more from the center of processing range 1 along the X, Y, and Z axes, out of the multiple pixels within processing range 1.

[0055] When the first determination method is used and the first rule is adopted in step S114 described later, the direction acquisition function 105d acquires information about which direction the edge is in which the number of pixels included in the extracted region 1 within the edge is greater than or equal to a predetermined threshold. Alternatively, the direction acquisition function 105d acquires information about which direction the edge is in which the above-mentioned ratio is greater than or equal to a predetermined threshold. In other words, the direction acquisition function 105d acquires information about the direction in which the region of interest 11 extends beyond the edge.

[0056] Next, the second determination method will be explained. For example, the determination function 105c determines whether the statistical quantities (statistical values) of multiple probabilities set for multiple pixels located at one of the multiple edges of the processing range 1, among the multiple pixels of the inference probability map 1, are greater than or equal to a predetermined threshold. Here, the statistical quantities of multiple probabilities are, for example, the sum, mean, or variance of multiple probabilities. More specifically, if the statistical quantities of multiple probabilities set for multiple pixels located at one of the multiple edges of the inference probability map 1 are greater than or equal to a predetermined threshold, the determination function 105c determines that the region of interest 11 extends beyond the processing range 1 in the direction of that edge. On the other hand, if the statistical quantities of multiple probabilities set for multiple pixels at all edges are less than the predetermined threshold, the determination function 105c determines that the region of interest 11 does not extend beyond the processing range 1.

[0057] When the second determination method is used and the first rule is adopted in step S114 described later, the direction acquisition function 105d acquires information about which direction the edge is in which the statistical quantities of multiple probabilities set for multiple pixels included in the inference probability map 1 within the edge are greater than or equal to a predetermined threshold. In other words, the direction acquisition function 105d acquires information about which direction the region of interest 11 extends beyond.

[0058] As described above, the determination function 105c determines whether the extracted region 1 is part of the region of interest 11 by determining whether at least a portion of the extracted region 1 is located at the edge of the processing range 1. The determination function 105c also determines whether the extracted region 1 is part of the region of interest 11 by determining whether the probability statistics of the inference probability map 1 located at the edge of the processing range 1 are greater than or equal to a predetermined value.

[0059] Furthermore, when the determination function 105c determines that the extracted region 1 is part of the region of interest 11, the direction acquisition function 105d acquires the direction in which the processing range 2, described later, is set based on the processing range 1. Processing range 2 is an example of a second processing range. In addition, the direction acquisition function 105d acquires the direction in which the extracted region 1 is determined to be part of the region of interest 11, based on whether or not the extracted region 1 is determined to be part of the region of interest 11 in any direction from the center of the processing range 1, as the direction in which the processing range 2 is set based on the processing range 1.

[0060] If the area of ​​interest 11 does not extend beyond the processing range 1 (step S112: No), the first extraction function 105a uses the extraction area 1 as the final extraction area (step S113) and terminates the area extraction process.

[0061] In this embodiment, the image processing device 100 outputs the final extracted region. Specifically, the display control function 105g controls the display 104 to display the final extracted region. This allows a user, such as a doctor, to confirm the extracted region displayed on the display 104 as the region of interest 11. Alternatively, the display control function 105g may superimpose the final extracted region onto the medical image data and control the display 104 to display the medical image with the final extracted region superimposed. In this case, the display control function 105g may superimpose the final extracted region as a semi-transparent color image or as a graphic such as a boundary line. This allows the user to understand the location of the region of interest 11 on the medical image by visually identifying the extracted region located on the medical image.

[0062] On the other hand, if the area of ​​interest 11 extends beyond the processing range 1 (step S112: Yes), the second extraction function 105e obtains a list of one or more processing ranges 2 based on predetermined rules, and obtains a list of partial image data 2 by acquiring partial image data 2 within each processing range 2 from the medical image data (step S114). The list of processing ranges 2 is, for example, information in which the coordinates of one or more processing ranges 2 are registered. The list of partial image data 2 is, for example, a set of partial image data 2.

[0063] Here, as a rule for obtaining the list of processing range 2, for example, either the first rule or the second rule is adopted. That is, the second extraction function 105e obtains the list of processing range 2 based on either of the two rules.

[0064] First, let's explain the first rule. The first rule obtains a list of processing ranges 2 shown in Figure 9, based on information about which direction the area of ​​interest 11 extends beyond.

[0065] Specifically, the first rule obtains the coordinates of the position in processing range 2 by shifting the position of processing range 1 by a predetermined length L1 in each of the directions in which the area of ​​interest 11 is deemed to extend beyond the processing range 1. For example, consider the case where the directions in which the area of ​​interest 11 is deemed to extend beyond the processing range 1 are the negative direction of the X axis, the positive direction of the Y axis, and the positive direction of the Z axis. In this case, the second extraction function 105e obtains the coordinates of seven processing ranges 2 by shifting the position of processing range 1 by (-L1,0,0), (-L1,+L1,0), (0,+L1,0), (0,0,+L1), (-L1,0,+L1), (-L1,+L1,+L1), and (0,+L1,+L1) in each direction of (X,Y,Z) based on the first rule.

[0066] Here, length L1 is a predetermined multiple of the length of one side of processing range 1 (for example, 1x, 4 / 5x, 2 / 3x, or 1 / 2x). When the predetermined multiple is 1x, processing range 1 and processing range 2 are touching, and there is no overlap between them. On the other hand, when the predetermined multiple is less than 1x, there is an overlap between processing range 1 and processing range 2.

[0067] The advantage of the first rule is that it reduces the processing time because it reduces the number of processing ranges 2. If the shape of the region of interest 11 is predicted to be roughly convex, like an ellipsoid, it is advisable to use the first rule, which expands the processing range 2 only in a limited direction.

[0068] Next, the second rule will be explained. The second rule obtains the list of processing ranges 2 shown in Figure 10 based on the determination result in step S112 that the area of ​​interest 11 extends beyond the processing range 1. In other words, the second rule obtains the list of processing ranges 2 without considering the direction in which the area of ​​interest 11 extends beyond the processing range 1.

[0069] Specifically, the second rule obtains the coordinates of the processing range 2 position by shifting the position of processing range 1 by a predetermined length L1 in the positive or negative direction along at least one of the X, Y, and Z axes. For example, the second extraction function 105e obtains 26 coordinates of processing range 2 based on the second rule.

[0070] The advantage of the second rule is that it reduces the risk of missing parts of the region of interest 11 because it expands the processing range 2 in all directions. If the shape of the region of interest 11 is expected to be very complex, it is advisable to use the second rule that expands the processing range 2 in all directions.

[0071] Next, the second extraction function 105e obtains a list of inference probability maps 2 by using an inference model to obtain an inference probability map 2 corresponding to each part image data 2 included in the list of part image data 2 (S115). The list of inference probability maps 2 is, for example, a set of inference probability maps 2.

[0072] An example of the processing in step S115 is described below. The inference model is stored in the memory circuit 102, and when a predetermined fixed-size (fixed-size 1) partial image data 2 is input, it outputs an inference probability map 2, which is a map showing the probability that each pixel of the partial image data 2 is a pixel included in the region of interest 11. This probability is a value between 0.0 and 1.0. Note that the inference model used in step S103 and the inference model used in step S115 may be the same neural network or different neural networks. If the inference model used in step S103 and the inference model used in step S115 are different, the inference model that has been optimally trained as a region extraction means is used in each step.

[0073] In step S115, the second extraction function 105e retrieves the inference model stored in the memory circuit 102. Then, the second extraction function 105e inputs the partial image data 2 to the retrieved inference model and obtains the inference probability map 2 output from the inference model.

[0074] In step S115, the second extraction function 105e may resize the partial image data 2 so that the pixel spacing of the partial image data 2 becomes 1 × 1 × 1 [mm], and input the isotropized image data obtained as a result of the resizing into the inference model. In this case, the coordinates of the specified point 12 are converted to the coordinates corresponding to the isotropized image data.

[0075] As described above, the second extraction function 105e extracts an inferred probability map 2 obtained by calculating the probability that each pixel of the partial image data 2 is a pixel included in the region of interest 11.

[0076] Next, the integration function 105f obtains a new inference probability map 1 by integrating inference probability map 1 and all inference probability maps 2 included in the list of inference probability maps 2 obtained in step S115, so that their positions match based on their respective coordinates (step S116).

[0077] In step S116, any method is acceptable for integrating inference probability map 1 with all inference probability maps 2 included in the list of inference probability maps 2. Three specific examples are given below to illustrate this.

[0078] In the first example, if there are overlapping pixel locations in at least two of the inference probability maps included in the lists of inference probability map 1 and inference probability map 2, the integration function 105f takes the highest probability among the multiple probabilities at the overlapping pixel locations as the probability in the new inference probability map 1. Thus, in the first example, if there is an overlapping region between inference probability map 1 and inference probability map 2, the integration function 105f obtains a new inference probability map 1 by taking the probability of each pixel in the region corresponding to the overlapping region in the new inference probability map 1 as the larger of the probability of each pixel in the overlapping region of inference probability map 1 and the probability of each pixel in the overlapping region of inference probability map 2.

[0079] In the second example, if there are overlapping pixel locations in at least two of the inference probability maps included in the lists of inference probability map 1 and inference probability map 2, the integration function 105f takes the average of multiple probabilities at the overlapping pixel locations as the probability in the new inference probability map 1. Thus, in the second example, for example, if there is an overlapping region between inference probability map 1 and inference probability map 2, the integration function 105f obtains a new inference probability map 1 by taking the probability of each pixel in the region corresponding to the overlapping region in the new inference probability map 1 as the average of the probability of each pixel in the overlapping region of inference probability map 1 and the probability of each pixel in the overlapping region of inference probability map 2.

[0080] In the third example, if there are overlapping pixel locations in at least two of the inference probability maps included in the lists of inference probability map 1 and inference probability map 2, the integration function 105f assigns a weighted sum to each probability at the overlapping pixel locations, based on the distance from the center of the processing range to which each pixel belongs, and uses this sum as the probability in the new inference probability map 1. The center of the processing range tends to have higher probability accuracy, and the accuracy of the probability tends to decrease as you move away from the center of the processing range. For this reason, the integration function 105f multiplies the probability by a weight that increases as you move closer to the center of the processing range and decreases as you move away from the center of the processing range, and uses the sum of multiple weighted probabilities (e.g., two probabilities) as the probability in the new inference probability map 1. This makes it possible to obtain an inference probability map 1 with high-accuracy probabilities.

[0081] Thus, in the third example, if there is an overlapping region between inference probability map 1 and inference probability map 2, the integration function 105f calculates the probability of each pixel in the region corresponding to the overlapping region in the new inference probability map 1 by adding a first value obtained by multiplying the probability of each pixel in the overlapping region of inference probability map 1 by a weight that decreases as it moves further away from the center of processing range 1, and a second value obtained by multiplying the probability of each pixel in the overlapping region of inference probability map 2 by a weight that decreases as it moves further away from the center of processing range 2.

[0082] Next, the integration function 105f integrates processing range 1 and all processing ranges 2 included in the list of processing ranges 2 to obtain a new processing range 1 (step S117). For example, the integration function 105f integrates the processing range 1 shown in Figure 9 or Figure 10 with all processing ranges 2 so that their positions align, thereby obtaining a new processing range 1 as shown in Figure 11.

[0083] Next, the integration function 105f uses the binarization threshold T2 to perform a binarization process on the inference probability map 1 obtained in step 116 to obtain a new extracted region 1 (step S118).

[0084] An example of the processing in step S118 is described below. In step S118, the integration function 105f performs a binarization process on the inference probability map 1 using a binarization threshold T2 to obtain the three-dimensional binarized image data 15 shown in Figure 11. That is, pixels in the binarized image data 15 corresponding to pixels in the inference probability map 1 that have a value greater than or equal to the binarization threshold T2 are set to a value indicating that they are within the extraction region 1 (the value corresponding to black in the example in Figure 11). Also, pixels in the binarized image data 15 corresponding to pixels in the inference probability map 1 that have a value less than the binarization threshold T2 are set to a value indicating that they are outside the extraction region 1 (the value corresponding to white in the example in Figure 11). Then, as shown in Figure 11, the integration function 105f extracts a new extraction region 1 from all the pixels in the binarized image data 15 that are composed of pixels that have a value indicating that they are within the extraction region 1. Note that the example in Figure 11 shows the case where the integration function 105f extracts an extraction region 1 containing two subregions from the binarized image data 15.

[0085] Here, the new extracted region 1 obtained in step S118 includes the extracted region obtained from the partial image data 2 included in the list of partial image data 2 acquired in step S114. Here, the extracted region obtained from the partial image data 2 is referred to as "extracted region 2". This extracted region 2 is obtained by the second extraction function 105e and the integration function 105f. That is, the second extraction function 105e obtains an inference probability map 2 obtained by calculating the probability that each pixel of the partial image data 2 is a pixel included in the region of interest 11. Then, the integration function 105f extracts the extracted region 2 by binarizing the inference probability map 2. Specifically, the integration function 105f extracts the region consisting of pixels that have a value indicating they are within the extracted region 1 from all the pixels of the binarized image data 15 obtained by binarizing the inference probability map 2 as the extracted region 2. In other words, the second extraction function 105e and integration function 105f extract an extraction region 2 that is presumed to be the region of interest 11 from the medical image data, based on the assumption that the extracted region 1 extracted in the partial image data 1 is part of the region of interest 11, from the partial image data 2 that is adjacent to the edge of the processing range 1 or is included within the processing range 2 that includes at least a part of the edge. Partial image data 2 is an example of the second partial image data. Inference probability map 2 is an example of the second inference probability map. Extracted region 2 is an example of the second extracted region.

[0086] Furthermore, as described above, the second extraction function 105e sets the processing range 2 at a position shifted by a predetermined distance from the processing range 1 in the direction acquired by the direction acquisition function 105d. Alternatively, the second extraction function 105e sets the processing range 2 at positions shifted by a predetermined distance in all directions where the edges of the processing range 1 exist. Then, the integration function 105f extracts the extraction region 2 from the partial image data 2 contained within the set processing range 2.

[0087] Next, the deletion function 105b determines whether the extracted region 1 obtained in step S118 contains multiple subregions that are not connected to each other (step S119). For example, in the case shown in Figure 11, the deletion function 105b determines that the extracted region 1 contains two subregions that are not connected to each other (step S119: Yes).

[0088] If the extracted region 1 does not contain multiple subregions that are not connected to each other (step S119: No), that is, if the extracted region 1 is a single region, the deletion function 105b proceeds to step S122.

[0089] On the other hand, if the extracted region 1 contains multiple subregions that are not connected to each other (step S119: Yes), the deletion function 105b deletes the other subregions, leaving only a specific subregion (step S120). In step S120, for example, the deletion function 105b performs the same processing as in step S109.

[0090] As shown in Figure 12, in step S120, the deletion function 105b deletes the other subregions 16b, leaving only the subregion 16a containing the specified point 12. As a result, the deletion function 105b obtains subregion 16a as a new extracted region 1.

[0091] The process in step S120 removes unnecessary subregions that are highly likely to be mis-extracted, such as regions other than the region of interest 11. Furthermore, by removing these unnecessary subregions, the increase in computation time caused by expanding the processing range based on the mis-extracted regions can be suppressed.

[0092] Next, the integration function 105f sets the probability values ​​for all pixels of the inference probability map 1 obtained in step S116, excluding those corresponding to the pixels of the extraction region 1 (new extraction region 1) obtained in step S120, to "0.0" (step S121). Note that in step S121, the integration function 105f does not change the probabilities set for the pixels of the extraction region 1 obtained in step S120, among all pixels of the inference probability map 1 obtained in step S116. As a result, the integration function 105f obtains a new inference probability map 1 as shown in Figure 13. If an inference probability map 1 is obtained in step S121, the inference probability map 1 obtained in step S121 is used instead of the inference probability map 1 obtained in step S116 in the processing of steps S122 and later. The processing of steps S119 to S121 may be omitted.

[0093] Next, the integration function 105f increases the binarization threshold T2 according to a predetermined rule (step S122). Here, the predetermined rule is, for example, to add "0.2" to the value of the binarization threshold T2. The further the processing range 2 is from the designated point 12, the more likely it is to mistakenly extract a region that is not the region of interest 11. Therefore, the processing in step S122 can suppress the mistaken extraction of a region at a position far from the designated point 12.

[0094] Then, the integration function 105f returns to step S112. If the binarization threshold T2, which was increased in step S122, becomes greater than a predetermined maximum threshold (for example, 0.9 or 1.0), the integration function 105f sets the binarization threshold T2 to the maximum threshold and returns to step S112. Note that the processing in step S122 may be omitted.

[0095] By returning to step S112, the process from steps S112 to S122 is repeatedly executed as long as it is determined that the region of interest 11 extends beyond the extraction region 1. In other words, the processing range 1 is repeatedly expanded.

[0096] As described above, the integration function 105f sets the binarization threshold T2 used when binarizing the inference probability map 2 to be greater than the binarization threshold T1 used when binarizing the inference probability map 1 based on a predetermined rule. Then, in step S118, the integration function 105f uses the increased binarization threshold T2 to binarize the inference probability map 2 included in the new inference probability map 1, thereby extracting the extraction region 2 included in the new extraction region 1.

[0097] Furthermore, the integration function 105f integrates extraction region 1 and extraction region 2 to obtain a new extraction region.

[0098] The image processing apparatus 100 according to the first embodiment has been described above. According to the first embodiment, if the area of ​​interest 11 is larger than the processing range 1, each process from step S112 to step S122 is repeatedly executed the required number of times, thereby repeatedly expanding the processing range 1. As a result, areas of interest 11 of various sizes can be extracted with high accuracy.

[0099] Furthermore, the expansion of processing range 1 is stopped in step S113. Therefore, compared to conventional techniques that divide the entire image into a grid and process it, the number of processing steps can be reduced, resulting in faster processing.

[0100] Therefore, according to the image processing apparatus 100 of the first embodiment, the region of interest 11 captured in the medical image data can be extracted with high accuracy and at high speed.

[0101] (First modification of the first embodiment) Next, an image processing apparatus 100 according to a first modification of the first embodiment will be described. In the following description, the differences between the first embodiment and the first modification of the first embodiment will be mainly described, and descriptions of identical or similar components may be omitted. Also, in the following description, components identical or similar to those in the first embodiment may be denoted by the same reference numerals and their descriptions may be omitted.

[0102] Figure 14 is a diagram illustrating an example of processing performed by the image processing device 100 according to the first modification of the first embodiment. In the first modification of the first embodiment, for example, as shown in Figure 14, region information 102a indicating at least one of the regions of the body part or organ in which the region of interest 11 exists, and the regions of the body part or organ in which the region of interest 11 does not exist, is stored in the memory circuit 102. For example, if the region of interest 11 is a pulmonary nodule, the pulmonary nodule is located in the lung field region. Therefore, in step S114, the second extraction function 105e extracts the lung field region from the medical image data. The second extraction function 105e then stores the region information 102a indicating the extracted lung field region in the memory circuit 102.

[0103] Then, in step S114, the second extraction function 105e acquires region information 102a from the memory circuit 102. Then, in step S114, when the second extraction function 105e acquires a list of coordinates for the processing range 2, it acquires coordinates that are included only within the lung field region indicated by the region information 102a from among the multiple coordinates for the processing range 2 obtained based on the first rule or the second rule, as the coordinates for the processing range 2. In other words, among the multiple coordinates for the processing range 2 obtained based on the first rule or the second rule, the coordinates that are included only within the lung field region indicated by the region information 102a will be included in the list of coordinates for the processing range 2. To put it another way, among the multiple coordinates for the processing range 2 obtained based on the first rule or the second rule, coordinates that are included in the region outside the lung field region indicated by the region information 102a will not be included in the list of coordinates for the processing range 2.

[0104] Furthermore, for example, if the area of ​​interest 11 is a tumor that has metastasized to the lymph nodes, such a tumor will be located in the trunk region and not in the bone region. Therefore, in step S114, the second extraction function 105e extracts the trunk region and the bone region from the medical image data. The second extraction function 105e then stores the region information 102a indicating the extracted trunk region and bone region in the memory circuit 102.

[0105] Then, in step S114, the second extraction function 105e acquires region information 102a from the memory circuit 102. Then, in step S114, when the second extraction function 105e acquires a list of coordinates for the processing range 2, it acquires coordinates that are within the trunk region indicated by the region information 102a and are included in the region outside the bone region indicated by the region information 102a from among the multiple coordinates for the processing range 2 obtained based on the first rule or the second rule, as the coordinates for the processing range 2. In other words, among the multiple coordinates for the processing range 2 obtained based on the first rule or the second rule, coordinates that are within the trunk region indicated by the region information 102a and are included in the region outside the bone region will be included in the list of coordinates for the processing range 2.

[0106] Therefore, in the first modification of the first embodiment, the second extraction function 105e sets a processing range 2 using region information 102a indicating at least one of the regions in which the region of interest 11 exists and regions in which the region of interest 11 does not exist, and extracts an extraction region 2 that is presumed to be the region of interest 11 from the partial image data 2 included within the set processing range 2.

[0107] According to the image processing apparatus 100 of the first embodiment, which is a first modification, false detection of the area of ​​interest 11 in areas where the area of ​​interest 11 does not exist can be reduced. In addition, since the range in which the processing range 2 is set is limited, the processing time can be reduced.

[0108] (Second modification of the first embodiment) Next, an image processing apparatus 100 according to a second modified example of the first embodiment will be described. In the following description, the differences between the first embodiment and the second modified example of the first embodiment will be mainly described, and descriptions of identical or similar components may be omitted. Also, in the following description, components identical or similar to those in the first embodiment may be denoted by the same reference numerals and their descriptions may be omitted.

[0109] In the first embodiment, the case in step S113 where the extracted region 1 is the final extracted region was described. On the other hand, in this modified example, the inference probability map 1 becomes the final extracted region. This modified example will be described in detail. In this modified example, if in step S112 it is determined that the region of interest 11 does not extend beyond the processing range 1 (step S112: No), then in step S113, the new inference probability map 1 obtained in the most recent step S116 is set as the final extracted region. The new inference probability map 1 is then displayed on the display 104. If the processing in step S116 is not performed and a new inference probability map 1 is obtained in step S110, this inference probability map 1 is set as the final extracted region and displayed on the display 104. Furthermore, if the processing in step S116 and the processing in step S110 are not performed, the inference probability map 1 obtained in step S103 is set as the final extracted region and displayed on the display 104.

[0110] Here, the new inference probability map 1 obtained in step S116 is an inference probability map obtained by integrating the inference probability map 1 obtained in step S103 or S110 with one or more inference probability maps 2 included in the list of inference probability maps 2 obtained in step S115.

[0111] Therefore, in this modified example, the inference probability map 1 obtained in step S103 or S110, and one or more inference probability maps 2 included in the list of inference probability maps 2 acquired in step S115, are included in the final extraction region. That is, in this modified example, the first extraction function 105a extracts the inference probability map 1 obtained in step S103 or S110 as part of the final extraction region. Also in this modified example, the integration function 105f extracts one or more inference probability maps 2 included in the list of inference probability maps 2 acquired in step S115 as part of the final extraction region. Then, in step S113, the display control function 105g controls the display 104 to display a new inference probability map 1 that includes the inference probability map 1 obtained in step S103 or S110, and one or more inference probability maps 2 included in the list of inference probability maps 2 acquired in step S115.

[0112] The image processing device 100 according to the second modification of the first embodiment has been described above. According to the image processing device 100 according to the second modification of the first embodiment, the user can view a new inference probability map 1.

[0113] (Second embodiment) Next, an image processing apparatus 100 according to the second embodiment will be described. In the following description, the differences between the first embodiment and the second embodiment will be mainly described, and descriptions of identical or similar components may be omitted. Also, in the following description, components identical or similar to those in the first embodiment may be denoted by the same reference numerals and their descriptions may be omitted.

[0114] Figure 15 shows an example of a fixed size 1 used in the first embodiment described above. Figure 16 shows an example of a fixed size 2 used in the second embodiment.

[0115] The inference model (trained model, neural network) used in step S103 of the first embodiment is pre-trained using training data of a fixed size (fixed size 1). For example, the fixed size 1 shown in Figure 15 is larger than the average size of the region of interest 11. This is because a larger size reduces the number of times the processing range 1 is expanded for larger regions of interest 11, thereby reducing computation time.

[0116] However, if the size of the region of interest 11 is much smaller than the fixed size 1, using a region extraction method (inference model, trained model, neural network) trained with a fixed size smaller than fixed size 1 (fixed size 2 shown in Figure 16) will result in higher region extraction accuracy.

[0117] Therefore, in the second embodiment, when it is estimated that the size of the area of ​​interest 11 is smaller than the fixed size 2, the image processing device 100 performs the following processing to improve the accuracy of area extraction.

[0118] Figures 17A and 17B are flowcharts illustrating an example of the region extraction process performed by the image processing apparatus 100 according to the second embodiment. The region extraction process shown in Figures 17A and 17B is executed when the user inputs an instruction to the processing circuit 105 via the input interface 103, with the medical image data to be processed stored in the memory circuit 102.

[0119] As shown in Figures 17A and 17B, in the second embodiment, the image processing apparatus 100 executes the processes of steps S201 to S211 between the processes of step S101 and step S102. Also, the processes of steps S201 to S209 are the same as the processes of steps S102 to S110. However, in the processes of steps S201 to S209, a fixed size 2 smaller than fixed size 1 is used, and in the processes of steps S102 to S110, fixed size 1 is used. Figure 18 is a diagram illustrating an example of the processes executed by the image processing apparatus 100 according to the second embodiment.

[0120] As shown in Figures 17A and 18, the first extraction function 105a defines a predetermined fixed-size (fixed-size 2) three-dimensional image range including the specified point 12 in the medical image data as the processing range 3, obtains the coordinates of the processing range 3, and obtains three-dimensional partial image data 3 within the processing range 3 (step S201). The processing range 3 is, for example, an area indicated by a cube or rectangular prism of a predetermined size with the specified point 12 set as its center position. That is, the processing range 3 is an area defined by the six faces of a cube or rectangular prism of a predetermined size. In step S201, since the length of one side of the processing range 3 is predetermined and the orientation of each of the 12 sides of the processing range 3 in the image coordinate system, which is a three-dimensional orthogonal (X,Y,Z) coordinate system, the first extraction function 105a may obtain the coordinates of one vertex of the processing range 3 as coordinates that can define the processing range 3.

[0121] Next, the first extraction function 105a, as a region extraction means for extracting a region of interest from medical image data, uses a pre-trained inference model to obtain an inference probability map 3 corresponding to the partial image data 3 (S202). An example of the processing in step S202 will be described. For example, the inference model is pre-stored in the memory circuit 102. The inference model is a neural network that, when a predetermined fixed-size partial image data 3 of a fixed size 2 is input, outputs an inference probability map 3, which is a map showing the probability that each pixel of the partial image data 3 is a pixel included in the region of interest 11.

[0122] In step S202, the first extraction function 105a retrieves the inference model stored in the memory circuit 102. Then, the first extraction function 105a inputs the partial image data 3 to the retrieved inference model and obtains the inference probability map 3 output from the inference model.

[0123] In step S202, the first extraction function 105a may resize (isotropize) the partial image data 1 so that the pixel spacing of the partial image data 3 becomes 1 × 1 × 1 [mm], and input the isotropized image data obtained as a result of the resizing into the inference model. In this case, the coordinates of the specified point 12 are converted to the coordinates corresponding to the isotropized image data.

[0124] Next, the first extraction function 105a sets the threshold (binarization threshold) T3, which will be used in step S204 described later, to an initial value of "0.5" (step S203). Note that the initial value is not limited to "0.5". Any value greater than 0.0 and less than 1.0 may be used as the initial value.

[0125] Next, the first extraction function 105a uses a binarization threshold T3 to perform a binarization process on the inference probability map 3 obtained in step S202 to obtain the extracted region 3 (step S204).

[0126] Next, the first extraction function 105a calculates the size of the extracted region 3 extracted in step S204 and determines whether the calculated size is greater than or equal to a predetermined minimum size (step S205). For example, the size of the extracted region 3 may be the number of pixels or the volume of the extracted region 3. The minimum size may also be set by the user.

[0127] If the size of the extraction region 3 is greater than or equal to the minimum size (step S205: Yes), the first extraction function 105a proceeds to step S207. On the other hand, if the size of the extraction region 3 is less than the minimum size (step S205: No), the first extraction function 105a reduces the binarization threshold T3 according to a predetermined rule (step S206). Here, the predetermined rule is, for example, to multiply the binarization threshold T3 by "0.2".

[0128] Then, the first extraction function 105a returns to step S204. In this case, in step S204, the first extraction function 105a uses the binarization threshold T3 reduced in step S206 to perform a binarization process on the inference probability map 3 obtained in step S202 to obtain the extracted region 3. Then, the first extraction function 105a executes the processing of the steps from step S205 onwards again. Note that if the binarization threshold T3 reduced in step S206 becomes smaller than a predetermined minimum threshold (for example, 0.02), the first extraction function 105a sets the minimum threshold to the binarization threshold T3 and proceeds from step S206 to step S207 without returning from step S204.

[0129] The processing in steps S205 and S206 increases the likelihood of extracting an extraction region 3 that is larger than or equal to the minimum size. Note that the processing in steps S205 and S206 may be omitted.

[0130] As described above, the first extraction function 105a extracts an extraction region 3, which is presumed to be the region of interest 11, from the partial image data 3 contained within the processing range 3 that includes the specified point 12 of the medical image data. The processing range 3 is an example of the third processing range. The partial image data 3 is an example of the third partial image data. The extraction region 3 is an example of the third extraction region. Furthermore, the first extraction function 105a obtains an inference probability map 3, which is obtained by calculating the probability that each pixel of the partial image data 3 is a pixel contained in the region of interest 11, and extracts the extraction region 3 by binarizing the inference probability map 3. The inference probability map 3 is an example of the third inference probability map.

[0131] Furthermore, as described above, if the size of the extraction region 3 is smaller than a predetermined size (minimum size), the first extraction function 105a reduces the binarization threshold T3 used when binarizing the inference probability map 3 based on a predetermined rule, and then extracts the extraction region 3 by binarizing the inference probability map 3 again using the reduced binarization threshold T3.

[0132] Next, the deletion function 105b determines whether the extracted region 3 contains multiple subregions that are not connected to each other (step S207).

[0133] If the extracted region 3 does not contain multiple subregions that are not connected to each other (step S207: No), that is, if the extracted region 3 is a single region, the deletion function 105b proceeds to step S102 shown in Figure 17B.

[0134] On the other hand, if the extracted region 3 contains multiple subregions that are not connected to each other (step S207: Yes), the deletion function 105b deletes the other subregions, leaving only a specific subregion (step S208). In step S208 of the second embodiment, the deletion function 105b performs the same processing as in step S109 of the first embodiment.

[0135] The process in step S208 makes it possible to remove unnecessary subregions that are highly likely to be mis-extracted, such as regions other than the region of interest 11. Furthermore, by removing these unnecessary subregions that are highly likely to be mis-extracted, it is possible to suppress the increase in computation time caused by expanding the processing range based on the mis-extracted regions.

[0136] Next, the first extraction function 105a sets the probability value of all pixels in the inference probability map 3 acquired in step S202, excluding those corresponding to the pixels in the extraction region 3 (new extraction region 3) obtained in step S208, to "0.0" (step S209). Note that in step S209, the first extraction function 105a does not change the probability set for the pixels in the extraction region 3 obtained in step S208, among all pixels in the inference probability map 3 acquired in step S202. As a result, the first extraction function 105a obtains a new inference probability map 3. If an inference probability map 3 is obtained in step S209, the inference probability map 3 obtained in step S209 is used in place of the inference probability map 3 acquired in step S202 in the processing of steps S210 and later. The processing in steps S207 to S209 may be omitted.

[0137] Here, the extracted region 3 may be only a part of the region of interest 11. In other words, the first extraction function 105a may not have extracted the entire region of interest 11. Therefore, the determination function 105c determines whether or not the region of interest 11 extends beyond the processing range 3 (step S210). The determination method in step S210 is, for example, the same as the determination method in step S112.

[0138] If the area of ​​interest 11 does not extend beyond the processing range 3 (step S210: No), the first extraction function 105a sets the extraction area 3 as the final extraction area (step S211) and terminates the area extraction process. Then, in the second embodiment, the display control function 105g controls the display 14 to display the final extraction area, similar to the first embodiment.

[0139] On the other hand, if the area of ​​interest 11 extends beyond the processing range 3 (step S210: Yes), the image processing device 100 proceeds to step S102, as shown in Figure 17B. Then, the image processing device 100 executes the processing of the steps from step S102 onward, similar to the first embodiment.

[0140] The region extraction process according to the second embodiment has now been described. In the region extraction process according to the second embodiment, the first extraction function 105a extracts an extraction region 3 that is presumed to be the region of interest 11 from the partial image data 3 contained within a processing range 3 smaller than the processing range 1 containing the designated point 12 of the medical image data. Then, if the first extraction function 105a determines that the extraction region 3 is part of the region of interest 11, it extracts the extraction region 1. Furthermore, if the display control function 105g determines that the extraction region 3 is the entire region of the region of interest 11, it displays the extraction region 3 on the display 104.

[0141] The image processing apparatus 100 according to the second embodiment has been described above. In the second embodiment, the processing in steps S201 to S211 is performed before step S102. This makes it possible to improve the region extraction accuracy of the region of interest 11 that is smaller than the fixed size 3.

[0142] (First modified example of the second embodiment) Next, an image processing apparatus 100 according to the first modified example of the second embodiment will be described. In the following description, the differences between the second embodiment and the first modified example of the second embodiment will be mainly described, and descriptions of identical or similar components may be omitted. Also, in the following description, components identical or similar to those in the second embodiment may be denoted by the same reference numerals and their descriptions may be omitted.

[0143] In the second embodiment described above, the inference probability map 3 and the inference probability map 1 yield different probabilities at corresponding pixel positions. By utilizing both of these different probabilities (for example, by integrating two different probabilities), a more reliable probability can be obtained. Therefore, the image processing apparatus 100 according to the first modification of the second embodiment described below performs the process described below in order to obtain a more reliable probability.

[0144] Figures 19A and 19B are flowcharts illustrating an example of the flow of region extraction processing performed by the image processing device 100 according to the first modification of the second embodiment. The region extraction processing shown in Figures 19A and 19B is performed when the user inputs an instruction to the processing circuit 105 via the input interface 103, with the medical image data to be processed stored in the memory circuit 102.

[0145] As shown in Figure 19A, in the first modification of the second embodiment, the image processing device 100 performs the processes in steps S201 to S209, similar to the second embodiment. Then, without performing the processes in steps S210 and S211, the image processing device 100 proceeds to step S102, as shown in Figure 19B. Then, the image processing device 100 performs the processes in steps S102 to S110.

[0146] Then, if the extracted region 1 does not contain multiple subregions that are not connected to each other (step S108: No), and if the execution of the process in step S110 is completed, the integration function 105f proceeds to step S301. In step S301, the integration function 105f obtains a new inference probability map 1 by integrating the inference probability map 3 and the inference probability map 1 so that their positions match based on their respective coordinates. For example, in step S301, the integration function 105f obtains a new inference probability map 1 by integrating the inference probability map 3 and the inference probability map 1 in a similar manner to how the inference probability map 1 and the inference probability map 2 are integrated in step S116 to obtain a new inference probability map 1.

[0147] Then, the image processing device 100 proceeds to step S111. The image processing device 100 then executes the processing of the steps from step S111 onward, similar to the second embodiment.

[0148] The image processing apparatus 100 according to the first modification of the second embodiment has been described above. According to the image processing apparatus 100 according to the first modification of the second embodiment, by utilizing both the inference probability map 3 and the inference probability map 1, it is expected that the extraction accuracy of the region of interest 11 smaller than the fixed size 1 will be improved.

[0149] (Second modification of the second embodiment) Next, an image processing apparatus 100 according to a second modified example of the second embodiment will be described. In the following description, the differences between the second embodiment and the second modified example of the second embodiment will be mainly described, and descriptions of identical or similar components may be omitted. Also, in the following description, components identical or similar to those in the second embodiment may be denoted by the same reference numerals and their descriptions may be omitted.

[0150] In the second embodiment described above, a case was explained in which a fixed size 2 smaller than fixed size 1 is used. That is, in the second embodiment, the processing range 3 of fixed size 2 is smaller than the processing range 1 of fixed size 1. On the other hand, in the second modification of the second embodiment, fixed size 2 is larger than fixed size 1. That is, in the second modification of the second embodiment, the processing range 3 of fixed size 2 is larger than the processing range 1 of fixed size 1.

[0151] In the second modification of the second embodiment, the image processing device 100 does not perform the processing in steps S210 and S211. Specifically, instead of performing the processing in steps S210 and S211, the first extraction function 105a performs a determination process to determine whether the extraction region 3 fits within the processing range 1 of a fixed size 1.

[0152] If the extraction region 3 fits within the fixed-size processing range 1, the image processing device 100 uses a processing range 1 smaller than the processing range 3, rather than the processing range 3, to perform the processing steps from step S102 onwards, as shown in Figure 17B. In other words, the first extraction function 105a extracts the extraction region 1 using the processing range 1 if the extraction region 3 fits within the processing range 1.

[0153] On the other hand, if the extraction region 3 does not fit within the fixed-size processing range 1, the image processing device 100 uses the processing range 3 instead of the processing range 1 and performs the same processing as in the steps from step S104 onwards shown in Figure 17B, so that the target of processing becomes the extraction region 3 instead of the extraction region 1.

[0154] The image processing apparatus 100 according to the second modification of the second embodiment has been described above. In the image processing apparatus 100 according to the second modification of the second embodiment, if the extraction region 3 fits within the processing range 1 of a fixed size 1, the processing of the steps from step S102 onwards shown in Figure 17B is performed using a processing range 1 smaller than the processing range 3, rather than the processing range 3. Therefore, the extraction accuracy of the region of interest 11 can be improved.

[0155] In the above description, the term "processor" refers to circuits such as a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), an Application Specific Integrated Circuit (ASIC), or a programmable logic device (e.g., a Simple Programmable Logic Device (SPLD), a Complex Programmable Logic Device (CPLD), and a Field Programmable Gate Array (FPGA)). The processor performs its function by reading a program stored in a memory circuit and executing the read program. Alternatively, instead of storing the program in a memory circuit, the processor may be configured to directly incorporate the program into its circuitry. In this case, the processor performs its function by reading the program incorporated into the circuitry and executing the read program. In this embodiment, each processor is not limited to being configured as a single circuit; multiple independent circuits may be combined to form a single processor, and its function may be achieved in this way.

[0156] (Other embodiments) In addition to the embodiments described above, the device may be implemented in various other forms.

[0157] For example, each component of the illustrated device is a functional concept and does not necessarily have to be physically configured as shown. In other words, the specific forms of distribution and integration of each device are not limited to those shown, and all or part of them can be functionally or physically distributed and integrated in any unit according to various loads and usage conditions. Furthermore, each processing function performed by each device can be implemented, all or any part of it, by a CPU and the program that is analyzed and executed by that CPU, or by hardware using wired logic.

[0158] Furthermore, among the processes described in the embodiments, all or part of the processes described as being performed automatically can be performed manually, or all or part of the processes described as being performed manually can be performed automatically by known methods. In addition, the processing procedures, control procedures, specific names, and information including various data and parameters shown in the above document and drawings can be arbitrarily changed unless otherwise specified.

[0159] Furthermore, the method described in the embodiment can be implemented by executing a pre-prepared program on a computer such as a personal computer or workstation. This program can be distributed via a network such as the Internet. In addition, this control program can be recorded on a computer-readable non-transient recording medium such as a hard disk, flexible disk (FD), CD-ROM, MO, or DVD, and executed by reading it from the recording medium by a computer.

[0160] According to at least one embodiment described above, the region of interest captured in medical image data can be extracted with high accuracy and speed.

[0161] While several embodiments have been described, these embodiments are presented as examples only and are not intended to limit the scope of the invention. These embodiments can be implemented in a variety of other forms, and various omissions, substitutions, modifications, and combinations of embodiments are possible without departing from the spirit of the invention. These embodiments and their variations are included in the scope and spirit of the invention, as well as in the claims and their equivalents. [Explanation of Symbols]

[0162] 100 Image Processing Devices 105a First extraction function 105c Judgment function 105e Second extraction function 105f Integrated Functions

Claims

1. An image processing device for extracting a region of interest that includes a specified point on medical image data, A first extraction unit extracts a first extraction region that is presumed to be the region of interest from a first partial image data that is included in a first processing range including the designated point of the medical image data, If the first extracted region extracted in the first partial image data is a part of the region of interest, the second extraction unit extracts a second extracted region that is presumed to be the region of interest from the medical image data, from a second partial image data that is in contact with the edge of the first processing range or is included in a second processing range that includes at least a part of the edge; Equipped with, Image processing apparatus, wherein the first extraction unit obtains a first inference probability map obtained by calculating the probability that each pixel of the first partial image data is a pixel included in the region of interest, and extracts the first extraction region by binarizing the first inference probability map.

2. An image processing device for extracting a region of interest that includes a specified point on medical image data, A first extraction unit extracts a first extraction region that is presumed to be the region of interest from a first partial image data that is included in a first processing range including the designated point of the medical image data, If the first extracted region extracted in the first partial image data is a part of the region of interest, the second extraction unit extracts a second extracted region that is presumed to be the region of interest from the medical image data, from a second partial image data that is in contact with the edge of the first processing range or is included in a second processing range that includes at least a part of the edge; Equipped with, The first extraction unit extracts a first inference probability map as the first extraction region, which is obtained by calculating the probability that each pixel of the first partial image data is a pixel included in the region of interest.

3. The image processing apparatus according to claim 1, wherein the second extraction unit obtains a second inference probability map obtained by calculating the probability that each pixel of the second partial image data is a pixel included in the region of interest, and extracts the second extraction region by binarizing the second inference probability map.

4. The image processing apparatus according to claim 2, wherein the second extraction unit extracts a second inference probability map as the second extraction region, which is obtained by calculating the probability that each pixel of the second partial image data is a pixel included in the region of interest.

5. The image processing apparatus according to claim 1, further comprising a determination unit that determines whether the first extraction region is part of the region of interest by determining whether at least a portion of the first extraction region is located at the edge of the first processing range.

6. The image processing apparatus according to claim 1, further comprising a determination unit that determines whether the first extraction region is part of the region of interest by determining whether the statistic of the probability of the first inference probability map located at the edge of the first processing range is greater than or equal to a predetermined value.

7. An image processing device for extracting a region of interest that includes a specified point on medical image data, A first extraction unit extracts a first extraction region that is presumed to be the region of interest from a first partial image data that is included in a first processing range including the designated point of the medical image data, If the first extracted region extracted in the first partial image data is a part of the region of interest, the second extraction unit extracts a second extracted region that is presumed to be the region of interest from the medical image data, from a second partial image data that is in contact with the edge of the first processing range or is included in a second processing range that includes at least a part of the edge; A determination unit that determines whether the first extraction region is part of the region of interest, When the determination unit determines that the first extraction region is part of the region of interest, the direction acquisition unit acquires the direction in which the second processing region is set based on the first processing region, Equipped with, The second extraction unit sets the second processing range at a position shifted by a predetermined distance from the first processing range in the direction acquired by the direction acquisition unit, and extracts the second extraction region from the second partial image data included within the set second processing range.

8. The image processing apparatus according to claim 7, wherein the direction acquisition unit acquires a direction in which the first extraction region is determined to be part of the area of ​​interest, based on whether or not the first extraction region is determined to be part of the area of ​​interest in any direction from the center of the first processing region, as the direction in which the second processing region is set with respect to the first processing region.

9. An image processing device for extracting a region of interest that includes a specified point on medical image data, A first extraction unit extracts a first extraction region that is presumed to be the region of interest from a first partial image data that is included in a first processing range including the designated point of the medical image data, If the first extracted region extracted in the first partial image data is a part of the region of interest, the second extraction unit extracts a second extracted region that is presumed to be the region of interest from the medical image data, from a second partial image data that is in contact with the edge of the first processing range or is included in a second processing range that includes at least a part of the edge; Equipped with, The second extraction unit sets the second processing range at positions shifted by a predetermined distance in all directions where the edge of the first processing range exists, and extracts the second extraction region from the second partial image data included within the set second processing range.

10. An image processing device for extracting a region of interest including a specified point on medical image data, A first extraction unit extracts a first extraction region that is presumed to be the region of interest from a first partial image data that is included in a first processing range including the designated point of the medical image data, If the first extracted region extracted in the first partial image data is a part of the region of interest, the second extraction unit extracts a second extracted region that is presumed to be the region of interest from the medical image data, from a second partial image data that is in contact with the edge of the first processing range or is included in a second processing range that includes at least a part of the edge; When the first extraction region includes a plurality of subregions that are not connected to each other, a deletion unit deletes the subregions from the plurality of subregions that do not include the specified point, An image processing device equipped with the following features.

11. An image processing device for extracting a region of interest that includes a specified point on medical image data, A first extraction unit extracts a first extraction region that is presumed to be the region of interest from a first partial image data that is included in a first processing range including the designated point of the medical image data, If the first extracted region extracted in the first partial image data is a part of the region of interest, the second extraction unit extracts a second extracted region that is presumed to be the region of interest from the medical image data, from a second partial image data that is in contact with the edge of the first processing range or is included in a second processing range that includes at least a part of the edge; When the first extraction region includes multiple subregions that are not connected to each other, the deletion unit calculates the centroid of each of the multiple subregions and deletes all subregions except the one whose centroid is closest to the specified point. An image processing device equipped with the following features.

12. An image processing device for extracting a region of interest that includes a specified point on medical image data, A first extraction unit extracts a first extraction region that is presumed to be the region of interest from a first partial image data that is included in a first processing range including the designated point of the medical image data, If the first extracted region extracted in the first partial image data is a part of the region of interest, the second extraction unit extracts a second extracted region that is presumed to be the region of interest from the medical image data, from a second partial image data that is in contact with the edge of the first processing range or is included in a second processing range that includes at least a part of the edge; When the first extraction region includes a plurality of subregions that are not connected to each other, the deletion unit deletes subregions from the plurality of subregions other than the subregion whose center of the circumscribing rectangle is closest to the specified point, An image processing device equipped with the following features.

13. An image processing device for extracting a region of interest including a specified point on medical image data, A first extraction unit extracts a first extraction region that is presumed to be the region of interest from a first partial image data that is included in a first processing range including the designated point of the medical image data, If the first extracted region extracted in the first partial image data is a part of the region of interest, the second extraction unit extracts a second extracted region that is presumed to be the region of interest from the medical image data, from a second partial image data that is in contact with the edge of the first processing range or is included in a second processing range that includes at least a part of the edge; When the first extraction region includes multiple subregions that are not connected to each other, a deletion unit deletes all subregions except the one with the largest size among the multiple subregions, An image processing device equipped with the following features.

14. The image processing apparatus according to claim 1, wherein, if the size of the first extraction region is smaller than a predetermined size, the first extraction unit reduces the binarization threshold used when binarizing the first inference probability map based on a predetermined rule, and extracts the first extraction region by re-binarizing the first inference probability map using the reduced binarization threshold.

15. The image processing apparatus according to claim 3, wherein the second extraction unit sets the binarization threshold used when binarizing the second inference probability map to be larger than the binarization threshold used when binarizing the first inference probability map based on a predetermined rule, and extracts the second extraction region by binarizing the second inference probability map using the larger binarization threshold.

16. An image processing device for extracting a region of interest that includes a specified point on medical image data, A first extraction unit extracts a first extraction region that is presumed to be the region of interest from a first partial image data that is included in a first processing range including the designated point of the medical image data, If the first extracted region extracted in the first partial image data is a part of the region of interest, the second extraction unit extracts a second extracted region that is presumed to be the region of interest from the medical image data, from a second partial image data that is in contact with the edge of the first processing range or is included in a second processing range that includes at least a part of the edge; Equipped with, The first extraction unit extracts a third extraction region, which is presumed to be the region of interest, from a third partial image data contained within a third processing range smaller than the first processing range containing the designated point of the medical image data, and if the third extraction region is a part of the region of interest, it extracts the first extraction region. An image processing apparatus further comprising a display control unit that causes the third extraction region to be displayed on a display unit, assuming that the third extraction region is the entire area of ​​the region of interest.

17. The first extraction unit obtains a first inference probability map obtained by calculating the probability that each pixel of the first partial image data is a pixel included in the region of interest, and a third inference probability map obtained by calculating the probability that each pixel of the third partial image data is a pixel included in the region of interest. The system further includes an integration unit that integrates the first inference probability map and the third inference probability map to obtain a new inference probability map. The image processing apparatus according to claim 16.

18. An image processing device for extracting a region of interest that includes a specified point on medical image data, A first extraction unit extracts a first extraction region that is presumed to be the region of interest from a first partial image data that is included in a first processing range including the designated point of the medical image data, If the first extracted region extracted in the first partial image data is a part of the region of interest, the second extraction unit extracts a second extracted region that is presumed to be the region of interest from the medical image data, from a second partial image data that is in contact with the edge of the first processing range or is included in a second processing range that includes at least a part of the edge; Equipped with, The first extraction unit further, From a third partial image data that is included within a third processing range that is larger than the first processing range that includes the specified point of the medical image data, a third extraction region that is presumed to be the region of interest is extracted. An image processing apparatus that extracts the first extraction region using the first processing range when the third extraction region falls within the first processing range.

19. An image processing device for extracting a region of interest that includes a specified point on medical image data, A first extraction unit extracts a first extraction region that is presumed to be the region of interest from a first partial image data that is included in a first processing range including the designated point of the medical image data, If the first extracted region extracted in the first partial image data is a part of the region of interest, the second extraction unit extracts a second extracted region that is presumed to be the region of interest from the medical image data, from a second partial image data that is in contact with the edge of the first processing range or is included in a second processing range that includes at least a part of the edge; An integration unit that integrates the first extraction region and the second extraction region to obtain a new extraction region, An image processing device equipped with the following features.

20. The system further includes an integration unit that integrates the first inference probability map and the second inference probability map to obtain a new inference probability map, The image processing apparatus according to claim 4, wherein, if there is an overlapping region between the first inference probability map and the second inference probability map, the integration unit obtains a new inference probability map by setting the probability of each pixel in the region corresponding to the overlapping region in the new inference probability map to the larger of the probability of each pixel in the overlapping region of the first inference probability map and the probability of each pixel in the overlapping region of the second inference probability map.

21. The system further includes an integration unit that integrates the first inference probability map and the second inference probability map to obtain a new inference probability map, The image processing apparatus according to claim 4, wherein, if there is an overlapping region between the first inference probability map and the second inference probability map, the integration unit obtains a new inference probability map by taking the probability of each pixel in the region corresponding to the overlapping region in the new inference probability map as the average value of the probability of each pixel in the overlapping region of the first inference probability map and the probability of each pixel in the overlapping region of the second inference probability map.

22. The system further includes an integration unit that integrates the first inference probability map and the second inference probability map to obtain a new inference probability map, The image processing apparatus according to claim 4, wherein, if there is an overlapping region between the first inference probability map and the second inference probability map, the integration unit calculates the probability of each pixel in the region corresponding to the overlapping region in the new inference probability map by adding a first value obtained by multiplying the probability of each pixel in the overlapping region of the first inference probability map by a weight that decreases as it moves further away from the center of the first processing range, and a second value obtained by multiplying the probability of each pixel in the overlapping region of the second inference probability map by a weight that decreases as it moves further away from the center of the second processing range.

23. The image processing apparatus according to claim 1, wherein the second extraction unit sets a second processing range using region information indicating at least one of the regions in which the region of interest exists and regions in which the region of interest does not exist, and extracts a second extraction region that is presumed to be the region of interest from the second partial image data included within the set second processing range.

24. An image processing method for extracting a region of interest that includes a specified point in medical image data, From the first partial image data included within the first processing range containing the specified point of the medical image data, a first extraction region presumed to be the region of interest is extracted. If the first extraction region is a part of the region of interest, then a second extraction region, which is presumed to be the region of interest, is extracted from the medical image data from a second partial image data that is in contact with the edge of the first processing range or is included in a second processing range that includes at least a part of the edge. Includes, Image processing method comprising extracting the first extraction region, obtaining a first inference probability map obtained by calculating the probability that each pixel of the first partial image data is a pixel included in the region of interest, and extracting the first extraction region by binarizing the first inference probability map.

25. An image processing method for extracting a region of interest that includes a specified point on medical image data, From the first partial image data included within the first processing range containing the specified point of the medical image data, a first extraction region presumed to be the region of interest is extracted. If the first extraction region is a part of the region of interest, then a second extraction region, which is presumed to be the region of interest, is extracted from the medical image data from a second partial image data that is in contact with the edge of the first processing range or is included in a second processing range that includes at least a part of the edge. Includes, Image processing method comprising extracting the first extraction region, which includes extracting a first inference probability map obtained by calculating the probability that each pixel of the first partial image data is a pixel included in the region of interest, as the first extraction region.

26. An image processing method for extracting a region of interest that includes a specified point on medical image data, From the first partial image data included within the first processing range containing the specified point of the medical image data, a first extraction region presumed to be the region of interest is extracted. If the first extraction region is a part of the region of interest, a second extraction region presumed to be the region of interest is extracted from the medical image data from a second partial image data that is adjacent to the edge of the first processing range or is included in a second processing range that includes at least a part of the edge. Determine whether the first extraction region is part of the region of interest, When it is determined that the first extraction region is part of the region of interest, the direction in which the second processing range is set relative to the first processing range is obtained. Includes, Image processing method comprising extracting the second extraction region, setting the second processing region at a position shifted by a predetermined distance from the first processing region in the acquired direction, and extracting the second extraction region from the second partial image data included within the set second processing region.

27. ​​An image processing method for extracting a region of interest that includes a specified point on medical image data, From the first partial image data included within the first processing range containing the specified point of the medical image data, a first extraction region presumed to be the region of interest is extracted. If the first extraction region is a part of the region of interest, then a second extraction region, which is presumed to be the region of interest, is extracted from the medical image data from a second partial image data that is in contact with the edge of the first processing range or is included in a second processing range that includes at least a part of the edge. Includes, Image processing method for extracting the second extraction region, which includes setting the second processing range at positions shifted by a predetermined distance in all directions where the edge of the first processing range exists, and extracting the second extraction region from the second partial image data included within the set second processing range.

28. An image processing method for extracting a region of interest that includes a specified point on medical image data, From the first partial image data included within the first processing range containing the specified point of the medical image data, a first extraction region presumed to be the region of interest is extracted. If the first extraction region is a part of the region of interest, a second extraction region presumed to be the region of interest is extracted from the medical image data from a second partial image data that is adjacent to the edge of the first processing range or is included in a second processing range that includes at least a part of the edge. If the first extraction region includes multiple subregions that are not connected to each other, delete the subregions from among the multiple subregions that do not include the specified point. Image processing methods, including those mentioned above.

29. An image processing method for extracting a region of interest that includes a specified point on medical image data, From the first partial image data included within the first processing range containing the specified point of the medical image data, a first extraction region presumed to be the region of interest is extracted. If the first extraction region is a part of the region of interest, a second extraction region presumed to be the region of interest is extracted from the medical image data from a second partial image data that is adjacent to the edge of the first processing range or is included in a second processing range that includes at least a part of the edge. If the first extraction region includes multiple subregions that are not connected to each other, calculate the centroid of each of the multiple subregions and delete the subregions other than the one whose centroid is closest to the specified point. Image processing methods, including those mentioned above.

30. An image processing method for extracting a region of interest that includes a specified point on medical image data, From the first partial image data included within the first processing range containing the specified point of the medical image data, a first extraction region presumed to be the region of interest is extracted. If the first extraction region is a part of the region of interest, a second extraction region presumed to be the region of interest is extracted from the medical image data from a second partial image data that is adjacent to the edge of the first processing range or is included in a second processing range that includes at least a part of the edge. If the first extraction region includes multiple subregions that are not connected to each other, delete the subregions from among the multiple subregions, except for the subregion whose center of the circumscribing rectangle is closest to the specified point. Image processing methods, including those mentioned above.

31. An image processing method for extracting a region of interest that includes a specified point on medical image data, From the first partial image data included within the first processing range containing the specified point of the medical image data, a first extraction region presumed to be the region of interest is extracted. If the first extraction region is a part of the region of interest, a second extraction region presumed to be the region of interest is extracted from the medical image data from a second partial image data that is adjacent to the edge of the first processing range or is included in a second processing range that includes at least a part of the edge. If the first extraction region includes multiple subregions that are not connected to each other, delete all subregions except the one with the largest size. Image processing methods, including those mentioned above.

32. An image processing method for extracting a region of interest that includes a specified point on medical image data, From the first partial image data included within the first processing range containing the specified point of the medical image data, a first extraction region presumed to be the region of interest is extracted. If the first extraction region is a part of the region of interest, then a second extraction region, which is presumed to be the region of interest, is extracted from the medical image data from a second partial image data that is in contact with the edge of the first processing range or is included in a second processing range that includes at least a part of the edge. Includes, Extracting the first extraction region includes extracting a third extraction region, which is presumed to be the region of interest, from a third partial image data contained within a third processing range smaller than the first processing range including the designated point of the medical image data, and then extracting the first extraction region if the third extraction region is a part of the region of interest. An image processing method further comprising displaying the third extracted region on a display unit, assuming that the third extracted region is the entire region of interest.

33. An image processing method for extracting a region of interest that includes a specified point on medical image data, From the first partial image data included within the first processing range containing the specified point of the medical image data, a first extraction region presumed to be the region of interest is extracted. If the first extraction region is a part of the region of interest, then a second extraction region, which is presumed to be the region of interest, is extracted from the medical image data from a second partial image data that is in contact with the edge of the first processing range or is included in a second processing range that includes at least a part of the edge. Includes, Extracting the first extraction region described above further involves, From a third partial image data that is included within a third processing range that is larger than the first processing range that includes the specified point of the medical image data, a third extraction region that is presumed to be the region of interest is extracted. An image processing method that, when the third extraction region falls within the first processing range, extracts the first extraction region using the first processing range.

34. An image processing method for extracting a region of interest that includes a specified point on medical image data, From the first partial image data included within the first processing range containing the specified point of the medical image data, a first extraction region presumed to be the region of interest is extracted. If the first extraction region is a part of the region of interest, a second extraction region presumed to be the region of interest is extracted from the medical image data from a second partial image data that is adjacent to the edge of the first processing range or is included in a second processing range that includes at least a part of the edge. To obtain a new extraction region by integrating the first extraction region and the second extraction region, Image processing methods, including those mentioned above.

35. A program for causing a computer to execute the image processing method described in any one of claims 24 to 34.

Images