Image processing device, image processing method, and program

Abstract

To extract abnormal regions from medical image data more reliably and with a higher detection rate. [Solution] The image processing apparatus according to the embodiment comprises an image acquisition means, a focus area setting means, a processing target area setting means, and an identification means. The image acquisition means acquires an image. The focus area setting means sets a focus area within the image. The processing target area setting means sets a plurality of processing target areas for the focus area. The identification means performs an identification process on the plurality of processing target areas. The identification means terminates the identification process when a predetermined termination condition is met.

Description

Technical Field

[0001] The embodiments disclosed in this specification and the drawings relate to an image processing apparatus, an image processing method, and a program.

Background Art

[0002] In the medical field, diagnoses are made using three-dimensional images acquired by various image imaging devices (modalities) such as computed tomography devices (hereinafter referred to as CT (Computed Tomography) devices). In this diagnosis, regions representing abnormalities shown in medical image data are detected. However, with the advancement of imaging devices in recent years, the number of images has increased, so an increase in the burden on doctors has become an issue. Therefore, various techniques have been proposed for a long time regarding automatic or semi-automatic abnormal region detection / extraction techniques from images in order to reduce the labor of doctors.

[0003] [[ID=十六]]

[0004] However, in the above techniques, there is a risk that the detection rate is not stable regarding the setting of the processing target region in the inference of the second stage of the abnormal region extraction process, and further, there is a problem that the detection rate is not increased to the maximum.

Prior Art Documents

Non-Patent Documents

[0005] ​[Non-Patent Document 1] Noguchi et. Al. “Deep learning-based algorithm improved radiologists' performance in bone metastases detection on CT” Eur Radiol.2022 Nov;32(11):7976-7987. [Overview of the project] [Problems that the invention aims to solve]

[0006] One of the problems that the embodiments disclosed herein and in the drawings aim to solve is to extract abnormal regions contained in medical image data more stably and with a higher detection rate. However, the problems that the embodiments disclosed herein and in the drawings aim to solve are not limited to the above problem. Problems corresponding to the effects of each configuration shown in the embodiments described later can also be positioned as other problems. [Means for solving the problem]

[0007] The image processing apparatus according to this embodiment includes an image acquisition means, a focus area setting means, a processing target area setting means, and an identification means. The image acquisition means acquires an image. The focus area setting means sets a focus area within the image. The processing target area setting means sets a plurality of processing target areas for the focus area. The identification means performs an identification process on the plurality of processing target areas. The identification means terminates the identification process when a predetermined termination condition is met. [Brief explanation of the drawing]

[0008] [Figure 1] Figure 1 is a block diagram showing an example configuration of an image processing system including an image processing apparatus according to the first embodiment. [Figure 2] Figure 2 is a flowchart showing an example of the processing procedure of the image processing apparatus according to the first embodiment. [Figure 3] Figure 3 shows an example of a cross-sectional view of a CT image. [Figure 4] Figure 4 shows an example of the results of coarse extraction of bone metastasis areas from CT images. [Figure 5] Figure 5 shows an example of setting the target region for identification processing based on the rough extraction results of the bone metastasis region. [Figure 6] Figure 6 is a flowchart showing an example of the processing procedure of the image processing apparatus according to the second embodiment. [Modes for carrying out the invention]

[0009] The following describes in detail embodiments of the image processing apparatus, image processing method, and program with reference to the drawings.

[0010] <First Embodiment> The image processing device according to the first embodiment is a device for extracting regions from an image. This device has the function of extracting a region of interest for the user from an image. Here, the region of interest can be rephrased as information representing the contour shape of the region of interest. Furthermore, in the process of performing the extraction, this device is characterized by performing a two-stage inference process using an inference model constructed from training data, and performing a second stage of inference based on the result of the first stage of inference, thereby obtaining a more stable extraction result. According to this embodiment, highly accurate region extraction results can be provided to the user.

[0011] This embodiment describes an example of extracting bone metastasis regions from 3D CT images. However, the effects of this embodiment can also be obtained by performing similar processing when extracting regions of images obtained by other modalities or other anatomical structures.

[0012] The configuration and processing of the image processing apparatus of this embodiment will be described below with reference to Figure 1. Figure 1 is a block diagram showing an example configuration of an image processing system (also called a medical image processing system) including an image processing apparatus according to the first embodiment. The image processing system comprises, functionally, an image processing apparatus 10, a network 21, and a database 22. The image processing apparatus 10 is communicably connected to the database 22 via the network 21. The network 21 includes, for example, a LAN (Local Area Network) or a WAN (Wide Area Network).

[0013] Database 22 stores and manages images of subjects and information associated with those images. The information managed in database 22 also includes information for constructing a trained model calculated from a certain set of training data. The image processing device 10 can acquire information such as images stored in database 22 via network 21.

[0014] The image processing device 10 includes a communication interface 31 (communication unit), a read-only memory 32, a random access memory 33, a storage unit 34, an operation unit 35, a display unit 36, and a control unit 37.

[0015] The communication IF31 (communication unit) is composed of a LAN card or the like and enables communication between an external device (e.g., a database 22) and the image processing device 10. The ROM32 is composed of non-volatile memory or the like and stores various programs. The RAM33 is composed of volatile memory or the like and temporarily stores various information as data. The storage unit 34 is composed of an HDD (Hard Disk Drive) or the like and stores various information as data. The operation unit 35 is composed of a keyboard, mouse, touch panel, etc. and inputs instructions from the user (e.g., a doctor or a medical technologist) to the various devices.

[0016] The display unit 36 is composed of a display or the like and displays various information to the user. The control unit 37 is composed of a CPU (Central Processing Unit) or the like and comprehensively controls the processing in the image processing apparatus 10. As its functional configuration, the control unit 37 includes an image acquisition unit 51, a region of interest setting unit 52, a processing target region setting unit 53, an identification unit 54, and a display processing unit 55.

[0017] The image acquisition unit 51 acquires an image to be processed from the database 22. That is, the image acquisition unit 51 corresponds to an example of image acquisition means. This image is an image of a subject acquired by various modalities. In this embodiment, an example where the image is a three-dimensional CT image will be described, but other types of images may also be used. This embodiment is applicable to images of two dimensions or more (a plurality of two-dimensional images, two-dimensional moving images, three-dimensional still images, a plurality of three-dimensional images, or three-dimensional moving images, etc.). Also, it is applicable regardless of the type of modality.

[0018] The region of interest setting unit 52 sets the contour information or coordinate information of the region of interest existing in the image acquired by the image acquisition unit 51. That is, the region of interest setting unit 52 corresponds to an example of region of interest setting means.

[0019] The processing target region setting unit 53 sets a processing target region that is the target of the subsequent identification processing from the image acquired by the image acquisition unit 51 and the information on the region of interest set by the region of interest setting unit 52. That is, the processing target region setting unit 53 corresponds to an example of processing target region setting means.

[0020] The identification unit 54 performs an identification process from the image acquired by the image acquisition unit 51 and the information on the processing target region set by the processing target region setting unit 53. That is, the identification unit 54 corresponds to an example of identification means.

[0021] The display processing unit 55 causes the display unit 36 to display an image, the contour information of the target region, and / or the identification result in a display form that allows easy visual recognition based on the result calculated by the identification unit 54 within the image display area of the display unit 36. That is, the display processing unit 55 corresponds to an example of display means.

[0022] Each component of the above-described image processing apparatus 10 functions, for example, according to a computer program. For example, the control unit 37 (CPU) reads and executes a computer program stored in the ROM 32 or the storage unit 34 or the like using the RAM 33 as a work area, thereby realizing the functions of each component. Note that some or all of the functions of the components of the image processing apparatus 10 may be realized using dedicated circuits. Also, some of the functions of the components of the control unit 37 may be realized using a cloud computer.

[0023] For example, an arithmetic device located at a location different from the image processing apparatus 10 is communicably connected to the image processing apparatus 10 via the network 21, and by performing data transmission and reception between the image processing apparatus 10 and the arithmetic device, the functions of the components of the image processing apparatus 10 or the control unit 37 may be realized.

[0024] Next, an example of the processing of the image processing apparatus 10 in FIG. 1 will be described using FIG. 2.

[0025] FIG. 2 is a flowchart showing an example of the processing procedure of the image processing apparatus 10 according to the first embodiment. In this embodiment, an example in the case where the bone metastasis region is the target region will be described. However, this embodiment is also applicable when other sites, abnormal shadows, or regions combined with a plurality of regions are used as the target region.

[0026] (Step S101: Acquisition and display of an image) In step S101, when the user instructs the user to acquire an image via the operation unit 35, the image acquisition unit 51 acquires the image specified by the user from the database 22 and stores it in the RAM 33. At this time, the display processing unit 55 may also display the image in the image display area of ​​the display unit 36. An example of an image is shown in Figure 3. Figure 3 shows an example where the image is a CT image. In the following explanation, the number of pixels in the x direction that make up the image is assumed to be Nx, and the number of pixels in the y direction is assumed to be Ny. The number of pixels in the z direction is assumed to be Nz (the size in the z direction is not shown in Figure 3). That is, the total number of pixels that make up the image is Nx × Ny × Nz.

[0027] (Step S102: Setting the area of ​​interest (Setting the initial area / Performing rough sampling)) In step S102, the area of ​​interest setting unit 52 sets an area of ​​interest within the image acquired by the image acquisition unit 51. The processing in step S102 corresponds to preprocessing for the subsequent identification process, and its purpose is to extract candidate areas from the image that will be input to the identification process and set them as areas of interest. This is the first stage of the extraction process in the two-stage abnormal area extraction process described in Non-Patent Literature 1. Here, the area of ​​interest in this embodiment refers to the rough extraction result of the bone metastasis area shown in the CT image. Depending on the image, this area of ​​interest may be set for multiple locations within the image. An example of bone metastasis is when abnormal shadows are observed in different parts / locations, such as bone metastasis to the ribs and bone metastasis to the pelvis.

[0028] One example of how to define this area of ​​interest is to use an automated or semi-automated segmentation method using a machine learning model, including a deep learning model. Alternatively, it may be extracted using known region extraction methods such as thresholding or graph cut segmentation.

[0029] This section describes how to set areas of interest based on automated extraction using a machine learning model. In area extraction using a machine learning model, the first step is to train a model to extract bone metastasis areas (areas of interest) based on training data. In supervised machine learning, the training data consists of CT images and information representing the bone metastasis areas visible in those images (i.e., ground truth areas). A model is trained that automatically extracts these ground truth areas using the machine learning framework. Then, by applying the trained model to unknown (unused) images, bone metastasis areas visible in the unknown images can be automatically extracted. As an example, as described in Non-Patent Document 1, known models such as U-net, a type of deep learning, can be used for area extraction. However, this area extraction process may also be performed using other deep learning models or machine learning models other than deep learning (for example, support vector machines or boosting techniques).

[0030] In step S102, if the focus region setting unit 52 uses a machine learning model, one possible method is to read a pre-stored trained model from the database 22 and utilize it during processing. That is, after acquiring an image in step S101, the focus region setting unit 52 reads a trained model from the database 22, performs inference on the acquired image using the model, and then sets the inference result as the focus region. More specifically, the region extracted as a bone metastasis region in the inference result, or the region with a high likelihood / probability of a bone metastasis region extracted by thresholding, is set as the focus region. In this embodiment, the information representing the focus region is assumed to be information (image) in which pixel values ​​of 1 or greater are stored in the focus region, and pixel values ​​of 0 are stored in other regions. Note that this information representing the focus region can be an image in which pixels at the position corresponding to the focus region and other pixels can be distinguished. Alternatively, it may be the coordinate values ​​of the focus region in 3D space or a list composed of multiple coordinate values.

[0031] Here, an example of an image is shown in Figure 4. Figure 4 shows an example in which, using the CT image in Figure 3 as input, inference of the area of ​​interest was performed, and the shaded areas R1 and R2 in the figure were extracted as candidate areas for bone metastasis. In this embodiment, the R1 and R2 areas are set as independent areas of interest.

[0032] This embodiment is configured to extract and detect bone metastasis regions in a two-stage inference process (similar to Non-Patent Literature 1, the first stage is rough extraction using U-net, and the second stage is identification using ResNet). Therefore, when adjusting the parameters during training or inference of the deep learning model described above, it is preferable to adjust them so that as many true positives (bone metastasis regions) as possible are obtained while tolerating a certain degree of false positives. This is because the identification process in the second step is performed based on the inference results of the first step, so if a true positive is overlooked in the first step, it becomes difficult to detect a new region in the second step. In other words, the first step is designed to minimize false positives as much as possible while minimizing the oversight of true positives, and the second stage of inference processing is intended to reduce false positives through identification. By dividing the inference and identification process into two stages, the second stage of inference can perform learning and inference processing specialized for the areas of interest set by the first stage of inference, thus improving the accuracy of the inference.

[0033] The area of ​​interest in step S102 may be set by the user via the operation unit 35, or it may be set by reading information representing the area of ​​interest that has been previously stored in the database 22.

[0034] (Step S103: Setting the area to be identified) In step S103, the processing target area setting unit 53 sets the processing target area (i.e., input data or input image) that will be processed in the second stage inference process, step S104, based on the area of ​​interest set in the area of ​​interest setting unit 52. In this step, it is desirable to set the processing target area in a way that maximizes the recognition accuracy while considering the inference process in step S104. This setting of the processing target area is done for each area of ​​interest, and one or more processing target areas are set for each area of ​​interest. Specifically, for example, the circumscribing rectangular prism of the area of ​​interest may be set as the processing target area, or a cube or rectangular prism of a predetermined size centered on the centroid of the area of ​​interest may be set as the processing target area (in the case of 3D; a square or rectangle in the case of 2D). Alternatively, a cube or rectangular prism of a predetermined size centered on a part of the area of ​​interest may be set as the processing target area. In this case, two or more cubes or rectangular prisms of different sizes may be set as the processing target area, or two or more cubes or rectangular prisms of the same size may be set. Furthermore, the shape of this processing area is not limited to a rectangular prism; it can be any shape.

[0035] Below, as an example of how to set the processing target area, we describe an example of how to set the processing target area of ​​a cube according to a default algorithm using the information representing the area of ​​interest mentioned above.

[0036] One possible method for defining the processing area is to define it based on the distance from the contour of the area of ​​interest. Here, the distance from the contour is set to a distance value of 0 at the location corresponding to the contour of the area of ​​interest, and the value increases as you move further inward from the area. The distance is measured using Euclidean distance, but this can also be measured using other known methods such as Manhattan distance. The following shows an example of the procedure for defining the processing area. If there are multiple areas of interest, the following processing should be performed independently for each of them.

[0037] First, the distance to the contour of the region is calculated for all pixels representing the region of interest. Next, one possible method is to define the processing area as a cube-shaped region centered on the pixel with the greatest distance to the contour among the pixels within the region of interest. Alternatively, when setting multiple processing areas for a single region of interest, one can repeatedly set the center position of the processing areas based on predetermined conditions and evaluation metrics. A concrete example of these predetermined conditions would be to satisfy the following three items.

[0038] 1. A pixel that lies within a certain predetermined range. 2. A pixel that is located more than a predetermined distance from the center of the already defined processing area. 3. Among the set of pixels that satisfy conditions 1 and 2 above, this is the pixel with the farthest distance value to the contour.

[0039] The default range in condition 1 above may be defined as the area of ​​interest, or as the area within a predetermined distance from the point furthest from the contour of the area of ​​interest. Alternatively, it may be defined as the area within a predetermined distance outside the contour, or as the entire range of the input image. The number of processing target areas can be set by adding areas until no more areas satisfy the conditions described above. Alternatively, it may be set up to a fixed number, or an upper limit may be set on the number.

[0040] In the above example, the processing area was set based on the distance from the contour of the area of ​​interest, but it may also be set based on other information about the area of ​​interest. For example, the processing area may be set based on the distance from the centroid of the area of ​​interest or the distance from the center of the circumscribed rectangular prism of the area of ​​interest. Alternatively, if there is a relative magnitude of the inference values ​​for each location in the image with respect to the extraction result of the area of ​​interest extracted in step S102, the processing area may be set based on those inference values. In other words, if U-net is used for the first stage of inference, the output value of U-net itself may be used as an evaluation index for setting the processing area.

[0041] Furthermore, the distance from the above contour may be a signed distance value (or a value with the sign reversed) where the distance value towards the inside of the region is negative and the distance value towards the outside is positive.

[0042] Here, we will discuss the effect of setting the processing target region, which is the input for step S104, based on the region of interest. A characteristic of machine learning models is that the inference accuracy tends to be high when the region to be extracted and identified is located near the center of the input data during inference, and conversely, it tends to be low when it is located at the edge of the input data. Therefore, in a two-stage inference process, when using such a model in the second stage of inference, the inference accuracy of the second stage can be improved by utilizing the processing result of the first stage (coarse extraction of the region of interest).

[0043] Figure 5 shows an example of setting the processing target area. Figure 5 is an enlarged portion of the image in Figure 4, and shows an example in which the processing target areas VOI1 and VOI2 (VOI: Volume Of Interest) are set for the region of interest R1. P1 in the figure represents the position corresponding to the center of VOI1. The above example of setting the processing target area corresponds to setting VOI1 (or P1) to the location furthest from the contour of the region of interest R1, based on the contour information of the region of interest R1. In addition, multiple processing target areas may be set, as shown in VOI2 in Figure 5.

[0044] Here, we discuss the effect of setting multiple processing areas for a single area of ​​interest. Inference using machine learning models can change significantly in output even with slight changes in the input image. In other words, even if processing areas of the same size are set in close proximity, the identification results may change. Therefore, depending on how the processing areas are set, there is a risk that bone metastases that should be identified as positive may not be identified (overlooked). For this reason, one approach is to set multiple processing areas for a single area of ​​interest obtained in the first stage of inference, and integrate their results to improve the stability and accuracy of the identification process. Specifically, to increase the detection rate of bone metastases, one can consider methods such as adopting the maximum, mean, or median of multiple identification results, or taking a majority vote of multiple identification results.

[0045] (Step S104: Identification process) In step S104, the identification unit 54 performs identification processing on the processing target regions set in the processing target region setting unit 53. One example of this identification processing is the use of an automatic or semi-automatic segmentation method using a machine learning model, including a deep learning model. Alternatively, identification can be performed using a known identification processing method such as a statistical model. Similar to step S102, when performing identification processing using a machine learning model, the trained model can be stored in the database 22 in advance and used for inference processing (identification processing), and the same applies when using a statistical model. Below, an example is described in which identification processing is performed on all processing target regions for a certain area of ​​interest set in step S103, but it is also acceptable to perform identification processing on only some of the processing target regions. As an example of identification processing, as described in Non-Patent Literature 1, a known model such as ResNet (Residual Neural Network) can be used for identification processing.

[0046] In step S104, if the identification unit 54 uses a machine learning model, it reads a pre-stored trained model from the database 22 and performs inference processing on all regions set as processing regions. Furthermore, if multiple regions are set as processing regions, the inference results corresponding to them can be integrated to perform identification processing on a single region of interest. Here, the inference results are assumed to be scalar values ​​representing the likelihood or probability that the processing region is positive for bone metastasis. For these integration methods, known statistical values ​​such as the maximum, median, mean, and minimum values ​​of the inference results can be used. In addition, this integration process may use the results of all processing regions set for a single region of interest, or it may use only the upper or lower values ​​of the data, such as predetermined percentiles. A specific example is the process shown in Figure 5, where identification processing is performed on VOI1 and VOI2 set for R1, the maximum value of their identification processing results is calculated, and it is determined whether R1 is a bone metastasis or not.

[0047] In this embodiment, we have shown an example where multiple processing target areas are first determined in step S103, and then identification processing is performed in step S104. However, the processing flow is not limited to this. For example, the same effect can be obtained by repeatedly setting processing target areas and performing the identification processing in step S104 each time a processing target area is set in step S103, and then setting a different inference target area from the one set in step S103.

[0048] (Step S105: Display the image based on the identification result) In step S105, the display processing unit 55 displays the input image and the contour information of the area of ​​interest identified as positive by the identification unit 54 within the image display area. At this time, the estimated contour information and the image may be displayed superimposed. By performing superimposed display, both the area identified as positive and the image can be observed, so the user can easily visually confirm whether the identification processing result is correct.

[0049] Furthermore, if the purpose is to analyze or measure a region of interest, the processing in step S105 is not necessarily required, and the results identified in step S104 may be saved. Alternatively, the results displayed in step S105 may be shown, and the user may be allowed to select which of the displayed identification results to save and which to not save.

[0050] Furthermore, although the above explanation used the example of a 3D image, the same processing can be performed on 2D images as well.

[0051] According to this embodiment, a two-stage inference process is performed using an inference model constructed from training data, and the second stage of inference is performed based on the results of the first stage of inference, which has the effect of providing users with more stable extraction results.

[0052] Although the first embodiment has been described above, the embodiment is not limited thereto and can be modified or altered as appropriate.

[0053] <Second Embodiment> In the first embodiment, an example was shown in which a region of interest is set in step S102, a region to be identified based on the region of interest is set in step S103, and identification processing is performed in step S104. This method improves the stability and accuracy of processing by setting multiple processing regions related to a certain region of interest, but increasing the number of regions to be identified has the disadvantage of increasing computational cost. In this embodiment, a processing method is shown that reduces computational cost while retaining the advantages of improving processing stability and the detection rate of abnormal shadows by setting multiple processing regions. In the following example, a bone metastasis region is described as the region of interest, but this embodiment can also be applied to other parts of the body and abnormal shadows.

[0054] Figure 6 is a flowchart showing an example of the processing procedure of the image processing apparatus 10 according to the second embodiment. In the steps shown in this flowchart, steps S201 to S203 perform the same processing as steps S101 to S103 in the first embodiment shown in Figure 2. Also, step S206 performs the same processing as step S105 in the first embodiment. In other words, the only processes that differ from the first embodiment are steps S204 and S205. The differences from the first embodiment will be explained below.

[0055] (Step S204: Setting the termination conditions and processing order for the identification process) In step S204, the identification unit 54 sets the termination conditions for the identification process and the processing order for the processing target area set by the processing target area setting unit 53.

[0056] Below, we will first show an example of setting the termination conditions for the identification process, and then explain an example of setting the processing order.

[0057] The termination conditions for the identification process can be either a condition that terminates the process when it is determined to be positive (positive determination condition), a condition that terminates the process when it is determined to be negative (negative determination condition), or both. A concrete example of a termination condition is to set a condition based on a predetermined threshold. For example, as a positive determination condition, a condition can be set to terminate the identification process when the result of the identification process is greater than or equal to a predetermined threshold. Furthermore, the number of times this condition is met can also be set. That is, the process can be terminated if an identification result that satisfies the condition of being greater than or equal to the predetermined threshold is obtained a predetermined number of times or more. For example, the termination condition for the process could be "when an identification result that satisfies the predetermined condition is obtained two or more times." On the other hand, as a negative determination condition, conversely to the above, a condition can be set to terminate the identification process when the result of the identification process is less than or equal to a predetermined threshold. Furthermore, upper and lower limits can be set for the predetermined threshold. That is, the process can be terminated when the result of the identification process is greater than or equal to a predetermined threshold, or when it is less than or equal to a predetermined threshold. In this way, by setting a termination condition for processing, when an identification process is performed and a processing result that satisfies the condition is obtained, the number of calculations can be reduced, thereby reducing computational costs.

[0058] In this example, setting a termination condition in step S204 is described, but the image processing device 10 may start processing after the termination condition has been set in advance. Alternatively, the termination condition may be changed depending on the area of ​​interest set in step S202. Specifically, a termination condition may be set if the volume or area of ​​the area of ​​interest set in step S202 is large, and not set otherwise. In addition to this, the setting of a termination condition may be determined based on the number of areas to be identified, set in step S203. For example, if the number of areas to be identified is less than a predetermined number, a termination condition may not be set. Furthermore, the image processing device 10 may be automatically set to set a termination condition if there are images waiting to be processed, and not set a termination condition if there are no images waiting to be processed. The user may also be allowed to select whether or not to set a termination condition.

[0059] Next, we will explain how to set the processing order. In the identification process in step S205, the computational cost can be further reduced by combining the setting of the termination conditions described above with determining the order in which the processes are performed. For example, when determining whether bone metastases are positive, if the order is set to prioritize processing areas that are likely to be identified as positive, the effect of reducing computational cost will be greater. In other words, if the termination condition for the identification process is set to be determined to be positive (for example, the inference result being above a certain threshold), and the identification process is performed on multiple target areas, the faster a positive determination can be made, the more computational cost can be reduced.

[0060] As for the method of setting the processing order described above, one method is to determine the processing order of the processing target regions based on a predetermined evaluation function, as used by the processing target region setting unit 53 in step S203. More specifically, for example, it is conceivable to use the distance from the contour of the region of interest at the center of the processing target region as an evaluation value, and to perform the identification process in order from the processing target regions with larger distance values. The reason for this is that if the machine learning model tends to show higher inference accuracy when the region to be identified is located near the center of the input data, then prioritizing the processing target regions set near the center can be expected to reduce computational costs.

[0061] In this embodiment, an evaluation function based on the distance from the contour of the region of interest is described. However, as in step S203, other information related to the region of interest may be used as evaluation values. For example, the distance from the center of the circumscribed rectangular parallelepiped, the distance from the centroid of the region of interest, or the extraction result of the region of interest extracted in step S202 (inferred value by U-net) may be used as evaluation values. Alternatively, an evaluation value combining multiple of these evaluation values ​​may be used.

[0062] Alternatively, one could prioritize processing points that are far apart, such as the center and endpoints, and then gradually process points that are closer together.

[0063] Note that setting this processing order is not strictly necessary; the process can be performed in the order of image scanning (raster scan order) or in a random order. The benefit of reducing computational costs can be maintained if the process can be terminated when the default conditions are met.

[0064] (Step S205: Identification process) In step S205, the identification unit 54 performs identification processing on the processing target area set by the processing target area setting unit 53, similar to step S104 in the first embodiment. However, it differs from the first embodiment in that it terminates the processing based on the conditions set in step S204 during the process of performing identification processing. That is, the identification unit 54 in this embodiment is additionally equipped with a function to terminate the identification processing if the inference result of the set processing target area satisfies a predetermined condition. Furthermore, if the order of identification processing is determined in step S204, the identification processing is performed according to that order. If the order is not determined, the identification processing may be performed in any order.

[0065] Here, we show an example of the bone metastasis region detection process described in the first embodiment. In the first embodiment, in order to increase the detection rate of bone metastasis regions, we showed an example in which multiple identification target regions are set in a certain region of interest and the maximum value among them is adopted. In this case, we consider a process in which, in the positive / negative determination of bone metastasis regions, if this maximum value is greater than or equal to a predetermined threshold, it is determined to be positive for bone metastasis. The result of the positive / negative determination for the region of interest by this process will be exactly the same whether or not the identification process is terminated, if the same determination condition is also set in the termination condition of the identification process in step S204. Furthermore, since it is possible to reduce the number of calculations for the identification target regions, it is possible to reduce computational costs while maintaining the effect of increasing the detection rate of the bone metastasis region detection process of the first embodiment.

[0066] Furthermore, regarding the example of adopting the maximum value of multiple target regions for identification processing, we will also discuss the effect of setting a lower limit as a termination condition for the identification processing. This effect is expected to reduce computational cost when the region of interest is negative for bone metastasis. When a target region for processing is set in a negative location, the inference result of the classifier that determines whether bone metastasis is positive is trained to produce a lower inference value compared to a positive result. In other words, if a sufficiently low inference value is obtained, it is suggested that the region of interest is negative, and from this result, the region of interest can be determined as negative and processing can be terminated, thereby reducing the number of calculations for the target region for identification processing. However, when a lower limit is set as a termination condition, the result of the positive / negative determination of the region of interest may change depending on whether the condition is set or not. This is because there is no guarantee that the inference result of the first processed target region will be greater than the inference result of a later processed target region. However, as described in step S204, this risk can be reduced by appropriately setting the identification processing order of the target regions. In other words, by performing inference on the target region in the order in which the inference result values ​​are expected to be higher, the risk of incorrectly identifying a region of interest that should be positive as negative and terminating the process is minimized, while also enabling faster processing.

[0067] Furthermore, the upper and lower limits of the termination conditions described above may be set as both conditions. In that case, the effect of reducing the computational cost of both can be obtained, and further speed improvements can be expected.

[0068] In the above example, the problem setting was described as determining whether a particular region of interest is positive or negative for bone metastasis, and the case of terminating the determination process for that region was explained as an example, but the embodiments are not limited to this. For example, if only one identification result is needed for the entire input image, the termination condition may be set across multiple regions of interest. For example, in a problem setting of determining whether the entire input image is positive or negative for bone metastasis (determination on a case-by-case basis), the processing of the entire image (processing for all regions of interest) may be terminated when any of the processing target regions is determined to be positive. Furthermore, regarding the determination of the processing order in step S204 in this case, if there are multiple regions of interest, the processing order can be determined for multiple regions of interest. For example, as the processing order for multiple regions of interest, the regions of interest with a higher likelihood / probability of positivity may be prioritized based on the processing result of step S202. Alternatively, regions of interest with a larger size may be prioritized. Then, the processing order may be determined for each region of interest as described above. Alternatively, the processing target areas for multiple areas of interest may be combined, and then the order of the identification process may be determined.

[0069] In the above embodiment, an example is described in which both the termination condition for the identification process and the processing order are set in step S204, but it is also acceptable to set only one of them.

[0070] The processing of the second embodiment is executed according to the processing procedure described above. This has the effect of extracting abnormal regions contained in medical image data more stably, with a higher detection rate, and with lower computational cost.

[0071] In the above-described embodiment, the identification unit 54 performed a positive / negative determination of specific anatomical structures, such as bone metastases, as an identification process for the processing target area set by the processing target area setting unit 53. However, the embodiment is not limited to this. For example, the identification unit 54 may perform a process other than positive / negative determination of anatomical structures as an identification process for the processing target area set by the processing target area setting unit 53. For example, the identification unit 54 may perform a segmentation process of specific anatomical structures as an identification process for the processing target area set by the processing target area setting unit 53.

[0072] The above is just one example of an embodiment, but the technology disclosed in this application is not limited to the embodiments shown above and in the drawings, and can be implemented by modifying it as appropriate without changing its essence.

[0073] <Other Embodiments> Furthermore, the disclosed technology can take the form of, for example, a system, apparatus, method, program, or recording medium (storage medium). Specifically, it may be applied to a system consisting of multiple devices (for example, a host computer, interface devices, imaging devices, web applications, etc.), or to an apparatus consisting of a single device.

[0074] Furthermore, it goes without saying that the objective of the technology disclosed in this application is achieved as follows: a recording medium (or storage medium) containing program code (computer program) of software that realizes the functions of the embodiments described above is supplied to a system or device. Such storage medium is, needless to say, a computer-readable storage medium. The computer (or CPU or MPU) of the system or device then reads and executes the program code stored on the recording medium. In this case, the program code read from the recording medium itself realizes the functions of the embodiments described above, and the recording medium containing that program code constitutes the technology disclosed in this application.

[0075] Furthermore, in the embodiment described above, the control unit 37 is implemented by a processing circuit such as a processor. In this case, the functions of each processing unit described above are stored in the ROM 32 in the form of a program that can be executed by a computer. The control unit 37 then reads and executes each program stored in the ROM 32 to realize the function corresponding to each program. In other words, the control unit 27, with each program read, has the processing units shown in Figure 1.

[0076] Furthermore, the embodiments described above illustrate an example in which the image acquisition means, focus area setting means, processing target area setting means, and identification means in this specification are implemented by the image acquisition unit 51, focus area setting unit 52, processing target area setting unit 53, and identification unit 54 of the control unit 37, respectively. However, the embodiments are not limited to this. For example, the image acquisition means, focus area setting means, processing target area setting means, and identification means in this specification may be implemented not only by the image acquisition unit 51, focus area setting unit 52, processing target area setting unit 53, and identification unit 54 of the control unit 37 as described in the embodiments, but also by hardware alone, software alone, or a combination of hardware and software.

[0077] Furthermore, in the embodiments described above, the control unit 37 is not limited to being implemented by a single processor. For example, the control unit 37 may be configured by combining multiple independent processors, with each processor executing a program to realize the functions of each processing unit. Also, the functions of each processing unit in the control unit 37 may be implemented by appropriately distributing or integrating them across one or more processing circuits. Moreover, the functions of each processing unit in the control unit 37 may be implemented by a mixture of hardware such as circuits and software. In addition, although an example in which programs corresponding to the functions of each processing unit are stored in a single ROM 32 has been described here, the embodiments are not limited to this. For example, programs corresponding to the functions of each processing unit may be distributed and stored in multiple memory circuits, and the control unit 37 may read and execute each program from each memory circuit.

[0078] Furthermore, the term "processor" used in the above-described embodiments refers to circuits such as CPUs, GPUs (Graphics Processing Units), Application Specific Integrated Circuits (ASICs), and programmable logic devices (e.g., Simple Programmable Logic Devices (SPLDs), Complex Programmable Logic Devices (CPLDs), and Field Programmable Gate Arrays (FPGAs)). Here, instead of storing the program in a memory circuit, the processor may be configured to directly incorporate the program into its circuitry. In this case, the processor realizes its function by reading and executing the program incorporated into the circuitry. Moreover, each processor in this embodiment is not limited to being configured as a single circuit; multiple independent circuits may be combined to form a single processor and realize its function.

[0079] Furthermore, in the embodiments described above, each component of each illustrated device is a functional concept and does not necessarily have to be physically configured as shown. In other words, the specific form of distribution or integration of each device is not limited to that shown, and all or part of them can be functionally or physically distributed or integrated in any unit according to various loads and usage conditions. Moreover, each processing function performed by each device can be implemented, in whole or in any part, by a CPU and a program that is analyzed and executed by the CPU, or by hardware using wired logic.

[0080] Furthermore, among the processes described in the embodiments and modifications described above, all or part of the processes described as being performed automatically can be performed manually, or all or part of the processes described as being performed manually can be performed automatically by known methods. In addition, the processing procedures, control procedures, specific names, and information including various data and parameters shown in the above document and drawings can be arbitrarily changed unless otherwise specified.

[0081] The various types of data discussed in this specification are typically digital data.

[0082] According to at least one embodiment described above, abnormal regions included in medical image data can be extracted more stably and with a higher detection rate.

[0083] While several embodiments have been described, these embodiments are presented as examples only and are not intended to limit the scope of the invention. These embodiments can be implemented in a variety of other forms, and various omissions, substitutions, modifications, and combinations of embodiments are possible without departing from the spirit of the invention. These embodiments and their variations are included in the scope and spirit of the invention, as well as in the claims and their equivalents. [Explanation of Symbols]

[0084] 10 Image Processing Device 37 Control Unit 51 Image acquisition unit 52 Area of ​​Interest Setting Unit 53 Processing Target Area Setting Unit 54 Identification unit

Claims

1. Image acquisition means for acquiring images, A means for setting a region of interest in the aforementioned image, Regarding the aforementioned area of ​​interest, a means for setting a processing target area for setting multiple processing target areas, Identification means for performing identification processing on the plurality of processing target areas Equipped with, The identification means terminates the identification process when a predetermined termination condition is met. Image processing device.

2. The identification means performs the identification process according to the processing order of the plurality of processing target areas determined based on a default evaluation function. The image processing apparatus according to claim 1.

3. Image acquisition means for acquiring images, A means for setting a region of interest in the aforementioned image, Regarding the aforementioned area of ​​interest, a means for setting a processing target area for setting multiple processing target areas, Identification means for performing identification processing on the plurality of processing target areas Equipped with, The processing target area setting means sets the plurality of processing target areas based on the information regarding the area of ​​interest, The identification means determines the processing order of the plurality of processing target areas based on a predetermined evaluation function, and performs the identification process according to the processing order. Image processing device.

4. The identification means terminates the identification process when a predetermined termination condition is met. The image processing apparatus according to claim 3.

5. The processing target area setting means sets the plurality of processing target areas based on the distance from the contour of the area of ​​interest. The image processing apparatus according to any one of claims 1 to 4.

6. The aforementioned default evaluation function evaluates the distance from the contour of the region of interest, The identification means determines the processing order of the plurality of processing target areas based on the evaluation result of the default evaluation function. The image processing apparatus according to any one of claims 2 to 4.

7. The aforementioned default termination condition includes either a positive determination condition, which is a condition for determining the result of the identification process as positive, or a negative determination condition, which is a condition for determining the result of the identification process as negative, and is a condition for terminating the identification process when the positive determination condition is met or when the negative determination condition is met. The image processing apparatus according to claim 1 or 4.

8. The positive determination condition is a condition for terminating the identification process when the result of the identification process is obtained to be equal to or greater than a predetermined threshold. The image processing apparatus according to claim 7.

9. The negative determination condition is a condition for terminating the identification process when the result of the identification process is obtained to be less than a predetermined threshold. The image processing apparatus according to claim 7.

10. The aforementioned default termination condition includes both a positive determination condition, which is a condition for determining the result of the identification process as positive, and a negative determination condition, which is a condition for determining the result of the identification process as negative, and is a condition for terminating the identification process when the positive determination condition is met or when the negative determination condition is met. The image processing apparatus according to claim 1 or 4.

11. Steps to acquire an image, The steps include setting a region of interest in the aforementioned image, The steps include setting multiple processing target areas for the aforementioned area of ​​interest, The steps include performing identification processing on the plurality of processing target areas. Includes, The step of performing the identification process includes terminating the identification process when a predetermined termination condition is met. Image processing methods.

12. Steps to acquire an image, The steps include setting a region of interest in the aforementioned image, The steps include setting multiple processing target areas for the aforementioned area of ​​interest, The steps include performing identification processing on the plurality of processing target areas. Includes, The step of setting the plurality of processing target areas includes setting the plurality of processing target areas based on information about the area of ​​interest, The step of performing the identification process includes determining the processing order of the plurality of processing target areas based on a default evaluation function, and performing the identification process according to that processing order. Image processing methods.

13. On the computer, The procedure for obtaining the image, The procedure for setting the area of ​​interest in the aforementioned image, The procedure for setting multiple processing target areas for the aforementioned area of ​​interest, A procedure for performing identification processing on the aforementioned multiple processing target areas. Make it run, The procedure for performing the identification process includes terminating the identification process when a predetermined termination condition is met. program.

14. On the computer, The procedure for obtaining the image, The procedure for setting the area of ​​interest in the aforementioned image, The procedure for setting multiple processing target areas for the aforementioned area of ​​interest, A procedure for performing identification processing on the aforementioned multiple processing target areas. Make it run, The procedure for setting the multiple processing target areas includes setting the multiple processing target areas based on information about the area of ​​interest, The procedure for performing the identification process includes determining the processing order of the multiple processing target areas based on a default evaluation function, and performing the identification process according to that processing order. program.

Images