Image processing device, radiographic system, operation method of image processing device, and program

Abstract

To acquire a cervical region ROI, which is a target region in the bone density measurement of the thighbone with a high degree of precision.SOLUTION: An image processing device for processing a medical image includes acquisition means for acquiring a region of interest of a cervical region including a cervical region of the thighbone in an input image by using the input image as input data on a learned model acquired by using learning data including information indicating a thighbone region in an image of the thighbone and learning data including information indicating a region of interest of the cervical region including the cervical region of the thighbone in the input image.SELECTED DRAWING: Figure 4

Description

[Technical field]

[0001] The disclosure of this specification relates to an image processing device, a radiation imaging system, an operation method of the image processing device, and a program. [Background technology]

[0002] Bone density is measured to diagnose osteoporosis and evaluate the effectiveness of treatments. 2 This is the bone mass per unit area. A known technique for measuring bone density is to use images acquired by DXA (Dual energy X-ray Absorptiometry). In DXA, X-rays are taken using two types of tube voltage, and bone and soft tissue and other materials are differentiated based on the difference in absorption rate. Images acquired in this way using DXA are called energy subtraction images (hereafter referred to as energy subtraction images) or material decomposition images.

[0003] For bone density measurement, it is necessary to obtain a target region for bone density measurement from the bone region of the ENE-sub image. When measuring bone density of the femur, the target region for bone density measurement is generally the neck region in the proximal part of the femur. Furthermore, the region around the thinnest part of the neck, which is approximately perpendicular to the cervical axis of the neck, is set as the neck ROI (Region Of Interest).

[0004] Patent Document 1 discloses a technique for automatically extracting a femoral neck region in an image using machine learning and obtaining a neck ROI based on this neck region. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Patent No. 7226199 Summary of the Invention [Problem to be solved by the invention]

[0006] In the above-mentioned Patent Document 1, the femoral neck region is extracted by a trained model created by a learning process using a teacher image with a label that can distinguish the femoral neck region from the background region. However, if there is an inference error in the output result of the trained model, an error will occur in the angle and position of the neck ROI. For example, if the shape of the bone in the input image is distorted or the quality of the input image is low due to a positional shift between images of two types of tube voltage, an inference error in the neck region is likely to occur.

[0007] Therefore, one of the objectives of one embodiment of the disclosure of this specification is to obtain, with high accuracy, a neck ROI, which is a target region for bone density measurement of the femur. [Means for solving the problem]

[0008] An image processing device according to one embodiment disclosed in this specification is an image processing device that processes medical images, and is equipped with an acquisition means that acquires a neck region of interest including the femoral neck in an input image by using an input image as input data for a trained model obtained using training data including information indicating a femur region in an image of the femur and training data including information indicating a neck region of interest including the femoral neck. Effect of the Invention

[0009] According to at least one embodiment of the disclosure of the present specification, the neck ROI, which is the target region for bone density measurement of the femur, can be obtained with high accuracy. [Brief description of the drawings]

[0010] [Figure 1] 1 shows an example of a schematic configuration of an X-ray imaging system according to a first embodiment. [Diagram 2] 3 shows an example of an input image according to the first embodiment. [Diagram 3] 3 shows an example of a neck ROI according to the first embodiment. [Figure 4] 4 shows an example of a flowchart of image processing according to the first embodiment. [Diagram 5]13 illustrates an example of an acquisition result by a first acquisition unit according to the first embodiment. [Figure 6] 4 shows an example of the position and angle of a neck ROI according to the first embodiment. [Figure 7] 4 illustrates an example of an input image to a second acquisition unit according to the first embodiment. [Figure 8] 13 illustrates another example of the image input to the second acquisition unit according to the first embodiment. [Figure 9] 3 shows an example of a neck shaft according to Example 1. [Figure 10] 11 shows an example of a radiation image according to the second embodiment. [Figure 11] 13 shows an example of a flowchart of image processing according to the second embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Hereinafter, exemplary embodiments for carrying out the present invention will be described in detail with reference to the drawings. In the drawings, the same reference numerals are used between the drawings to indicate identical or functionally similar elements. In addition, in each drawing, some components, members, and processes that are not important for the description may be omitted.

[0012] In the following embodiments, an imaging system using X-rays as an example of radiation will be described, but the imaging system according to the present disclosure may use other radiation. Here, the term radiation includes electromagnetic radiation such as X-rays and gamma rays, as well as particle radiation such as alpha rays, beta rays, particle beams, proton beams, heavy ion beams, and meson beams.

[0013] In the following, the machine learning model refers to a learning model based on a machine learning algorithm. Specific examples of machine learning algorithms include nearest neighbor, naive Bayes, decision trees, and support vector machines. In addition, deep learning, which uses a neural network to generate features and connection weighting coefficients for learning, can also be mentioned. In addition, as an algorithm using a decision tree, a method using gradient boosting such as LightGBM and XGBoost can also be mentioned. Appropriately, any of the above algorithms that can be used can be used in the following embodiments and modifications. In addition, teacher data refers to learning data, and is composed of a pair of input data and output data. In addition, output data of learning data is also called correct answer data.

[0014] A trained model refers to a model that has been trained (learned) in advance using appropriate teacher data (learning data) for a machine learning model that follows any machine learning algorithm such as deep learning. However, although a trained model is obtained in advance using appropriate learning data, this does not mean that no further learning is performed, and additional learning can be performed. Additional learning can also be performed after the device is installed at the site of use.

[0015] Example 1 A radiation imaging system, an image processing device, and an operation method of the image processing device according to a first embodiment of the present invention will be described below with reference to Figs. 1 to 9. Fig. 1 shows an example of the configuration of an X-ray imaging system 100, which is an example of a radiation imaging system according to this embodiment. The X-ray imaging system 100 includes an X-ray generator control unit 102, an X-ray tube 103, an FPD (Flat Panel Detector, FPDe) 106, and an image processing device 120. The configuration of the X-ray imaging system 100 is also simply referred to as an X-ray imaging device. The image processing device 120 processes information based on a captured X-ray image.

[0016] When the exposure switch is pressed, the X-ray generator control unit 102 applies a high voltage pulse to the X-ray tube 103 to generate X-rays, and the X-ray tube 103 irradiates the X-rays. When the X-ray tube 103 irradiates X-rays, the X-rays pass through an object (not shown) on a gantry 104 and a grid 105, and reach the FPD 106. The X-rays that reach the object are converted into visible light by the FPD 106 and output as an X-ray image. The output X-ray image is transferred to the image processing device 120 via the control unit 101.

[0017] The FPD 106 has an X-ray detection unit (not shown) equipped with a pixel array for generating a signal according to the X-ray. The X-ray detection unit detects the X-ray that has passed through a grid as an image signal. In the X-ray detection unit, pixels that output a signal according to the incident light are arranged in an array (two-dimensional area). The photoelectric conversion element of each pixel converts the X-ray that has been converted into visible light by a phosphor into an electric signal and outputs it as an image signal. In this way, the X-ray detection unit is configured to detect the X-ray that has passed through the grid and obtain an image signal (X-ray image). The FPD 106 outputs the image signal (X-ray image) read out according to an instruction from the control unit 101 to the control unit 101. The X-ray detection unit may be an indirect conversion type detection unit that converts the X-ray into visible light using a scintillator or the like and converts the visible light into an electric signal using a photoelectric conversion element, or may be a direct conversion type detection unit that directly converts the incident X-ray into an electric signal.

[0018] The control unit 101 transfers the X-ray image acquired from the FPD 106 to an image processing device 120 that processes the X-ray image, and to a storage unit 109 that stores the results of the image processing and various programs. The storage unit 109 is configured, for example, with a ROM (Read Only Memory) or a RAM (Random Access Memory). The storage unit 109 can store images output from the control unit 101, images processed by the image processing device 120, and calculation results calculated by the image processing device 120.

[0019] The image processing device 120 processes the input image and outputs the calculation result, the image, and the like. The image processing device 120 is provided with a first acquisition unit 121 and a second acquisition unit 122. The first acquisition unit 121 extracts and acquires a proximal femur region (femur region) including a region of the proximal part of the femur from the input image using a first learned model described later. The second acquisition unit 122 acquires a neck ROI, which is a target region of bone density measurement of the femur, based on the proximal femur region acquired by the first acquisition unit 121 using a second learned model described later. The details of the processing of the first acquisition unit 121 and the second acquisition unit 122 will be described later. In addition, the image processing device 120 can perform a bone density analysis on the acquired neck ROI and calculate the bone density.

[0020] The control unit 101 and the image processing device 120 can be configured by a computer provided with a processor and a memory. The control unit 101 and the image processing device 120 may be configured by a general computer or a computer dedicated to the X-ray imaging system.

[0021] The components of the control unit 101 and each component of the image processing device 120, such as the first acquisition unit 121, are functionally configured using, for example, one or more processors such as CPUs (Central Processing Units) and programs read from the storage unit 109. The processor may be, for example, an MPU (Micro Processing Unit), a GPU (Graphical Processing Unit), or an FPGA (Field-Programmable Gate Array). The components of the control unit 101 and each component of the image processing device 120 may be configured with an integrated circuit performing a specific function, such as an ASIC. The internal configuration of the control unit 101 and the image processing device 120 may also include a graphic control unit such as a GPU, a communication unit such as a network card, and an input / output control unit such as a keyboard, a display, or a touch panel.

[0022] The display unit 107 (monitor) displays the X-ray image (digital image) received by the control unit 101 from the FPD 106, the image processed by the image processing device 120, patient information, user notifications, and the like. The operation unit 108 can input instructions to the image processing device 120 and the FPD 106 via a user interface. The operation unit 108 may be configured using input devices such as a mouse and a keyboard. The control unit 101 functions as an example of a display control means that can control the display unit 107 in response to instructions from the operation unit 108, and the like.

[0023] In this embodiment, the image processing device 120 and the control unit 101 are described as separate devices, but they can be configured as the same device, for example, a single computer. The display unit 107 can be configured as a touch panel display, and in this case, the display unit 107 can also be used as the operation unit 108.

[0024] Fig. 2 shows an example of an image of the vicinity of the proximal femur acquired by the X-ray imaging system 100. For the purpose of explanation, Fig. 2 shows a head region 21 including the head of the femur, a trochanter region 22 including the trochanter of the femur, a subtrochanter region 23 including the lower part of the trochanter of the femur (in this specification, the shaft in the image is also treated as being divided into this region), and a pelvis region 24 including the pelvis. The proximal femur region is made up of the head region 21, the trochanter region 22, and the subtrochanter region 23.

[0025] Although the following describes the case where the left leg is photographed as shown in Fig. 2, the same processing may be performed when the opposite leg is photographed. In addition, when either the left or right side is set as the reference side and the opposite leg is photographed, the following processing may be performed on an image obtained by flipping the image obtained by the X-ray imaging system 100 from left to right.

[0026] Furthermore, Fig. 3 shows an example of a neck ROI (neck region of interest) according to this embodiment. In Fig. 3, a rectangle 25 corresponding to the neck ROI, which is the target of measurement of the bone density of the femur, is shown by a dashed line. The neck ROI is generally set as a rectangle. However, the neck ROI is not limited to a rectangle, and may be set in any shape according to a desired configuration.

[0027] The image processing device 120 in this embodiment acquires a neck ROI as shown in FIG. 3 using an image of the vicinity of the proximal femur as shown in FIG. 2 as an input image. The input image may be an energy sub-image acquired by the DXA method. This energy sub-image may be generated from an image X-rayed at two types of tube voltages by the kV switching method. This energy sub-image may also be generated from images of each layer obtained by one exposure using a stacked sensor formed of two types of materials (phosphors, scintillators) with different X-ray absorption rates as the FPD 106. As an example, these two types of materials may contain CsI (Tl) (thallium-activated cesium iodide) and GOS (gadolinium oxide) as main components. This stacked sensor may be configured so that the material on the side where the X-rays are incident (the subject side) contains CsI (Tl) and the material on the opposite side contains GOS. In contrast, a plain X-ray image acquired by one exposure (plain X-ray photography) may be used as the input image. Also, if the FPD 106 is a stacked sensor, an image of one of the layers (for example, the upper layer on the subject side) may be used as the input image.

[0028] Hereinafter, image processing of the image processing device 120 for acquiring the neck ROI in this embodiment will be described with reference to Fig. 4. Fig. 4 shows an image processing flow of the image processing device 120 for acquiring the neck ROI in this embodiment. When the image processing flow according to this embodiment is started, the process proceeds to step S11 (first acquisition step).

[0029] In step S11, the first acquisition unit 121 of the image processing device 120 extracts (segments) and acquires the femoral head region, the trochanter region, the subtrochanter region, and the pelvis region from the input image using the first learned model. In this way, the first acquisition unit 121 is an example of a first acquisition means that acquires the femoral proximal region in the input image using the input image as input data for the first learned model. In this embodiment, the first acquisition unit 121 acquires the pelvis region, which is a bone region other than the femoral region, in addition to the femoral proximal region, in order to use it when acquiring the background region in a later step. In the following, the acquisition process of various regions such as the femoral head region, the trochanter region, the subtrochanter region, the pelvis region, and the neck ROI includes the acquisition of images (information) showing these regions and the identification of these regions.

[0030] An example of the results extracted (acquired) by the first acquisition unit 121 is shown in FIG. 5(a) to FIG. 5(d). The white area in FIG. 5(a) shows the extraction result (acquisition result) of the femoral head area, the white area in FIG. 5(b) shows the extraction result of the trochanter area, the white area in FIG. 5(c) shows the extraction result of the subtrochanter area, and the white area in FIG. 5(d) shows the extraction result of the pelvis area. As an example of the brightness value of the image showing the extraction result of these areas, for example, the brightness value of the white area which is the bone area may be 1, and the brightness value of the black area which is the background area may be 0. An image including a label indicating a predetermined area, such as an image in which the bone area is expressed by a brightness value, (annotated) is hereinafter referred to as a label image. Note that the label is not limited to a brightness value, and may be any numerical value, character, or the like according to a desired configuration. Also, a different label may be assigned to each area.

[0031] The first trained model may be stored in the storage unit 109, and the first acquisition unit 121 may call up and use the first trained model stored in the storage unit 109. The trained models for extracting the regions shown in FIG. 5(a) to FIG. 5(d) may be a common model. When an image of the vicinity of the proximal femur is input, such a trained model can output the extraction results of the femoral head region, the trochanter region, the subtrochanter region, and the pelvis region from each channel. The trained models for extracting each region may be provided separately. These trained models may have the same structure, with only the weights being different. In this embodiment, the X-ray imaging system 100 is configured to include the trained model, but the trained model may be provided outside the X-ray imaging system 100.

[0032] The first trained model here can be generated, for example, by training a combination of an image of the area around the proximal femur and an image in which the femoral head region, trochanter region, subtrochanter region, and pelvic region within that image are labeled as training data.

[0033] In addition, when a trained model for extracting each region is provided separately, the labeled images may be trained separately. Specifically, a trained model for extracting the femoral head region may be generated by using a combination of an image around the proximal femur and an image in which the femoral head region in the image is labeled as training data. Similarly, a trained model for extracting the trochanter region may be generated by using a combination of an image around the proximal femur and an image in which the trochanter region in the image is labeled as training data. In addition, a trained model for extracting the subtrochanter region may be generated by using a combination of an image around the proximal femur and an image in which the subtrochanter region in the image is labeled as training data. Furthermore, a trained model for extracting the pelvic region may be generated by using a combination of an image around the proximal femur and an image in which the pelvic region in the image is labeled as training data.

[0034] In this embodiment, for example, the control unit 101 can function as an example of a learning unit that learns the learned model. However, the first acquisition unit 121 may use the first learned model that has been learned by the image processing device 120 or another learning device.

[0035] Moreover, the GPU can perform efficient calculations by processing more data in parallel. For this reason, when learning is performed multiple times using a machine learning algorithm such as deep learning, it is effective to perform the processing with the GPU. Therefore, in this embodiment, the processing by the control unit 101 functioning as an example of a learning unit may use a GPU in addition to a CPU. Specifically, when a learning program including a learning model is executed, learning can be performed by the CPU and the GPU working together to perform calculations. In addition, in the processing of the learning unit, calculations may be performed only by the CPU or the GPU. In addition, the extraction processing according to this embodiment may also be realized using a GPU like the learning unit. In addition, when the learned model is provided in an external device, the control unit 101 does not need to function as a learning unit.

[0036] The learning unit may also include an error detection unit and an update unit, both not shown. The error detection unit obtains an error between correct data and output data output from the output layer of the neural network according to input data input to the input layer. The error detection unit may use a loss function to calculate an error between the output data from the neural network and correct data. The update unit updates the connection weighting coefficients between the nodes of the neural network, etc., based on the error obtained by the error detection unit, so as to reduce the error. The update unit updates the connection weighting coefficients, etc., using, for example, an error backpropagation method. The error backpropagation method is a method of adjusting the connection weighting coefficients between the nodes of each neural network so as to reduce the error.

[0037] As the machine learning model according to the present embodiment, for example, FCN (Fully Convolutional Network), SegNet, etc. can be used. As the machine learning model for performing object recognition, for example, RCNN (Region CNN), fastRCNN, or fasterRCNN can be used. Furthermore, as the machine learning model for performing object recognition in units of regions, YOLO (You Only Look Once), SSD (Single Shot Detector, or Single Shot MultiBox Detector) can be used.

[0038] Furthermore, the image processing device 120 may perform preprocessing on the input image before the segmentation process. An example of the preprocessing may be a process for improving contrast. As an example of the process for improving contrast, for example, a process for linearly extending the luminance gradation by setting the maximum luminance value in the image to 1 and the minimum luminance value in the image to 0 may be mentioned. As an example of the preprocessing, a process such as histogram equalization may be mentioned. As an example of the process for enhancing the edge, for example, a filter process may be mentioned. As an example of the preprocessing, a standardization process for setting the average value of the luminance value of the image to 0 and the variance to 1 may be mentioned. Furthermore, the image processing device 120 may perform a combination of the above preprocessing. Note that these preprocessing processes may also be performed on the learning data to be learned by the first learned model.

[0039] The input image may be an image obtained by trimming an image acquired by the X-ray imaging system 100 in response to an instruction from the operator via the operation unit 108 while viewing the image displayed on the display unit 107. The trimmed image may be resized to a specified image size. The specified image size referred to here may be, for example, the input image size of the trained model. The input image may be an image automatically trimmed by the image processing device 120 based on the imaging range so as to leave the proximal part of the femur.

[0040] Furthermore, if the input image is an image cropped by an operator, the input image may not include the subtrochanteric region. In this case, the output of the first trained model may not include the subtrochanteric region. Also, the image acquired by the X-ray imaging system 100 may not include the subtrochanteric region.

[0041] In addition, if a specifically small area is also extracted outside the extracted area for each target area, the first acquisition unit 121 may regard this area as a segmentation error and remove it from the extraction result, leaving only the area with the largest area.

[0042] Furthermore, the control unit 101 may cause the display unit 107 to display the proximal femur region acquired by the first acquisition unit 121, and the operator may use the operation unit 108 to instruct corrections such as addition or deletion of a bone region. In this case, the control unit 101 sends the operator's instructions to the image processing device 120, and the image processing device 120 can make corrections such as addition or deletion of a bone region in response to the acquired instructions from the operator.

[0043] Here, when the label of the femoral head region and the label of the trochanter region acquired by the first acquisition unit 121 are common, an instruction to add a bone region can be given by specifying the location where the same label value is to be added to the displayed proximal femoral region.

[0044] Furthermore, when the label of the femoral head region and the label of the trochanter region acquired by the first acquisition unit 121 are different, the operator can specify the location of the femoral head region and the trochanter region where the bone region is to be added to the displayed proximal femur region separately. By such an operation, it is possible to give an instruction to add and correct the femoral head region and the trochanter region, respectively.

[0045] Even if the label of the femoral head region and the label of the trochanter region acquired by the first acquisition unit 121 are different, the operator may instruct additional correction using a bone region label common to the femoral head region and the trochanter region. In this case, the first acquisition unit 121 may distinguish whether the added bone region is a femoral head region or a trochanter region. The label of the added bone region can be set to, for example, the label of the adjacent pixel of its outer frame.

[0046] The image processing device 120 may also use the first acquired model to acquire a label image in which the femoral head region and the trochanter region of the input image are regions with a common label attached, by the first acquired model, through the first acquisition unit 121. In this case, the first acquisition unit 121 may further acquire a label image in which the femoral head region and the trochanter region are separate regions from the acquired label image. The first acquisition unit 121 may use the third acquired model to acquire a label image in which the femoral head region and the trochanter region are separately extracted from a label image in which the femoral head region and the trochanter region are commonly labeled. Here, the learning data of the third acquired model may be a label image in which the femoral head region and the trochanter region are commonly labeled as input data, and a label image in which the femoral head region and the trochanter region in the label image are separately labeled as output data. The first acquisition unit 121 may acquire a labeled image in which the femoral head region and the trochanter region are separately extracted from a labeled image in which a common label is assigned to the femoral head region and the trochanter region by rule-based processing. The first acquisition unit 121 may include a third acquisition unit and a fourth acquisition unit that perform these processes, respectively.

[0047] When the neck region is extracted from the input image using a trained model, it is difficult to confirm the cause of any abnormality in the results or to correct it midway. In contrast, the image processing device 120 can more reliably obtain the final neck ROI by displaying and correcting the proximal femur region obtained midway as described above.

[0048] Next, in step S12 (second acquisition step), the second acquisition unit 122 of the image processing device 120 acquires a neck ROI from the acquisition result of step S11 (FIG. 5) using the second trained model.

[0049] As described above, when an image processing device is used to extract the femoral neck region and neck ROI from an input image using one trained model, if there is an inference error in the output result of the trained model, an error in the angle and position of the neck ROI will occur. For example, if the shape of the bone in the input image is distorted or the quality of the input image is low due to misalignment between images of two types of tube voltage, an inference error in the neck region is likely to occur. In addition, the size of the neck ROI to be set may differ depending on the policy of the facility or the technician. When extracting the neck region using a trained model, the size of the neck region and neck ROI depends on the training data. Since re-training is required to change the output of the trained model, it is not easy to change the settings of the neck ROI in accordance with the facility's policy. Alternatively, the operator will have to take the trouble of correcting the output of the trained model.

[0050] In contrast, the second acquisition unit 122 according to this embodiment acquires the position and angle of the neck ROI in the input image from the femur proximal region, which is the acquisition result of step S11, by using the second learned model. In this way, the second acquisition unit 122 is an example of a second acquisition means that acquires a neck region of interest, which is a target region for femur bone density measurement, by using the femur proximal region acquired by the first acquisition means as input data for the second learned model.

[0051] FIG. 6 shows an example of the position and angle of the neck ROI (rectangle 25) according to this embodiment. The position of the neck ROI here can be set as the center position RP of the neck ROI (rectangle 25) as shown in FIG. 6. The angle of the neck ROI can be set as the angle of a line parallel to the long side of the rectangle (crossing the neck) as shown by line RA in FIG. 6, relative to a line in the left-right direction of the image. However, the method of taking the position and angle of the neck ROI is not limited to these, and any representative position and angle can be set.

[0052] The second acquisition unit 122 can set a fixed value in advance as the size of the neck ROI (length and width of the rectangle). In this way, the second acquisition unit 122 can acquire the neck ROI placed at the correct position by setting the size of the neck ROI according to the policy of the facility, and then specifying the output of the trained model as the position and angle of the neck ROI. The size of the neck ROI may be changed depending on the subject's physique, the size and positional relationship of the femoral head region and the trochanter region acquired in step S11, and the like. In this case, the second acquisition unit 122 may determine the size of the neck ROI by, for example, referring to a table that stores the correspondence between the subject's physique, the size and positional relationship of the femoral head region and the trochanter region, and the size of the neck ROI, or by using a function related to the correspondence.

[0053] The image input by the second acquisition unit 122 to the second trained model may include a label image of the femoral head region as shown in FIG. 5(a) and a label image of the trochanter region as shown in FIG. 5(b). Here, the second trained model may be configured to input a tensor having these two images in separate channels. The input data of the second trained model may be a single label image in which the femoral head region and the trochanter region are labeled as shown in FIG. 7, and in this case, the labels of the femoral head region and the trochanter region may be a common label. The input data of the second trained model may be a label image in which the brightness values ​​of the femoral head region (shaded area in FIG. 8) and the trochanter region (white area in FIG. 8) are set individually as shown in FIG. 8.

[0054] The second trained model used in this embodiment may be a regression model in which the input data is a labeled image as described above, and the input image or the position and angle of the neck ROI in the labeled image is output data. In this case, the second trained model can be generated by, for example, training a combination of the labeled image as described above and the input image corresponding to the labeled image or the position and angle of the neck ROI in the labeled image as training data.

[0055] The task of specifying the position and angle of the neck ROI of the learning data may be a task of specifying a numerical value. The task of specifying the position and angle of the neck ROI of the learning data may be a task of specifying the center position RP of the neck in the image and a line RA that passes through the center position RP and is parallel to the long side of the rectangle, as shown in FIG. 6. In this case, the angle of the line RA can be obtained by post-processing, and the angle of the line RA can be specified as the angle of the neck ROI. The task of specifying the position and angle of the neck ROI of the learning data may be a task of specifying a rectangle 25 as shown in FIG. 6, and the center position and angle of the rectangle can be obtained by post-processing. The rectangle specified at this time may be a size different from that of the neck ROI.

[0056] The second trained model may be a model that extracts a region corresponding to the line RA as shown in FIG. 6, which is specified as training data, from the input labeled image by processing such as semantic segmentation. In this case, the second trained model may be obtained by training using training data in which the above-mentioned labeled image is used as input data and the input image or the region corresponding to the line RA in the labeled image is used as output data. The second acquisition unit 122 may acquire an approximation line of the line RA by performing post-processing such as linear approximation on the extracted region, and acquire the center position and angle of the approximation line as the position and angle of the neck ROI. In this case, the center position of the neck ROI may be the center of gravity position of the region where the acquired approximation line and the bone region extracted by the first acquisition unit 121 overlap.

[0057] The second trained model may be a model that extracts a rectangular region 25 corresponding to the neck ROI as shown in FIG. 6, which is specified as training data, from the input label image by segmentation processing. In this case, the second trained model may be obtained by training training data in which the above-mentioned label image is used as input data and the input image or the region of the rectangular region 25 in the label image is used as output data. The second acquisition unit 122 may approximate the extracted region to a rectangle, and then acquire the center position and the angle of the long side of the approximated rectangular region as the position and angle of the neck ROI. For example, by providing training data as a rectangular region for the input label image, the appropriate size of the neck ROI can also be trained.

[0058] The second trained model may be a single model that outputs the position and angle of the cervical ROI in separate channels. In addition, a model that outputs the position of the cervical ROI and a model that outputs the angle of the cervical ROI may be provided separately as the second trained model. In this case, the model that outputs the position of the cervical ROI may be obtained by training learning data in which the label image as described above is used as input data and the position of the cervical ROI in the input image or the label image is used as output data. In addition, the model that outputs the angle of the cervical ROI may be obtained by training learning data in which the label image as described above is used as input data and the angle of the cervical ROI in the input image or the label image is used as output data.

[0059] The second trained model may output the position of the cervical ROI and the angle of the cervical axis 26 shown in Fig. 9. The cervical axis 26 may be set as a line that passes through the center position of the cervical ROI and is perpendicular to the long side of the cervical ROI, for example. In this case, the second trained model may be obtained by training learning data in which the label image as described above is used as input data, and the position of the cervical ROI in the input image or the label image and the angle of the cervical axis 26 are used as output data.

[0060] In addition, the control unit 101 may determine the neck axis 26 based on the position of the neck ROI and the angle of the long side output by the second learned model, and cause the display unit 107 to display the neck axis 26.

[0061] Furthermore, the second acquisition unit 122 may acquire an offset value for the position or angle of the neck ROI output by the second trained model. This offset value may be determined according to the policy of the facility or the technician.

[0062] The input data to the second trained model may be an image in which an input image (such as an energy-subimage) is superimposed on the label image acquired by the first acquisition unit 121. In this case, the input data of the training data may also be a similar image.

[0063] Furthermore, the input data to the second trained model may be two images, a label image acquired by the first acquisition unit 121 and an input image (such as an energy-subimage). These two images may be input to separate channels, or may be combined and input to one channel. In this case, the input data of the training data may also be similar images.

[0064] The second acquisition unit 122 in this embodiment acquires the position and angle of the neck ROI based on the label image (information) indicating the femur region acquired by the first acquisition unit 121. In this way, by acquiring the global structure of the femur region once approximately and searching for the neck ROI on it, it is possible to acquire the neck ROI with higher accuracy even when the shape of the bone is distorted or the quality of the input image is low. In addition, in this embodiment, the position and angle of the neck ROI are acquired by a learned model, and the size is arbitrarily set by a rule, so that the examination setting can be performed according to the policy of the facility or the medical technician.

[0065] In this way, the second acquisition unit 122 can extract and acquire the femoral neck ROI with high accuracy by referring to the acquisition result of the femoral proximal region including the femoral head region and the trochanter region. When the neck ROI is acquired in step S12, the process proceeds to step S13.

[0066] Next, in step S13, the second acquisition unit 122 sets a background region for calculating the transmittance of the bone. An example of a method for setting the background region will be described below.

[0067] An example of the background region may be a region other than the bone region in the input image as shown in FIG. 2, and is automatically selected. The bone region here is a region including the femoral head region, the trochanter region, the subtrochanter region, and the pelvis region acquired in step S11. Therefore, the second acquisition unit 122 can set, for example, a region in the input image excluding the bone region acquired in step S11 as the background region. Also, when the input image is an image trimmed by an operator, the background region may be a region other than the bone region in the trimmed image. In this case, the second acquisition unit 122 can set, for example, a region in the trimmed image excluding the bone region acquired in step S11 as the background region. However, the method of setting the background region is not limited to this, and any method may be used for setting. For example, the operator may specify the background region using the operation unit 108.

[0068] The control unit 101 may display the neck ROI set as described above on the display unit 107. The control unit 101 may also display the neck axis together. Furthermore, based on the neck ROI displayed on the display unit 107, the operator may use the operation unit 108 to correct and determine the position and angle of the neck ROI. Furthermore, the operator may correct the size of the neck ROI by specifying the positions of the four corners of the neck ROI using the operation unit 108. Note that the control unit 101 does not need to display the position of the background region on the display unit 107.

[0069] Furthermore, the control unit 101 can cause the display unit 107 to display a warning message when at least one of the results of the position and angle of the acquired neck ROI is determined to be abnormal.

[0070] An example of a condition for determining an abnormality here is when the position of the neck ROI is closer to the pelvis than the center position of the femoral head region. The position of the neck ROI here may refer to the left-right position (X coordinate) of the center of the neck ROI (center position RP in FIG. 6) acquired in step S12. The center position of the femoral head region here may refer to the left-right position (X coordinate) of the calculated center position, for example, calculated for the femoral head region (FIG. 5(a)) acquired in step S11. Then, if the X coordinate of the neck ROI is closer to the pelvis than the X coordinate of the center of the femoral head region, it may be determined to be abnormal. The "pelvis side" here refers to the left side of the image when the input image is an image of the left foot as shown in FIG. 6, and conversely, refers to the right side of the image when the input image is an image of the right foot.

[0071] Another example of a condition for determining an abnormality is when the angle of the neck ROI is different from the reference angle by more than a threshold value. The reference angle and threshold value here can be arbitrarily specified. For example, as shown in FIG. 6, when the input image is an image of the left leg, the reference angle of the neck ROI (the angle of the line RA in FIG. 6 relative to the line in the left-right direction of the image) may be set to 45 degrees, and the threshold angle may also be set to 45 degrees. That is, in this example, when the angle of the neck ROI is 90 degrees or more, it may be determined to be abnormal. As in these examples, the determination of whether or not there is an abnormality may be performed by rule-based processing.

[0072] The image processing device 120 calculates the bone density in the neck ROI by referring to the neck ROI and the background region acquired as described above. The bone density can be calculated by a known method, and may be calculated using information on the thickness of the bones and soft tissues in the image acquired by the DXA method, for example. The bone density may also be calculated using information on the atomic number and surface density acquired by the DXA method. The control unit 101 may display the calculated bone density on the display unit 107. Furthermore, the bone region referred to for the bone density calculation is the neck region in the neck ROI, and bone regions other than the neck region included in the neck ROI (such as the pelvic region) may not be referred to.

[0073] The control unit 101 may cause the display unit 107 to display any value useful for diagnosis based on the calculated bone density. For example, the control unit 101 may cause the display unit 107 to display a value compared with the average value of bone density for people of the same age or other ages. The control unit 101 may also cause the display unit 107 to display time-series data compared with, for example, past measurement results. In addition to the bone density value calculated within the neck ROI, the display unit 107 may display a bone density value calculated using the entire bone region or a part of the bone region in the input image and the background region.

[0074] As described above, the X-ray imaging system 100 according to the present embodiment functions as an example of a radiation imaging system and includes the image processing device 120, the X-ray tube 103, and the FPD 106. The X-ray tube 103 functions as an example of a radiation generating device that generates radiation. The FPD 106 functions as an example of a radiation detecting device that detects radiation and generates a radiation image. The image processing device 120 functions as an example of an image processing device that processes medical images and includes a first acquisition unit 121 and a second acquisition unit 122. The first acquisition unit 121 and the second acquisition unit 122 can function as an example of an acquisition means that acquires a neck region of interest including the neck of the femur in an input image by using an input image as input data of a trained model obtained using training data including information indicating a proximal femur region (femur region) in an image of the femur and training data including information indicating a neck region of interest including the neck of the femur.

[0075] More specifically, the trained model may include a first trained model obtained using training data including information indicating a femur region in an image of a femur, and a second trained model obtained using training data including information indicating a neck region of interest including a neck of the femur. The first acquisition unit 121 may function as an example of a first acquisition means for acquiring information indicating the femur region in the input image by using the input image as input data of the first trained model. The second acquisition unit 122 may function as an example of a second acquisition means for acquiring a neck ROI (neck region of interest) in the input image by using information indicating the femur region acquired as input data of the second trained model. The neck region of interest in the input image may be a target region for bone density measurement. The femur region may include a region of the proximal part of the femur.

[0076] With this configuration, the image processing device 120 according to this embodiment first obtains an approximate global structure of the femur region, and then searches for the neck ROI. Therefore, even if the bone shape is deformed or the quality of the input image is low, the neck ROI can be obtained with high accuracy.

[0077] The second acquisition unit 122 acquires the neck region of interest in the input image using the position and angle of the neck region of interest based on the output from the second trained model. The size of the neck region of interest in the input image may be set in advance or may be set by the second acquisition unit 122. With this configuration, the size of the neck ROI is set according to the policy of the facility, and then the output of the trained model is specified as the position and angle of the neck ROI, so that the neck ROI placed in the correct position can be acquired. The size of the neck region of interest in the input image can be set according to the subject's physique and the size and positional relationship of the acquired femoral head region and trochanter region. Therefore, the examination can be set according to the policy of the facility or the medical technician.

[0078] The first acquisition unit 121 can also acquire a femoral head region including a femoral head region in the input image. The first acquisition unit 121 can also acquire at least one of a trochanter region including a femoral trochanter region, a subtrochanter region including a subtrochanter region of the femur, and a pelvis region including a pelvis region in the input image. The first acquisition unit 121 can separately acquire information indicating the femoral head region and information indicating the trochanter region using the first trained model, or can acquire information indicating a combined region of the femoral head region and the trochanter region.

[0079] The input image may be an energy subtraction image, at least one of two images acquired using radiation with two types of tube voltages, at least one of two images acquired using a sensor having two types of materials with different radiation absorption rates (stacked sensor), or a simple radiation image. One of the two types of materials may include CsI, and the other of the two types of materials may include GOS.

[0080] The second acquisition unit 122 may further acquire a neck axis. Furthermore, the second acquisition unit 122 may acquire a neck region of interest in the input image by using an image in which the information indicating the femur region acquired by the first acquisition unit 121 and the input image are superimposed. In this case, the number of features that can be extracted by the second trained model increases, so that it is expected that the neck ROI can be acquired with higher accuracy.

[0081] Furthermore, the second acquisition unit 122 can acquire a background region in the input image based on the femur region acquired by the first acquisition unit 121. Therefore, the image processing device 120 can perform bone density measurement of the femur using the neck ROI set according to the position and angle of the acquired neck ROI and pixel values ​​in the acquired background region. Note that the first acquisition unit 121 may further extract a bone region other than the femur region in the input image, and the second acquisition unit 122 may use the bone region other than the femur region to extract the background region.

[0082] The image processing device 120 further includes a control unit 101. The control unit 101 can function as an example of a display control means for displaying the intermediate acquisition results of the femur region by the first acquisition unit 121 and the acquisition results of the neck ROI by the second acquisition unit 122 on the display unit 107. In this case, the operator can check the acquisition results by the first acquisition unit 121 or the acquisition results by the second acquisition unit 122 displayed on the display unit 107 and check whether they correspond to the desired position or angle of the femur region or neck ROI.

[0083] Moreover, the image processing device 120 may further include an operation unit 108 that allows the operator to operate the correction of the results acquired by the first acquisition unit 121 and the results acquired by the second acquisition unit 122 while viewing the display by the display unit 107. In this case, the operator can confirm the display on the display unit 107 and correct and determine the position and angle of the femur region and the neck ROI. When the results acquired by the first acquisition unit 121 are corrected, the second acquisition unit 122 can acquire the neck region of interest in the input image by using the femur region corrected using the operation unit 108 as input data for the second trained model.

[0084] Furthermore, when the neck ROI (acquisition result) acquired by the second acquisition unit 122 or the acquisition process is determined to be abnormal, the control unit 101 can display a warning such as a message indicating the abnormality on the display unit 107. In this case, the operator can grasp the abnormality of the acquisition result or the acquisition process, and change the settings according to a desired policy or the like, or stop image processing.

[0085] The first acquisition unit 121 may also acquire information indicating a single region in which the femoral head region and the trochanter region of the input image are treated as a single region by using the input image as input data for the first trained model. In this case, the first acquisition unit 121 may acquire information indicating the femoral region in the input image by separately extracting information indicating the femoral head region and information indicating the trochanter region from the information indicating the single region. In this case, the operation unit 108 may be used to operate the correction of the single region.

[0086] The first acquisition unit 121 may separately acquire information indicating the femoral head region and information indicating the trochanter region in the acquired single region by using information indicating the single region acquired as input data for the trained model. The trained model can be obtained using training data including information indicating a femoral region obtained by separately extracting the femoral head region and the trochanter region in a single region including the femoral head region and the trochanter region. In contrast, the first acquisition unit 121 may separately acquire information indicating the femoral head region and information indicating the trochanter region in the single region by rule-based processing.

[0087] In this embodiment, the image processing device 120 has a first acquisition unit 121 that acquires a proximal femur region using a first trained model, and a second acquisition unit 122 that acquires a neck ROI using a second trained model. Here, the first acquisition unit 121 and the second acquisition unit 122 may configure a single acquisition unit that acquires the neck ROI from an input image using a trained model obtained using training data including information indicating the femur region in an image of the femur and training data including information indicating a neck region of interest including the neck of the femur.

[0088] In addition, such an acquisition unit may be configured to acquire a neck ROI using an input image as input data of a trained model in which the first trained model and the second trained model are integrated. The integration of the first trained model and the second trained model can be performed, for example, by connecting the output of the first trained model to the input of the second trained model. Here, as described above, the first trained model may be obtained using training data including information indicating a femur region in an image of the femur. In addition, the second trained model may be obtained using training data including information indicating a neck region of interest including the neck of the femur.

[0089] The trained model in which the first trained model and the second trained model are integrated extracts a femoral proximal region including a femoral head region and a trochanter region from an input image used as input data by a configuration having parameters corresponding to the first trained model. The trained model also outputs the position and angle of the neck ROI based on the extracted femoral proximal region by a configuration having parameters corresponding to the second trained model. The acquisition unit using the trained model can acquire the neck ROI in the input image using the position and angle of the neck ROI based on the output from the trained model, similar to the second acquisition unit 122. Even with such a configuration, the global structure of the femoral region is approximately acquired, and the neck ROI is searched for in the acquired femoral region, so that the neck ROI can be acquired with higher accuracy even when the shape of the bone is deformed or the quality of the input image is low.

[0090] Such an acquisition unit can function as an example of an acquisition means for acquiring a neck region of interest including the femoral neck in an input image by using the input image as input data for a trained model obtained using training data including information indicating the femoral region in an image of the femur and training data including information indicating the neck region of interest including the femoral neck.

[0091] Example 2 Next, a second embodiment of the present invention will be described. A configuration example of an X-ray imaging system according to this embodiment is similar to the configuration example of the X-ray imaging system 100 according to the first embodiment, and is shown in Fig. 1. Note that the configuration example of the X-ray imaging system according to this embodiment will be assigned the same reference numerals as the configuration example of the X-ray imaging system 100 according to the first embodiment, and description thereof will be omitted.

[0092] The image processing device 120 in this embodiment extracts the femoral neck ROI using a radiographic image as shown in Fig. 10. The radiographic image shown in Fig. 10 may be, for example, a simple X-ray image, and is an X-ray image acquired over a wide area, not just around the neck ROI. The input image may be an image acquired by the DXA method. This embodiment will be described below, focusing on the differences from the first embodiment.

[0093] 11 shows an image processing flow of the image processing device 120 for extracting the cervical ROI in this embodiment. When the image processing according to this embodiment is started, the process proceeds to step S21.

[0094] In step S21, the first acquisition unit 121 uses the learned model to extract the femoral head region and the trochanter region of the leg to be measured from the left and right legs in the acquired X-ray image. Here, a case will be described in which the left leg (the side on which the reference region 31 is shown) shown in Fig. 10 is the measurement target.

[0095] The trained model here can be generated by training a combination of an X-ray image of a human body or a phantom including the femur and an image in which the femoral head region and the trochanter region in the X-ray image are labeled as training data. The femoral head region and the trochanter region may be extracted by the same process as step S11 in the first embodiment, and the first acquisition unit 121 may preprocess the X-ray image before inputting it to the trained model. The trained model may extract the femoral head region and the trochanter region separately, or may extract them together in one region.

[0096] The trained model used in step S21 may extract the femoral head region and the trochanter region of both legs. The first acquisition unit 121 may extract the femoral head region and the trochanter region of the leg to be measured by referring to whether the central coordinates of the femoral head region are on the right side or the left side of the image.

[0097] The trained model used in step S21 may extract the femoral head region, the trochanter region, the subtrochanter region, and the pelvis region in the X-ray image, so that the trained model used in step S21 can share the same model structure as the trained model used in the subsequent step S23.

[0098] Next, in step S22, the first acquisition unit 121 extracts a reference region 31 in the X-ray image based on the femoral head region and the trochanter region extracted in step S21. The reference region 31 is a region around the proximal femur, such as the range region of the input image (FIG. 2) in the first embodiment, as shown in FIG.

[0099] When the reference region 31 is a rectangle, the first acquisition unit 121 can draw each side of the rectangle at a position with a margin on the outside from the upper end, lower end, left end, and right end of the femoral head region and the trochanter region. The margin here can be set to any value, for example, 1 cm. The method of setting the reference region is not limited to this, and the reference region may be a rectangular region with a margin on the outside based on at least one of the femoral head region and the trochanter region. The reference region is not limited to a rectangle, and may be any shape, such as a circle.

[0100] In this manner, the first acquisition unit 121 can extract a reference region from the acquired X-ray image.

[0101] Next, in step S23, the first acquisition unit 121 extracts the femoral head region, the trochanter region, the subtrochanter region, and the pelvis region in the reference region. In this process, the first acquisition unit 121 can perform the same process as step S11 in the first embodiment by using the image of the reference region as the input image. In this embodiment, the reference region is cut out from the X-ray image, and then the femoral head region, the trochanter region, the subtrochanter region, and the pelvis region are extracted, thereby preventing a decrease in model accuracy due to other objects being captured during learning. In this way, the first acquisition unit 121 according to this embodiment is also an example of a first acquisition means for acquiring the femoral proximal region in the input image using a trained model.

[0102] In step S24, the second acquisition unit 122 extracts a neck ROI based on the femoral head region, the trochanter region, and the subtrochanter region in the extracted reference region. That is, the second acquisition unit 122 is an example of a second acquisition means that acquires a neck ROI, which is a target region for bone density measurement of the femur, based on the acquisition result of the first acquisition unit 121. The process of this step can be performed using the same process as step S12 of the first embodiment.

[0103] In step S25, the second acquisition unit 122 extracts a background region in the reference region based on the pelvis region in the extracted reference region. The method of extracting the background region is the same as in step S13 in the first embodiment.

[0104] By referring to the neck ROI and background region extracted as described above, the image processing device 120 according to this embodiment can calculate the bone density in the neck ROI in the same manner as in embodiment 1. Also, the control unit 101 according to this embodiment may cause the display unit 107 to display the calculated bone density, the set neck axis and neck ROI, the acquisition result or the acquisition process of the neck ROI, etc., in the same manner as in embodiment 1.

[0105] As described above, the first acquisition unit 121 according to this embodiment can extract a reference region including a measurement target from a medical image, and use an image showing the reference region as an input image. Here, the reference region can be extracted based on at least one of a femoral head region including a femoral head region in a radiological image and a trochanter region including a femoral trochanter region. With this configuration, even if an image obtained by photographing a wide range is acquired as a radiological image, a reference region limited to a site to be measured for bone density can be extracted. Therefore, it is possible to suppress a decrease in accuracy of an extraction process using a trained model due to unnecessary features being captured in the image.

[0106] In this embodiment, an acquisition unit may be provided that acquires a neck ROI by using a reference region as input data of a trained model that integrates the first trained model and the second trained model. In this case, the trained model may be configured, for example, by connecting the output of the first trained model described above to the input of the second trained model. In addition, the acquisition unit may extract a reference image from an input image in the same manner as the first acquisition unit 121.

[0107] A modification of at least one of the first and second embodiments of the present invention will be described below.

[0108] (Modification) In the above-mentioned embodiment 2, the neck ROI and background region are extracted after extracting the reference region of the input image, but the neck ROI and background region may be extracted directly from the input image captured over a wide range in the same processing flow as in embodiment 1. In this case, for example, the first acquisition unit 121 can acquire the femur approximate region from the input image captured over a wide range, and the second acquisition unit 122 can acquire the neck ROI and background region using the femur approximate region.

[0109] The trained models according to the above-mentioned first and second embodiments and the modified examples may extract the femoral head region in the input image and other bone regions in which the trochanter region, subtrochanter region, and pelvis region are grouped into one label. Then, the neck ROI and background region may be obtained in the same manner as in the first and second embodiments.

[0110] The image processing device 120 according to the above-mentioned first and second embodiments and the modified example may extract the neck ROI using a total of three images, including two images related to different radiation energies acquired by the DXA method and an energy sub-image generated from these images. For example, the first acquisition unit 121 may extract the proximal femur region from the three images when extracting the proximal femur region (or each of the femoral head region, the trochanter region, the subtrochanter region, and the pelvis region) using a learned model in step S11 of the first embodiment. In this case, the second acquisition unit 122 may use only the region where the extraction results overlap as the final proximal femur region. Also, the second acquisition unit 122 may use only the region extracted as the proximal femur region in two or more of the three images as the final proximal femur region.

[0111] In this case, the X-ray imaging system 100 may have trained models for the three images individually. In this case, each trained model may be generated by training the images corresponding to the respective types of the three images (the first and second images related to different radiation energies, and the energy-sub-images generated from these images) as input data of the respective training data. Note that, as output data of the training data, images in which the proximal femur region is labeled for each type of image may be used. Also, each input channel of one trained model may correspond to these three images. Similarly, the image processing device 120 may extract the proximal femur region using two of these three images.

[0112] As described above, the first acquisition unit 121 according to this modification uses at least two of the three images, namely, two images acquired using the DXA method and an energy subtraction image generated from the two images, as input images to acquire femoral regions for the at least two images. The second acquisition unit 122 acquires a neck region of interest based on the acquisition results of the femoral regions for the at least two images. By performing the process according to this modification, the effect of segmentation errors can be reduced.

[0113] The learning data of the trained model according to the above-mentioned first and second embodiments and the modified example is not limited to data obtained using the X-ray imaging system itself that actually performs imaging. The image data may be data obtained using the same type of X-ray imaging system or data obtained using the same kind of X-ray imaging system, depending on the desired configuration.

[0114] Furthermore, when multiple images are used as input to the trained model, the multiple input images can be input to multiple input channels of the trained model, respectively. Alternatively, the multiple input images may be combined into one image and input to one channel of the trained model. In this case, the single image obtained by combining the input images may be used as input data for the trained model as well.

[0115] In the above-mentioned first and second embodiments and the modified example, the control unit 101 may superimpose the segmentation result of the proximal femur on the display image and display it on the display unit 107 in response to an instruction from the operator, such as the operation of a button (not shown). When there are a plurality of display images and corresponding segmentation results, the control unit 101 may superimpose the segmentation results on each display image in a lump and display them on the display unit 107 in response to an instruction from the operator. Similarly, the control unit 101 may superimpose the neck ROI, the neck axis, the acquisition result of the femur region by the first acquisition unit 121, etc. on each display image in a lump and display them on the display unit 107 in response to an instruction from the operator. The control unit 101 may also display the acquisition results of the neck ROI and the background region in different colors. In this case, a plurality of images may be displayed in a lump and color-coded manner in response to an instruction from the operator.

[0116] In addition, in the trained models according to the above-mentioned Examples 1 and 2 and the modified examples, it is considered that the magnitude of the luminance values ​​of the input data images, the order and inclination of the bright and dark areas, the position, distribution, continuity, etc. are extracted as part of the features and used in the inference process.

[0117] Moreover, the various trained models according to the above-mentioned first and second embodiments and the modified examples can be provided in the control unit 101 or the image processing device 120. The trained model may be configured, for example, by a software module executed by a processor such as a CPU, MPU, GPU, or FPGA, or may be configured by a circuit that performs a specific function such as an ASIC. The trained model may also be provided in another server device connected to the control unit 101 or the image processing device 120. In this case, the image processing device 120 can use the trained model by connecting to a server or the like equipped with the trained model via any network such as the Internet. Here, the server equipped with the trained model may be, for example, a cloud server, a fog server, an edge server, or the like. Note that, when configuring a network within a facility, a site including a facility, or an area including multiple facilities to be capable of wireless communication, the reliability of the network may be improved by configuring the network to use radio waves in a dedicated wavelength band that is limited to the facility, site, area, or the like. The network may also be configured by wireless communication that is capable of high speed, large capacity, low latency, and multiple simultaneous connections.

[0118] In addition, the bone region may be extracted by a known threshold process instead of using a trained model. For example, the first acquisition unit 121 can extract and acquire pixels whose brightness is equal to or greater than a threshold as the bone region in the input image.

[0119] In the first and second embodiments and the modified examples, images captured using radiation are used as an example of medical images processed by the image processing device 120. The medical images to be processed may include images captured by, for example, digital radiography (DR) using an FPD or computed tomography (CT). The medical images may also include images captured by photon counting CT, magnetic resonance imaging (MRI), ultrasound imaging, or the like. The medical images may also include images captured by positron emission tomography (PET), single photon emission computed tomography (SPECT), or the like. The medical images may also be still images or moving images.

[0120] (Other Examples) The present invention can also be realized by supplying software (programs) that realize one or more functions of the various embodiments and modifications described above to a system or device via a network or a storage medium, and having a computer in the system or device read and execute the programs. It can also be realized by a circuit (e.g., ASIC) that realizes one or more functions. A computer has one or more processors or circuits, and may include separate computers or a network of separate processors or circuits to read and execute computer-executable instructions.

[0121] In this case, the processor or circuitry may include a central processing unit (CPU), a microprocessing unit (MPU), a graphics processing unit (GPU), an application specific integrated circuit (ASIC), or a field programmable gateway (FPGA). The processor or circuitry may also include a digital signal processor (DSP), a data flow processor (DFP), or a neural processing unit (NPU).

[0122] The above disclosure includes the following configurations, methods, and programs. (Configuration 1) An image processing device for processing medical images, comprising an acquisition means for acquiring a neck region of interest including the femoral neck in an input image by using an input image as input data for a trained model obtained using training data including information indicating a femur region in an image of the femur and training data including information indicating a neck region of interest including the femoral neck. (Configuration 2) The trained model is A first trained model obtained using training data including information indicating a femur region in an image of the femur; a second trained model obtained using training data including information indicative of a neck region of interest including a femoral neck; Including, The acquisition means includes: a first acquisition means for acquiring information indicating a femur region in the input image by using the input image as input data of the first trained model; a second acquisition means for acquiring a neck region of interest in the input image by using information indicating the acquired femur region as input data of the second trained model; 2. The image processing device according to claim 1, (Configuration 3) The image processing device of configuration 2, wherein the second acquisition means acquires a neck region of interest in the input image using a position and an angle of the neck region of interest based on an output from the second trained model. (Configuration 4) the femoral region includes a head and a trochanter; 4. The image processing device according to configuration 2 or 3, wherein the first acquisition means acquires information indicating a femoral head region and information indicating a trochanter region separately using the first trained model. (Configuration 5) the femoral region includes a head and a trochanter; 4. The image processing device according to configuration 2 or 3, wherein the first acquisition means acquires information indicating a region that combines a region of a femoral head and a region of a trochanter using the first trained model. (Configuration 6) 6. The image processing device according to any one of configurations 2 to 5, wherein the second acquisition means further acquires a neck axis. (Configuration 7) 7. The image processing device according to any one of configurations 1 to 6, wherein a size of the cervical region of interest in the input image is set in advance. (Configuration 8) 7. The image processing device according to any one of configurations 2 to 6, wherein a size of the cervical region of interest in the input image is set by the second acquisition means. (Configuration 9) The first acquisition means further extracts a bone region other than the femur region in the input image, 9. The image processing device according to any one of configurations 2 to 6 and 8, wherein a bone region other than the femur region is used for extracting a background region. (Configuration 10) An operation unit for operating a correction of the acquired femoral region, The image processing device according to any one of configurations 2 to 6, 8, and 9, wherein the second acquisition means acquires a neck region of interest in the input image by using the femur region corrected using the operation unit as input data for the second trained model. (Configuration 11) The image processing device according to any one of configurations 2 to 6 and 8 to 10, wherein the second acquisition means acquires a neck region of interest in the input image using an image in which the acquired information indicating the femur region and the input image are superimposed. (Configuration 12) The first acquisition means is By using an input image as input data for the first trained model, information indicating a single region in which a region of a femoral head and a region of a trochanter in the input image are defined as a single region is obtained; An image processing device according to any one of configurations 2 to 6 and 8 to 11, which obtains information indicating the femoral region in the input image by separately extracting information indicating the femoral head region and information indicating the trochanter region from the information indicating the single region. (Configuration 13) 13. The image processing device according to claim 12, further comprising an operation unit for operating the modification of the single region. (Configuration 14) The image processing device of configuration 12 or 13, wherein the first acquisition means acquires information indicating the femoral head region and information indicating the trochanter region in the acquired single region separately by using the information indicating the acquired single region as input data of a trained model obtained using training data including information indicating a femoral region in which the femoral head region and the trochanter region are separately extracted in a single region including the femoral head region and the trochanter region. (Configuration 15) 14. The image processing device according to configuration 12 or 13, wherein the first acquisition means acquires information indicating a femoral head region and information indicating a trochanter region in the single region separately by rule-based processing. (Configuration 16) The image processing device according to configuration 1, wherein the trained model is a trained model that integrates a first trained model obtained using training data including information indicating a neck region of interest including a femoral neck, and a second trained model obtained using training data including information indicating a neck region of interest including a femoral neck. (Configuration 17) 17. The image processing device of claim 16, wherein the acquisition means acquires a cervical region of interest in the input image using a position and angle of the cervical region of interest based on an output from the trained model. (Configuration 18) 18. The image processing device according to any one of configurations 1 to 17, wherein the neck region of interest in the input image is a target region for bone density measurement. (Configuration 19) a display control means for displaying a neck region of interest in the input image on a display unit; 19. The image processing device according to any one of configurations 1 to 18, further comprising: an operation unit that allows an operator to operate a correction of the cervical region of interest in the input image based on the display of the display unit. (Configuration 20) 19. The image processing device according to any one of configurations 1 to 18, further comprising a display control means for displaying a warning on a display unit when the acquired cervical region of interest is determined to be abnormal. (Configuration 21) 21. The image processing device according to any one of configurations 1 to 20, wherein the input image is any one of an energy subtraction image, at least one of two images acquired using radiation of two types of tube voltages, at least one of two images acquired using a sensor having two types of materials with different radiation absorption rates, or a simple radiation image. (Configuration 22) 22. The image processing device of claim 21, wherein one of the two types of materials includes CsI and the other of the two types of materials includes GOS. (Configuration 23) 23. The image processing device of any one of configurations 1 to 22, wherein the femur region includes a proximal femur region. (Configuration 24) 24. The image processing device according to any one of configurations 1 to 23, wherein the acquisition means extracts a reference region including a measurement object from a medical image, and uses an image indicating the reference region as the input image. (Configuration 25) An image processing device according to any one of configurations 1 to 24, A radiation generating device that generates radiation; a radiation detection device that detects the radiation and generates a radiation image; A radiation imaging system comprising: (Method 1) A method for operating an image processing device that processes medical images, the method including: obtaining a neck region of interest including the femoral neck in an input image by using an input image as input data for a trained model obtained using training data including information indicating a femur region in an image of the femur and training data including information indicating a neck region of interest including the femoral neck. (Program 1) A program that, when executed by a computer, causes the computer to execute the method for operating an image processing device according to method 1.

[0123] Although the present invention has been described above with reference to the examples and modifications, the present invention is not limited to the above examples and modifications. The present invention also includes inventions that have been modified within the scope of the present invention and inventions equivalent to the present invention. In addition, the above-mentioned examples and modifications can be appropriately combined within the scope of the present invention. [Explanation of symbols]

[0124] 120: image processing device, 121: first acquisition unit (acquisition unit), 122: second acquisition unit (acquisition unit)

Claims

1. An image processing device for processing medical images, comprising an acquisition means for acquiring a neck region of interest including the femoral neck in an input image by using an input image as input data for a trained model obtained using training data including information indicating a femur region in an image of the femur and training data including information indicating a neck region of interest including the femoral neck.

2. The trained model is A first trained model obtained using training data including information indicating a femur region in an image of the femur; a second trained model obtained using training data including information indicative of a neck region of interest including a femoral neck; Including, The acquisition means includes: a first acquisition means for acquiring information indicating a femur region in the input image by using the input image as input data of the first trained model; a second acquisition means for acquiring a neck region of interest in the input image by using information indicating the acquired femur region as input data of the second trained model; The image processing device according to claim 1 ,

3. The image processing device according to claim 2 , wherein the second acquisition means acquires a neck region of interest in the input image using a position and an angle of the neck region of interest based on an output from the second trained model.

4. the femoral region includes a head and a trochanter; The image processing device according to claim 2 , wherein the first acquisition means acquires information indicating a femoral head region and information indicating a trochanter region separately using the first trained model.

5. the femoral region includes a head and a trochanter; The image processing device according to claim 2 , wherein the first acquisition means acquires information indicating a region that is a combination of a femoral head region and a trochanter region using the first trained model.

6. The image processing apparatus according to claim 2 , wherein the second acquisition means further acquires a neck axis.

7. The image processing apparatus according to claim 1 , wherein a size of the cervical region of interest in the input image is preset.

8. The image processing apparatus according to claim 2 , wherein a size of the cervical region of interest in the input image is set by the second acquisition means.

9. The first acquisition means further extracts a bone region other than the femur region in the input image, The image processing apparatus according to claim 2 , wherein a bone region other than the femur region is used for extracting a background region.

10. An operation unit for operating a correction of the acquired femoral region, The image processing device according to claim 2 , wherein the second acquisition means acquires a neck region of interest in the input image by using the femur region corrected using the operation unit as input data for the second trained model.

11. The image processing apparatus according to claim 2 , wherein the second acquisition means acquires the neck region of interest in the input image by using an image in which the acquired information indicating the femur region and the input image are superimposed.

12. The first acquisition means includes: By using an input image as input data for the first trained model, information indicating a single region in which a region of a femoral head and a region of a trochanter in the input image are defined as a single region is obtained; The image processing device according to claim 2 , wherein information indicating the femur region in the input image is obtained by separately extracting information indicating the femoral head region and information indicating the trochanter region from the information indicating the single region.

13. The image processing device according to claim 12 , further comprising an operation unit for operating the modification of the single region.

14. The image processing device according to claim 12, wherein the first acquisition means acquires information indicating the femoral head region and information indicating the trochanter region in the acquired single region separately by using the information indicating the acquired single region as input data for a trained model obtained using training data including information indicating a femoral region in which the femoral head region and the trochanter region are separately extracted in a single region including the femoral head region and the trochanter region.

15. The image processing apparatus according to claim 12 , wherein the first acquisition means acquires information indicating a femoral head region and information indicating a trochanter region in the single region separately by rule-based processing.

16. 2. The image processing device according to claim 1, wherein the trained model is a trained model that is an integration of a first trained model obtained using training data including information indicating a neck region of interest including a femoral neck and a second trained model obtained using training data including information indicating a neck region of interest including a femoral neck.

17. The image processing device according to claim 16 , wherein the acquisition means acquires a neck region of interest in the input image using a position and an angle of the neck region of interest based on an output from the trained model.

18. The image processing apparatus according to claim 1 , wherein the neck region of interest in the input image is a target region for bone density measurement.

19. a display control means for displaying a neck region of interest in the input image on a display unit; The image processing apparatus according to claim 1 , further comprising: an operation unit that allows an operator to operate a correction of the cervical region of interest in the input image based on the display on the display unit.

20. The image processing apparatus according to claim 1 , further comprising a display control unit that displays a warning on a display unit when the acquired cervical region of interest is determined to be abnormal.

21. 2. The image processing device according to claim 1, wherein the input image is any one of an energy subtraction image, at least one of two images acquired using radiation of two types of tube voltages, at least one of two images acquired using a sensor having two types of materials with different radiation absorption rates, and a simple radiation image.

22. 22. The image processing apparatus of claim 21, wherein one of the two types of materials includes CsI and the other of the two types of materials includes GOS.

23. The imaging device of claim 1 , wherein the femur region includes a region of a proximal portion of the femur.

24. The image processing apparatus according to claim 1 , wherein the acquisition means extracts a reference region including the measurement object from the medical image, and uses an image indicating the reference region as the input image.

25. An image processing device according to any one of claims 1 to 24; A radiation generating device that generates radiation; a radiation detection device that detects the radiation and generates a radiation image; A radiation imaging system comprising:

26. A method for operating an image processing device that processes medical images, the method including: obtaining a neck region of interest including the femoral neck in an input image by using an input image as input data for a trained model obtained using training data including information indicating a femur region in an image of the femur and training data including information indicating a neck region of interest including the femoral neck.

27. A program which, when executed by a computer, causes the computer to carry out the method of operating an image processing apparatus according to claim 26.

Images