Image processing device, method for operating image processing device, and program

Abstract

To acquire a bone density analysis area of a lumbar spine accurately.SOLUTION: An image processing device includes: first acquisition means for acquiring a spinal region and an intervertebral region in an input image; and second acquisition means for acquiring a bone density analysis area, which is an object region to measure bone density of a lumbar spine, using the acquired spinal region and the acquired intervertebral region.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The disclosure of this specification relates to an image processing device, an operating method for an image processing device, and a program. [Background technology]

[0002] Bone density is measured to diagnose osteoporosis and assess the effectiveness of therapeutic drugs. 2 This is the bone mass per unit area. A known technique for measuring bone density is to use images acquired by DXA (Dual Energy X-ray Absorptiometry). DXA uses two types of tube voltage to take X-ray images, and differentiates between bone and soft tissue based on the difference in absorption rate. Images acquired using DXA in this way are called energy subtraction images (material decomposition images).

[0003] To measure bone density, it is necessary to obtain the target area for bone density measurement (hereinafter referred to as bone density analysis area) from the bone area of ​​the ENE-sub image. When measuring bone density of the lumbar vertebrae, the bone density analysis area is generally from the first to the fourth lumbar vertebrae or from the second to the fourth lumbar vertebrae.

[0004] Patent Document 1 discloses a technique for automatically acquiring a bone density analysis region using a machine learning model. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Patent Publication No. 2021-37164 Summary of the Invention [Problem to be solved by the invention]

[0006] In Patent Document 1, a bone density analysis region within the femur is obtained by machine learning. However, because the spine is made up of a continuous series of similar structures between vertebral bodies and intervertebral spaces, when segmentation of the lumbar vertebrae is performed using a machine learning model, there is a risk that regions that should be obtained will be omitted, resulting in the bone density analysis region being obtained shifted up or down.

[0007] Therefore, one of the objectives of the present disclosure is to obtain a lumbar bone density analysis region with high accuracy, and another objective is to obtain an analysis region in which similar structures are continuously connected with high accuracy. [Means for solving the problem]

[0008] An image processing device according to one embodiment disclosed in this specification has a first acquisition means for acquiring a spinal region and an intervertebral region in an input image, and a second acquisition means for acquiring a bone density analysis region, which is a target region for bone density measurement of the lumbar spine, using the acquired spinal region and the acquired intervertebral region. [Effects of the Invention]

[0009] According to at least one embodiment of the present disclosure, a lumbar bone density analysis region can be obtained with high accuracy. Also, according to at least another embodiment of the present disclosure, an analysis region in which similar structures are continuously connected can be obtained with high accuracy. [Brief explanation of the drawings]

[0010] [Figure 1] 1 shows an example of a schematic configuration of an X-ray imaging system according to a first embodiment. [Figure 2] 3 shows an example of an input image according to the first embodiment. [Figure 3] 3 shows an example of a flowchart of image processing according to the first embodiment. [Figure 4] 10 illustrates an example of an acquisition result obtained by a first acquisition unit according to the first embodiment. [Figure 5] 3 shows an example of an intervertebral region to be corrected according to the first embodiment. [Figure 6] 10 shows another example of an intervertebral region to be corrected according to the first embodiment. [Figure 7] 3 shows an example of a reference region according to the first embodiment. [Figure 8] 1 shows an example of a rotation correction method according to the first embodiment. [Figure 9] 3 shows an example of a method for generating a straight line according to the first embodiment. [Figure 10] 3 shows an example of a background region according to the first embodiment. [Figure 11] 10 shows an example of a flowchart of image processing according to the second embodiment. [Figure 12] 10 shows an example of a spine image according to the second embodiment. [Figure 13] 11 shows an example of a flowchart of image processing according to the third embodiment. [Figure 14] 13 illustrates an example of an acquisition result obtained by a first acquisition unit according to the third embodiment. [Figure 15] 13 shows an example of a flowchart of image processing according to the fourth embodiment. [Figure 16] 13 illustrates an example of an acquisition result obtained by a first acquisition unit according to the fourth embodiment. [Figure 17] 10 shows an example of a schematic configuration of a radiation imaging system according to a fifth embodiment. [Figure 18A] FIG. 10 is a diagram for explaining an example of a radiation imaging process according to the fifth embodiment. [Figure 18B] FIG. 10 is a diagram for explaining an example of a radiation imaging process according to the fifth embodiment. [Figure 19] 13A and 13B are diagrams for explaining an acquisition target and a correction target according to a fifth embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0011] Hereinafter, exemplary embodiments for carrying out the present invention will be described in detail with reference to the drawings. In the drawings, the same reference numerals are used between the drawings to indicate identical or functionally similar elements. In addition, in each drawing, some components, parts, and processes that are not important for the explanation may be omitted.

[0012] In the following examples, an imaging system using X-rays as an example of radiation will be described, but the imaging system according to the present disclosure may use other types of radiation. Here, the term "radiation" includes, in addition to X-rays, for example, alpha rays, beta rays, gamma rays, particle rays, and cosmic rays.

[0013] In the following, a machine learning model refers to a learning model based on a machine learning algorithm. Specific examples of machine learning algorithms include nearest neighbor algorithms, naive Bayes algorithms, decision trees, and support vector machines. Another example is deep learning, which uses a neural network to generate features and connection weighting coefficients for learning. Algorithms using decision trees also include gradient boosting techniques such as LightGBM and XGBoost. Any of the above algorithms can be used as appropriate and applied to the following embodiments and modifications. Training data refers to training data and is composed of pairs of input data and output data.

[0014] A trained model is a machine learning model that follows any machine learning algorithm, such as deep learning, and has been trained (learned) in advance using appropriate training data (learning data). However, although a trained model is obtained in advance using appropriate training data, it does not mean that it does not undergo further learning, and additional learning can also be performed. Additional learning can also be performed after the device is installed at the site of use.

[0015] Example 1 An image processing device and an operation method of the image processing device according to a first embodiment of the present invention will be described below with reference to Figs. 1 to 10. Fig. 1 is a diagram showing an example of the configuration of an X-ray imaging system 100, which is an example of a radiation imaging system according to this embodiment. The X-ray imaging system 100 includes an X-ray generator control unit 102, an X-ray tube 103, an FPD (Flat Panel Detector) 106, and an image processing device 120. The configuration of the X-ray imaging system 100 is also simply referred to as an X-ray imaging device. The image processing device 120 processes information based on a captured X-ray image.

[0016] When the exposure switch is pressed, the X-ray generator control unit 102 applies a high voltage pulse to the X-ray tube 103 to generate X-rays, and the X-ray tube 103 irradiates the X-rays. When the X-ray tube 103 irradiates X-rays, the X-rays pass through an object (not shown) on a gantry 104 and a grid 105, and reach the FPD 106. The arriving X-rays are converted into visible light by the FPD 106 and output as an X-ray image. The output X-ray image is transferred to the image processing device 120 via the control unit 101.

[0017] The FPD 106 has an X-ray detection unit (not shown) equipped with a pixel array for generating signals according to X-rays. The X-ray detection unit detects X-rays that have passed through a grid as image signals. The X-ray detection unit has pixels arranged in an array (two-dimensional area) that output signals according to incident light. The photoelectric conversion element of each pixel converts X-rays, which have been converted into visible light by a phosphor, into an electrical signal and outputs it as an image signal. In this way, the X-ray detection unit is configured to detect X-rays that have passed through the grid and acquire an image signal (X-ray image). The FPD 106 outputs the image signal (X-ray image) read out in accordance with instructions from the control unit 101 to the control unit 101.

[0018] The control unit 101 transfers the X-ray images acquired from the FPD 106 to an image processing device 120 that processes the X-ray images, and to a storage unit 109 that stores the results of image processing and various programs. The storage unit 109 is configured, for example, with a ROM (Read Only Memory) or a RAM (Random Access Memory). The storage unit 109 can store images output from the control unit 101, images processed by the image processing device 120, and calculation results calculated by the image processing device 120.

[0019] The image processing device 120 processes an input image and outputs a calculation result, an image, etc. The image processing device 120 is provided with a first acquisition unit 121 and a second acquisition unit 122. The first acquisition unit 121 extracts and acquires a spinal region and an intervertebral region from the input image using a trained model described later. The second acquisition unit 122 acquires a bone density analysis region, which is a target region for lumbar bone density measurement, using the spinal region and intervertebral region acquired by the first acquisition unit 121. Details of the processing by the first acquisition unit 121 and the second acquisition unit 122 will be described later. The image processing device 120 can also perform a bone density analysis on the acquired bone density analysis region and calculate bone density.

[0020] The control unit 101 and the image processing device 120 can be configured by a computer provided with a processor and a memory. The control unit 101 and the image processing device 120 may be configured by a general computer or a computer dedicated to the X-ray imaging system.

[0021] The components of the control unit 101 and each component of the image processing device 120, such as the first acquisition unit 121, are functionally configured using, for example, one or more processors, such as CPUs (Central Processing Units), and programs loaded from the storage unit 109. The processor may be, for example, an MPU (Micro Processing Unit), a GPU (Graphical Processing Unit), or an FPGA (Field-Programmable Gate Array). The components of the control unit 101 and each component of the image processing device 120 may be configured using an integrated circuit that performs a specific function, such as an ASIC. The internal configuration of the control unit 101 and the image processing device 120 may also include a graphics control unit such as a GPU, a communication unit such as a network card, and an input / output control unit such as a keyboard, display, or touch panel.

[0022] The display unit 107 (monitor) displays the X-ray image (digital image) received by the control unit 101 from the FPD 106, the image processed by the image processing device 120, patient information, user notifications, etc. The operation unit 108 can input instructions to the image processing device 120 and the FPD 106 via a user interface. The operation unit 108 may be configured using input devices such as a mouse and a keyboard. The control unit 101 functions as an example of a display control means that can control the display unit 107 in response to instructions from the operation unit 108, etc.

[0023] In this embodiment, the image processing device 120 and the control unit 101 are described as separate devices, but they can also be configured as the same device, for example, a single computer. The display unit 107 can be configured as a touch panel display, and in this case, the display unit 107 can also be used as the operation unit 108.

[0024] Fig. 2 shows an example of an image of the vicinity of the lumbar vertebrae acquired by the X-ray imaging system 100. For the sake of explanation, Fig. 2 also shows the first lumbar vertebra L1, the second lumbar vertebra L2, the third lumbar vertebra L3, and the fourth lumbar vertebra L4, which are examples of bone density measurement targets. The image processing device 120 in this embodiment uses the image of the vicinity of the lumbar vertebrae shown in Fig. 2 as an input image and extracts a bone density analysis region.

[0025] The input image may be an energy subtraction image (energy-subimage) acquired by DXA. This energy-subimage may be generated from images taken by X-ray imaging at two different tube voltages using a kV switching method. Furthermore, this energy-subimage may be generated from images of each layer obtained by a single exposure using a stacked sensor formed of two types of materials (phosphors) with different X-ray absorption rates as the FPD 106. A simple X-ray image acquired by a single exposure (simple X-ray imaging) may also be used as the input image. Furthermore, if the FPD 106 is a stacked sensor, an image of the upper layer (subject side) may also be used as the input image. Note that multiple images related to different energies acquired by DXA may also be used as the input images.

[0026] Hereinafter, image processing of the image processing device 120 for extracting a bone density analysis region in this embodiment will be described with reference to Fig. 3. Fig. 3 shows the image processing flow of the image processing device 120 for extracting a bone density analysis region in this embodiment. When the image processing flow according to this embodiment starts, the process proceeds to step S11.

[0027] In step S11, the first acquisition unit 121 of the image processing device 120 extracts (segments) a spine region and an intervertebral region from the input image using a trained model. The spine region here refers to a region including the outline of the spine and its interior. Alternatively, only the outline (outer frame) of the spine may be treated as the spine region. The intervertebral region here refers to a region including the outline of the intervertebral space and its interior. Alternatively, only the outline (outer frame) of the intervertebral space may be treated as the intervertebral region. Alternatively, the intervertebral region may be a line. In this way, the first acquisition unit 121 is an example of a first acquisition means that acquires the spine region and the intervertebral region in the input image using a trained model. Note that, hereinafter, the acquisition process of various regions such as the spine region, intervertebral region, and bone density analysis region includes the acquisition of images showing these regions and the identification of these regions.

[0028] Examples of the extraction (acquisition) results are shown in Figures 4(a) and 4(b). The white area in Figure 4(a) is the extraction result (acquisition result) of the spinal region, and the white area in Figure 4(b) is the extraction result of the intervertebral region. Figure 4(a) shows an example in which the spinous processes are not included in the spinal region, but the spinous processes may also be included.

[0029] The trained model may be stored in the storage unit 109, and the first acquisition unit 121 may call up and use the trained model stored in the storage unit 109. A trained model for extracting the spine region and a trained model for extracting the intervertebral region may be provided separately. These trained models may have the same structure, with only the weights differing. Alternatively, the trained model for extracting the spine region and the trained model for extracting the intervertebral region may be a common model, and extraction results for the spine region and the intervertebral region may be output from each channel.

[0030] The trained model here can be generated, for example, by training a combination of an image near the lumbar vertebrae and an image within the image labeled with the spinal region and intervertebral region as training data. If separate trained models for extracting spinal regions and intervertebral regions are provided, the labeled images can be trained separately. Specifically, each trained model can be generated by training a combination of an image near the lumbar vertebrae and an image within the image labeled with the spinal region, and a combination of an image near the lumbar vertebrae and an image within the image labeled with the intervertebral region, respectively, as training data.

[0031] Furthermore, the image processing device 120 may perform preprocessing on the input image before the segmentation process. One example of this processing may be a process to improve contrast. An example of the contrast improvement processing is a process of linearly stretching the maximum brightness value in the image to 1 and the minimum brightness value in the image to 0. Another example of the preprocessing may be a process of histogram equalization. Another example of the preprocessing may be a process of enhancing edges. An example of the edge enhancement processing is a filter process. Another example of the preprocessing may be a standardization process of setting the average brightness value of the image to 0 and the variance to 1. Furthermore, the image processing device 120 may perform a combination of the above preprocessing processes. Note that these preprocessing processes may also be performed on training data to be trained by a trained model.

[0032] The input image may be an image obtained by trimming an image acquired by the X-ray imaging system 100 using the operation unit 108 while the operator is viewing the image displayed on the display unit 107. The trimmed image may be resized to a specified image size. The specified image size may be, for example, the input image size of the trained model. The input image may also be an image obtained by trimming an area larger than the trimming area selected by the operator. For example, if the operator selects a trimming area to include the second lumbar vertebra L2 through the fourth lumbar vertebra L4, the input image may be an image obtained by vertically extending the trimming area so that the fifth lumbar vertebra L5 is also included in the input image. Therefore, as an example, the image processing device 120 may generate an input image to include an area obtained by vertically extending the area trimmed by the operator by 1 / 4 times in the direction of the fifth lumbar vertebra L5. The input image may also be an image automatically trimmed by the image processing device 120 based on the imaging range to leave the area near the lumbar vertebrae. In these cases, a similarly trimmed image may also be used as input data for the training data.

[0033] Furthermore, if there is a region extracted as an intervertebral area outside the extracted spinal area, the first acquiring unit 121 may regard this region as a segmentation error and remove it from the extraction result of the intervertebral area.

[0034] Next, in step S12, the second acquisition unit 122 of the image processing device 120 extracts a bone density analysis region from the extraction result of step S11 (FIGS. 4(a) and 4(b)). Here, a case will be described in which the region of the second lumbar vertebra L2 to the fourth lumbar vertebra L4 is extracted as the bone density analysis region. Note that the combination of bone density analysis regions is not limited to this, and for example, the region of the first lumbar vertebra L1 may also be extracted.

[0035] As described above, the spine is composed of a continuous series of similar structures between vertebral bodies and intervertebral spaces. Therefore, when lumbar vertebrae are segmented using a trained model that performs processing according to learning trends, regions that should be acquired may be omitted or excessive regions may be acquired. Figure 5(a) shows an example of the extraction result of an intervertebral region when an intervertebral region that should be acquired is omitted or excessive regions are acquired. When a bone density analysis region is set based on the intervertebral region shown in Figure 5(a), the region of the lumbar vertebrae extracted between the intervertebral regions (bone density analysis region) will be shifted from its original position due to the omitted intervertebral region and the excessive region acquired.

[0036] In contrast, the second acquisition unit 122 according to this embodiment performs a correction process for the intervertebral regions extracted using the trained model, using the relationship between the spinal region and the intervertebral regions included in the spinal region and the relationship between the intervertebral regions. The second acquisition unit 122 extracts the bone density analysis region based on the corrected intervertebral regions, thereby enabling the extraction of the bone density analysis region with high accuracy.

[0037] First, the second acquisition unit 122 calculates the center of gravity of each intervertebral region extracted in step S11. Next, the second acquisition unit 122 calculates the vertical position of these center of gravity positions (center of gravity Y coordinate). Note that, in the following, the vertical coordinate in the image is referred to as the Y coordinate, and the horizontal position is referred to as the X coordinate. Any known method may be used to calculate the center of gravity. In FIG. 5(a), the Y coordinate of the center of gravity for an example of the extraction result of the intervertebral region described above is indicated by a dotted line. Based on the interval between these center of gravity Y coordinates, the second acquisition unit 122 performs a correction process, as described below, as necessary.

[0038] As illustrated by coordinates H1 and H2 in FIG. 5(a), if the distance between the Y coordinates of the centers of gravity of the intervertebral regions is smaller than a set threshold, the second acquisition unit 122 may consider one of the corresponding intervertebral regions to be an outlier (or an inference error) and exclude it from the intervertebral regions. As an example, if the median value of the distance between the Y coordinates of the centers of gravity of the intervertebral regions is ΔYm, the threshold set here may be, for example, ΔYm / 4. However, the method for setting the threshold is not limited to this, and any method may be used. As an example of an exclusion method, of the two corresponding intervertebral regions, the intervertebral region with the larger area (H1 side in FIG. 5(a)) may be retained and the intervertebral region with the smaller area (H2 side in FIG. 5(a)) may be excluded. Alternatively, the intervertebral region on the lower side of the image may be retained and the intervertebral region on the upper side may be excluded. Conversely, the intervertebral region on the upper side may be retained and the intervertebral region on the lower side may be excluded.

[0039] Furthermore, if the distance between the Y coordinates of the centers of gravity of intervertebral regions is greater than a set threshold, as exemplified by coordinates H3 and H4 in FIG. 5(a), the second acquisition unit 122 may consider it an outlier (or an inference error) and add a new intervertebral region at the midpoint of those centers of gravity. As an example, if the median value of the distance between the Y coordinates of the centers of gravity of each intervertebral region is ΔYm, the threshold set here may be, for example, ΔYm×2. However, the method for setting the threshold is not limited to this and may be set by any method.

[0040] The intervertebral region added at this time may be a line segment with a width of 10 mm and a height of 1 mm, with its center of gravity located at the midpoint between the centers of gravity of the intervertebral regions at coordinates H3 and H4, as shown at the Y coordinate position of coordinate H3.5 in Fig. 5(b). However, the size and shape of the intervertebral region here are not limited to this and may be set to any size and shape.

[0041] Another example of the correction process will be described with reference to Figures 6(a) and 6(b). The second acquisition unit 122 calculates the vertical distance from the top end of the extracted spinal region (coordinate H1 in Figure 6(a)) to the Y coordinate of the center of gravity of the first intervertebral region counting inward from the top end of the spinal region toward the inside of the image (coordinate H2 in Figure 6(b)). If this distance is greater than a threshold, the second acquisition unit 122 may add an intervertebral region at the height position (coordinate H1.5 in Figure 6(b)) of the midpoint between the top end of the spinal region and the center of gravity of the first intervertebral region counting inward from the top end of the spinal region toward the inside of the image.

[0042] For example, the threshold value set here may be ΔYm×2, where ΔYm is the median value of the interval between the Y coordinates of the centers of gravity of each intervertebral region. The intervertebral region to be added here may be a line segment with a width of 10 mm and a height of 1 mm, with its center of gravity located at the midpoint between the midpoint of the spine region in the row at coordinate H1 and the center of gravity of the intervertebral region in coordinate H2, as shown at the Y coordinate position of coordinate H1.5 in FIG. 6(b). Note that rows refer to the horizontal arrangement of the image, and columns refer to the vertical arrangement of the image. However, the method for setting the threshold value and the intervertebral region to be added is not limited to this, and any method may be used. The second acquisition unit 122 may also simply set the center of gravity of the intervertebral region without setting the intervertebral region to be added as a line segment. If the spine region or intervertebral region extracted in step S12 is tilted, the second acquisition unit 122 may also tilt the intervertebral region to be added to match the tilt of the spine region or intervertebral region.

[0043] Similarly, the vertical distance from the lower end of the extracted spinal region (coordinate H4 in FIG. 6(a)) to the Y coordinate of the center of gravity of the first intervertebral region counting inward from the lower end of the spinal region (coordinate H3 in FIG. 6(b)) is calculated. If this distance is greater than a threshold, an intervertebral region may be added at the midpoint between the lower end of the spinal region and the center of gravity of the first intervertebral region counting inward from the lower end of the spinal region, as shown at the Y coordinate position of coordinate H3.5 in FIG. 6(b).

[0044] Regarding the intervertebral regions and their centers of gravity processed in this manner, the second acquisition unit 122 sets the center of gravity G1 of the fourth intervertebral region counting from the bottom (from the pelvis side) of the image, as shown in Fig. 7. Similarly, the second acquisition unit 122 sets the center of gravity G2 of the third intervertebral region counting from the bottom, the center of gravity G3 of the second intervertebral region counting from the bottom, and the center of gravity G4 of the first intervertebral region counting from the bottom.

[0045] Next, the second obtaining unit 122 sets horizontal lines LR1, LR2, LR3, and LR4 that pass through the centers of gravity G1, G2, G3, and G4, respectively (four dotted lines in FIG. 7).

[0046] If the number of extracted intervertebral regions is less than four (a predetermined number), the second acquisition unit 122 can draw four lines vertically dividing the extracted spinal region into five equal parts. The second acquisition unit 122 may extend these four lines to the left and right ends of the image and set them as horizontal lines LR1, LR2, LR3, and LR4, starting from the top of the image. This process allows the second acquisition unit 122 to divide the spinal region into four (a predetermined number). Furthermore, if this process is performed, the control unit 101 may display a message on the display unit 107, after extracting the bone density analysis region in the following process, indicating that the bone density analysis region may be misaligned and prompting the operator to manually correct it.

[0047] Then, as shown in FIG. 7, the second obtaining unit 122 sets horizontal reference regions SR1, SR2, and SR3 each having a vertical width and sandwiched between these horizontal lines.

[0048] Before performing this process, if the spine region extraction result in step S12 is tilted as shown in FIG. 8(a), the second acquisition unit 122 may perform rotation correction on the entire image. As an example of a method for setting the angle for rotation correction of the image, the second acquisition unit 122 determines the horizontal center of gravity position (black dots in FIG. 8(b)) for each row or thinned rows of the extracted spine region as shown in FIG. 8(b). Thereafter, the second acquisition unit 122 may perform rotation correction on the image so that the approximate line of the determined center of gravity position (dotted line in FIG. 8(b)) becomes a vertical line of the image as shown in FIG. 8(c). Note that the rotation correction may be performed before the correction process for the intervertebral regions, or before the process for setting the centers of gravity G1 to G4, etc.

[0049] Furthermore, instead of performing rotation correction on the image, the second acquisition unit 122 may set the lines LR1, LR2, LR3, and LR4 and the reference regions SR1, SR2, and SR3 obliquely based on the detected tilt of the spine region.

[0050] Furthermore, if the intervertebral region extracted in step S12 is tilted, the second acquisition unit 122 may draw the lines LR1, LR2, LR3, and LR4 diagonally. An example of a method for drawing the lines in this case will be described with reference to FIG. 9. FIG. 9 shows an example of an extracted intervertebral region 901, which is tilted. One example of a method for drawing lines in this intervertebral region 901 is to take the midpoints in the vertical direction of each column or thinned columns of the intervertebral region 901 (black dots in FIG. 9), and draw approximate straight lines between them (dotted lines in FIG. 9).

[0051] Another example of a method for drawing the straight lines is to draw an approximate straight line for the spinal region near the intervertebral region, as shown in FIG. 8(b), and then draw a straight line perpendicular to the approximate straight line and passing through the center of gravity of the intervertebral region. Furthermore, the second acquisition unit 122 may set reference regions SR1, SR2, and SR3, each of which has a vertical width and is parallel to the lines LR1, LR2, LR3, and LR4. The second acquisition unit 122 may also perform rotational correction on the image based on the inclination of the approximate straight line.

[0052] The second acquisition unit 122 extracts and acquires the spinal region (corresponding to the vertebral body) extracted in the reference region SR1 as the second lumbar vertebra L2, which is a bone density analysis region. Similarly, the second acquisition unit 122 extracts the spinal region in the reference region SR2 as the third lumbar vertebra L3, which is a bone density analysis region, and extracts the spinal region in the reference region SR3 as the fourth lumbar vertebra L4, which is a bone density analysis region.

[0053] In this way, the second acquisition unit 122 can extract and acquire the lumbar bone density analysis region with high accuracy by combining the extraction results of the spinal region and the intervertebral region. In other words, the second acquisition unit 122 is an example of a second acquisition means that acquires the bone density analysis region, which is the target region for lumbar bone density measurement, based on the acquisition results of the first acquisition unit 121.

[0054] Next, in step S13, the image processing device 120 sets a background region for calculating the transmittance of bones. An example of a method for setting the background region will be described below.

[0055] The background regions are set on the left and right of the second lumbar vertebra L2, the third lumbar vertebra L3, and the fourth lumbar vertebra L4 within the reference regions SR1, SR2, and SR3 set in step S12, as shown as background regions BG11, BG12, BG21, BG22, BG31, and BG32 in Fig. 10. An example of a method for searching these background regions will be described below.

[0056] The vertical widths of the background regions BG11, BG12, BG21, BG22, BG31, and BG32 can be set to the same value as the vertical widths of the respective reference regions SR1, SR2, and SR3. The horizontal widths of the background regions BG11, BG12, BG21, BG22, BG31, and BG32 can be set to a predetermined specified value. As an example, this width can be specified as 20 mm. The specified value of the horizontal width may be stored in advance in the storage unit 109. Alternatively, the specified value may be specified by the operator using the operation unit 108.

[0057] The position of the background region BG11 may be automatically selected in the region to the left of the second lumbar vertebra L2 in the reference region SR1 so that the variance of brightness values ​​within the background region is minimized. Similarly, the position of the background region BG12 may be automatically selected in the region to the right of the second lumbar vertebra L2 in the reference region SR1 so that the variance of brightness values ​​within the background region is minimized. Similarly, background regions BG21 and BG22 may be selected in the left and right regions of the third lumbar vertebra L3, and background regions BG31 and BG32 may be selected in the left and right regions of the fourth lumbar vertebra L4.

[0058] At this time, the spinous process region may be excluded so as not to be selected as a background region. For example, when searching for background region BG11, the image processing device 120 may exclude from the search range the range designated as the spinous process region to the left of the region extracted as the second lumbar vertebra L2. A range of, for example, 15 mm may be designated as the spinous process region. Similarly, when searching for background regions BG12, BG21, BG22, BG31, and BG32, the left and right spinous process regions of the second lumbar vertebra L2, the third lumbar vertebra L3, and the fourth lumbar vertebra L4 may be excluded.

[0059] The method for setting the background region is not limited to this, and any method may be used for setting the background region. For example, the operator may specify the background region using the operation unit 108.

[0060] The control unit 101 may display the bone density analysis region and background region extracted as described above on the display unit 107. Furthermore, the operator may use the operation unit 108 to modify and determine the bone density analysis region and background region based on the bone density analysis region and background region displayed on the display unit 107. Similarly, the operator may also use the operation unit 108 to modify and determine the center of gravity and the straight line that appear in the above calculation process. Furthermore, the background region does not have to be displayed on the display unit 107.

[0061] The image processing device 120 calculates the bone density in each bone density analysis region by referring to the bone density analysis region and the background region. The bone density can be calculated by a known method, for example, by using information on the thickness of bones and soft tissues in the image acquired by DXA. Alternatively, the bone density may be calculated by using information on atomic number and surface density acquired by DXA. The control unit 101 may display the calculated bone density on the display unit 107.

[0062] Furthermore, the control unit 101 may cause the display unit 107 to display any value useful for diagnosis based on the calculated bone density. For example, the control unit 101 may cause the display unit 107 to display a value compared with the average bone density of people of the same age or other age groups. The control unit 101 may also cause the display unit 107 to display time-series data compared with, for example, past measurement results. The control unit 101 may also cause the display unit 107 to display, for example, the average value of the bone densities at the second lumbar vertebra L2, the third lumbar vertebra L3, and the fourth lumbar vertebra L4 as the final bone density measurement value.

[0063] Furthermore, the control unit 101 may control the display unit 107 to display the acquisition results or acquisition process of the bone density analysis region to the operator. Here, the acquisition process includes each process by the second acquisition unit 122 described above, and the display of the acquisition process may display the region, center of gravity, line, etc. acquired and set in each process. For example, the acquired spinal region or intervertebral region, or the reference region or background region used when extracting the bone density analysis region may be displayed. Furthermore, the operation unit 108 can function as an example of an operation means that the operator operates while viewing the acquisition results or acquisition process by the second acquisition unit 122.

[0064] Example 2 Next, a second embodiment of the present invention will be described. An example of the configuration of an X-ray imaging system according to this embodiment is the same as the example of the configuration of the X-ray imaging system 100 according to the first embodiment, and is shown in FIG. 1. Note that the example of the configuration of the X-ray imaging system according to this embodiment will be described by using the same reference numerals as the example of the configuration of the X-ray imaging system 100 according to the first embodiment, and description thereof will be omitted. As in the first embodiment, the image processing device 120 according to this embodiment extracts a bone density analysis region using an image such as that shown in FIG. 2 as an input image. Hereinafter, this embodiment will be described, focusing on the differences from the first embodiment.

[0065] 11 shows the image processing flow of the image processing device 120 for extracting a bone density analysis region in this embodiment. When image processing according to this embodiment starts, the process proceeds to step S21.

[0066] In step S21, the first acquisition unit 121 extracts (segments) a spine region from the input image using the trained model. An example of the extracted result is shown in FIG. 4(a) as in Example 1. Note that the trained model here can be generated by training a combination of an image of the vicinity of the lumbar vertebrae and an image in which the spine region within the image is labeled, as training data.

[0067] In step S22, the image processing device 120 generates a spine image by extracting the region extracted in step S21 from the input image. An example of the spine image is shown in Fig. 12. This process can be performed by applying a mask to the input image, in which the region extracted in step S21 is set to 1 and other regions are set to 0.

[0068] In step S23, the first acquisition unit 121 extracts (segments) intervertebral regions from the spine image using the trained model. An example of the extraction result is the same as that shown in FIG. 4(b). That is, the image processing device 120 according to this embodiment extracts intervertebral regions from the inside of the spine, not from the entire image. Note that the trained model here can be generated by training a combination of a spine image and an image in which the intervertebral regions within the spine image are labeled, as training data.

[0069] In step S24, the second acquisition unit 122 combines the extraction results of the extracted spinal region and the intervertebral region to extract a lumbar bone density analysis region in the same manner as in Example 1. Even in this case, by combining the extraction results of the spinal region and the intervertebral region, it is possible to extract and acquire a lumbar bone density analysis region with high accuracy. Furthermore, by acquiring the intervertebral region from the spine image, the trained model narrows the region where processing using image features is performed, which is expected to reduce the amount of calculation, speed up processing, and reduce the processing load.

[0070] In step S25, the image processing device 120 extracts the background region in the same manner as in step S13 of the first embodiment.

[0071] In the bone density analysis region and background region extracted as described above, the image processing device 120 according to this embodiment calculates the bone density in each bone density analysis region in the same manner as in the first embodiment.

[0072] Example 3 Next, a third embodiment of the present invention will be described. An example of the configuration of an X-ray imaging system according to this embodiment is the same as the example of the configuration of the X-ray imaging system 100 according to the first embodiment, and is shown in FIG. 1. Note that the example of the configuration of the X-ray imaging system according to this embodiment will be described by using the same reference numerals as the example of the configuration of the X-ray imaging system 100 according to the first embodiment, and description thereof will be omitted. As in the first embodiment, the image processing device 120 according to this embodiment extracts a bone density analysis region using an image such as that shown in FIG. 2 as an input image. Hereinafter, this embodiment will be described, focusing on the differences from the first embodiment.

[0073] 13 shows the image processing flow of the image processing device 120 for extracting a bone density analysis region in this embodiment. When image processing according to this embodiment starts, the process proceeds to step S31.

[0074] In step S31, the first acquisition unit 121 extracts (segments) the spine region, intervertebral region, and pelvis region from the input image using the trained model. Examples of the extraction results are shown in Figures 14(a) to 14(c). Figure 14(a) shows an example of the spine region extraction result, Figure 14(b) shows an example of the intervertebral region extraction result, and Figure 14(c) shows an example of the pelvis region extraction result.

[0075] The trained models may be stored in the storage unit 109 and may be recalled for use. A trained model for extracting the spine region, a trained model for extracting the intervertebral region, and a trained model for extracting the pelvic region may be provided separately. These trained models may have the same structure, with only the weights differing. Alternatively, the trained model here may be a common model, and extraction results for the spine region, intervertebral region, and pelvic region may be output from each channel.

[0076] The trained model here can be generated, for example, by training a combination of an image near the lumbar vertebrae and an image within the image labeled with the spinal region, intervertebral region, and pelvic region as training data. Note that if a trained model for extracting the spinal region, a trained model for extracting the intervertebral region, and a trained model for extracting the pelvic region are separately provided, the labeled images can be trained separately. Specifically, a trained model for extracting the pelvic region can be generated by training a combination of an image near the lumbar vertebrae and an image within the image labeled with the pelvic region as training data. Note that the trained model for extracting the spinal region and the trained model for extracting the intervertebral region can be generated in the same manner as in Example 1.

[0077] The second acquisition unit 122 may perform a correction process for the extraction results based on these extraction results. One example of the correction process for the extraction results may be a process of extending the lower end of the spinal region so that it reaches the pelvic region when the extracted spinal region and pelvic region are separated by a predetermined distance or more. An example of the predetermined distance set here may be 10 mm.

[0078] An example of a process for correcting the extraction result may be a process for shortening the lower end of the extracted spinal region to the upper boundary of the pelvic region when the lower end of the extracted spinal region is inside the pelvic region.

[0079] An example of the process for correcting the extraction results may include a process in which the control unit 101 displays a warning message on the display unit 107 when the extracted intervertebral region and pelvic region are separated by a predetermined distance or more. The warning message may be, for example, a message indicating that the bone density analysis region may be misaligned, that an intervertebral region may have been omitted from extraction, or a message prompting the operator to make manual corrections. An example of the predetermined distance set here may be 10 mm.

[0080] Furthermore, the second acquisition unit 122 may determine that the extracted intervertebral region and the pelvic region are separated by a predetermined distance or more, possibly resulting in an intervertebral region being omitted from extraction. In this case, the second acquisition unit 122 may add a new intervertebral region at a midpoint between the first intervertebral region counting up from the pelvic region and the upper boundary line of the pelvic region. The intervertebral region added at this time may be a line segment with a width of 10 mm and a height of 1 mm, as in the example shown in FIG. 5(b). An example of the center of gravity of the added intervertebral region may be the midpoint between the center of gravity of the first intervertebral region counting up from the pelvic region and a point on the upper boundary line of the pelvic region whose horizontal position (X coordinate) is the X coordinate of the center of gravity of the pelvic region.

[0081] An example of a correction process for the extraction results may be a process for removing an extracted intervertebral region if the intervertebral region is located inside the pelvic region.

[0082] As described above, by referring to the extraction results of the pelvic region, the influence of segmentation errors in the spinal region and intervertebral region can be reduced.

[0083] Next, in step S32, the second acquisition unit 122 combines the extraction result of the extracted spinal region and the extraction result of the intervertebral region to extract a lumbar bone mineral density analysis region in the same manner as in Example 1. Even in this case, by combining the extraction result of the spinal region and the extraction result of the intervertebral region, it is possible to extract and acquire a lumbar bone mineral density analysis region with high accuracy.

[0084] In step S33, the image processing device 120 extracts the background region in the same manner as in step S13 in the first embodiment.

[0085] In the bone density analysis region and background region extracted as described above, the image processing device 120 according to this embodiment calculates the bone density in each bone density analysis region in the same manner as in the first embodiment.

[0086] Example 4 Next, a fourth embodiment of the present invention will be described. An example of the configuration of an X-ray imaging system according to this embodiment is the same as the example of the configuration of the X-ray imaging system 100 according to the first embodiment, and is shown in FIG. 1. Note that the example of the configuration of the X-ray imaging system according to this embodiment will be described by using the same reference numerals as the example of the configuration of the X-ray imaging system 100 according to the first embodiment, and description thereof will be omitted. As in the first embodiment, the image processing device 120 according to this embodiment extracts a bone density analysis region using an image such as that shown in FIG. 2 as an input image. Hereinafter, this embodiment will be described, focusing on the differences from the first embodiment.

[0087] 15 shows the image processing flow of the image processing device 120 for extracting a bone density analysis region in this embodiment. When image processing according to this embodiment starts, the process proceeds to step S41.

[0088] In step S41, the first acquisition unit 121 extracts (segments) a spine region, an intervertebral region, and a rib region from the input image using the trained model. Examples of the extraction results are shown in Figures 16(a) to 16(c). Figure 16(a) shows an example of the spine region extraction result, Figure 16(b) shows an example of the intervertebral region extraction result, and Figure 16(c) shows an example of the rib region extraction result.

[0089] The trained models may be stored in the storage unit 109 and may be recalled for use. A trained model for extracting the spine region, a trained model for extracting the intervertebral region, and a trained model for extracting the rib region may be provided separately. These trained models may have the same structure, with only the weights differing. Alternatively, the trained model here may be a common model, and extraction results for the spine region, intervertebral region, and rib region may be output from each channel.

[0090] The trained model here can be generated, for example, by training a combination of an image near the lumbar vertebrae and an image within the image labeled with the spine region, intervertebral region, and rib region as training data. Note that if a trained model for extracting the spine region, a trained model for extracting the intervertebral region, and a trained model for extracting the rib region are separately provided, the labeled images can be trained separately. Specifically, a trained model for extracting the rib region can be generated by training a combination of an image near the lumbar vertebrae and an image within the image labeled with the rib region as training data. Note that the trained model for extracting the spine region and the trained model for extracting the intervertebral region can be generated in the same manner as in Example 1.

[0091] Alternatively, the sternum region may be extracted instead of or in addition to the rib region. In this case, for example, images labeled with the sternum region but not the pelvis region may be used as training data, or images labeled with the spine region, intervertebral region, pelvis region, and sternum region may be used.

[0092] Next, in step S42, the second acquisition unit 122 extracts a bone density analysis region from the extraction result of step S41. Here, a case where the second lumbar vertebra L2 to the fourth lumbar vertebra L4 are extracted as the bone density analysis region will be described. Note that the combination of bone density analysis regions is not limited to this, and for example, the first lumbar vertebra L1 may also be extracted.

[0093] The second acquisition unit 122 extracts regions (vertebral bodies) separated by intervertebral spaces from the extracted spinal region. At this time, the second acquisition unit 122 regards the vertebral bodies adjacent to the rib region as thoracic vertebrae. Then, as shown in FIG. 2, the second acquisition unit 122 extracts the vertebral body one below the thoracic vertebra as the first lumbar vertebra L1. Furthermore, the second acquisition unit 122 extracts the vertebral bodies below the first lumbar vertebra L1 as the second lumbar vertebra L2, the third lumbar vertebra L3, and the fourth lumbar vertebra L4, in that order.

[0094] Furthermore, similar to the first embodiment, the second acquiring unit 122 may set corresponding reference regions SR1, SR2, and SR3 from the intervertebral regions above and below the second lumbar vertebra L2, the third lumbar vertebra L3, and the fourth lumbar vertebra L4. Further, similar to the first embodiment, the second acquiring unit 122 may set the spinal region within these reference regions as the final bone density analysis region (the second lumbar vertebra L2, the third lumbar vertebra L3, and the fourth lumbar vertebra L4).

[0095] Similarly to the first embodiment, the second acquiring unit 122 may extract the second to fourth vertebrae counting from the bottom (from the pelvis side) as the fourth lumbar vertebra L4, the third lumbar vertebra L3, and the second lumbar vertebra L2. In this case, the second acquiring unit 122 may extract the fifth vertebra counting from the bottom as the first lumbar vertebra L1, and perform correction processing by comparing this with the region extracted as the ribs.

[0096] An example of the correction process here may include a process in which the control unit 101 displays on the display unit 107 a message indicating that the extraction result of the bone density analysis region may be misaligned when the first lumbar vertebra L1 is in contact with the rib region, and a message prompting the operator to make manual corrections.

[0097] Furthermore, an example of the correction process here may include a process in which the control unit 101 displays on the display unit 107 a message indicating that the extraction result of the bone density analysis region may be inaccurate when the vertebral body one level above the first lumbar vertebra L1 is not in contact with the rib region, and a message prompting the operator to make manual corrections.

[0098] Furthermore, if the vertical position of the upper end of the extracted spine region is a predetermined distance below the lower end of the extracted rib region, the image processing device 120 may perform a correction process to extend the upper end of the spine region to a predetermined distance below the lower end of the rib region. An example of the predetermined distance here may be set to 5 mm.

[0099] As described above, by referring to the rib extraction results, it is possible to reduce the influence of segmentation errors in the spinal and intervertebral regions on the extraction of lumbar vertebrae. Furthermore, by combining the extraction results of the spinal region and the intervertebral region, it is possible to extract and obtain the lumbar bone density analysis region with high accuracy.

[0100] In step S43, the image processing device 120 extracts the background region in the same manner as in step S13 of the first embodiment.

[0101] In the bone density analysis region and background region extracted as described above, the image processing device 120 according to this embodiment calculates the bone density in each bone density analysis region in the same manner as in the first embodiment.

[0102] (Variation) The following describes a modified example of at least one of Example 1, Example 2, Example 3, and Example 4 of the present invention. The image processing device 120 may extract, as an implant region, a region of an object that is neither bone nor soft tissue, such as metal. The implant region may be extracted by segmentation using a trained model based on machine learning or rule-based segmentation based on the variance of brightness values, etc. The extracted implant region may be excluded from the analysis target during bone density analysis. Note that the trained model may be generated by training, for example, a combination of an image of the vicinity of the lumbar vertebrae and an image in which regions of an object that is neither bone nor soft tissue, such as metal, within the image are labeled.

[0103] Furthermore, by combining Example 3 and Example 4, in addition to the spine region and intervertebral region, both the pelvic region and the rib region may be extracted, and the results of these may be combined to improve the extraction accuracy of the bone density analysis region. For example, if the vertical position of the lower end of the extracted spine region is higher than the upper end of the pelvic region by more than a threshold, the second acquisition unit 122 may extend the vertical position of the lower end of the spine region to the vertical position of the upper end of the pelvic region. Also, if the vertical position of the upper end of the extracted spine region is lower than the lower end of the rib region by more than a threshold, the second acquisition unit 122 may extend the vertical position of the upper end of the spine region to the vertical position of the lower end of the rib region. An example of the threshold here may be set to 10 mm. However, the threshold value is not limited to this, and any value may be set.

[0104] Furthermore, the second acquisition unit 122 may divide the lumbar region set in this manner into six equal parts vertically, and set the fourth region from the bottom (from the pelvis side) as the second lumbar vertebra L2, the third region from the bottom as the third lumbar vertebra L3, and the second region from the bottom as the fourth lumbar vertebra L4.

[0105] The first acquiring unit 121 may acquire an outer frame region obtained by fusing at least one of the pelvic region and the rib region with the spinal region. Furthermore, the second acquiring unit 122 may use the outer frame region instead of the spinal region and combine it with the extraction result of the intervertebral region in the same manner as above to extract the bone density analysis region.

[0106] The image processing device 120 may also extract the bone density analysis region using a total of three images, including two images acquired by DXA using different radiation energies and an energy-sub-image generated from these images. For example, when extracting the spine region and the intervertebral region using a trained model (e.g., step S11 in Example 1), the first acquisition unit 121 may extract the spine region and the intervertebral region from three images. In this case, the second acquisition unit 122 may use only the regions where the extraction results overlap as the final spine region and the intervertebral region. Furthermore, the second acquisition unit 122 may use only the regions extracted as the spine region and the intervertebral region in two or more of the three images as the final spine region and the intervertebral region.

[0107] In this case, the X-ray imaging system 100 may have individual trained models for the three images. In this case, each trained model can be generated by training the images corresponding to each type of the three images (first and second images related to different radiation energies, and energy-sub-images generated from these images) as input data for the respective training data. Note that images labeled with the spinal region and intervertebral region for each type of image can be used as output data for the training data. Furthermore, each input channel of one trained model may correspond to these three images. Similarly, the image processing device 120 may extract the spinal region and intervertebral region using two of these three images.

[0108] By performing the above correction process, the influence of segmentation errors can be reduced.

[0109] In the above-described first to fourth embodiments, the image processing device 120 acquires a spinal region and an intervertebral region from an input image of the lumbar vertebrae using a trained model, and acquires a bone density analysis region using the acquired spinal region and intervertebral region. However, the image processing according to the present disclosure can also be applied to other regions.

[0110] The missing regions and the acquisition of redundant regions that occur in segmentation using a trained model described above are thought to be due to the presence of a series of similar structures in the region to be acquired. Therefore, for example, when performing region extraction on the cervical vertebrae, thoracic vertebrae, or other regions where similar structures are continuously connected, a similar situation may occur. Even in such cases, applying the image processing disclosed herein can correct the missing regions and the acquired redundant regions, ultimately enabling the analysis region to be acquired with high accuracy.

[0111] As in the above-described first to fourth embodiments, the correction process may be performed by using the relationship between the acquired first region and a larger second region including the first region, or the relationship between multiple regions within the first region. For example, the acquired first region, which is the segmentation result using the trained model, may be corrected using the distance between the centers of gravity of the first regions or the distance between the center of gravity of the first region and the edge of the second region.

[0112] In the above-described first to fourth embodiments, the lumbar region between the intervertebral regions is set as the region to be finally acquired (analysis region). However, the analysis region to be finally acquired may be a region acquired by segmentation using a trained model by the first acquisition unit 121 and corrected by the second acquisition unit 122.

[0113] Example 5 17 to 19, a fifth embodiment of the present disclosure will be described below. In the first to fourth embodiments, the technology of the present disclosure is applied to an X-ray imaging system that captures medical images. In contrast, in this embodiment, the technology of the present disclosure is applied to a radiography system used in in-line automatic inspection.

[0114] In-line automated inspections widely use techniques that use tomographic images reconstructed from multiple projection images captured using X-rays. For example, in-line automated inspections, it is desirable to perform the inspection while the object being inspected (e.g., a flat substrate) is held close to the X-ray source for magnification. However, if X-rays are irradiated while the X-ray source is held close to the thickness of the substrate, the X-rays do not penetrate well in the width direction of the substrate, which is longer than its thickness, and the desired inspection results may not be obtained. In response to this issue, techniques have been proposed that irradiate the object with X-rays at an oblique angle (e.g., techniques known as oblique CT, lamino CT, or planar CT). This technique allows the object to be brought closer to the X-ray source, making it easier to adjust the magnification ratio and enabling the inspection equipment to be more compact.

[0115] Here, a plurality of projection images taken by irradiating such an inspection object with X-rays at an oblique angle and a tomographic image reconstructed from the projection images can be used as input to a trained model to acquire a target region for analyzing and detecting the presence of a material such as solder. However, even in such a case, since similar structures such as solder are continuously connected, when segmentation is performed using a machine learning model, there is a risk that some regions to be acquired will be omitted, resulting in an incorrectly acquired analysis region.

[0116] Therefore, one of the objectives of this embodiment is to accurately acquire an analysis region in which similar structures are continuously connected. In relation to this, this embodiment uses a trained model to perform segmentation on multiple projection images captured by irradiating the object to be inspected obliquely with radiation and on tomographic images reconstructed from the projection images. Furthermore, through segmentation, a solder region (first region) and a larger chip region (second region) including the solder region are acquired, and the solder region acquired through segmentation is corrected using the relationship between the solder region and the chip region and the relationship between regions within the solder region.

[0117] First, the configuration of a radiation imaging system according to this embodiment will be described with reference to Fig. 17. Fig. 17 shows an example of the overall configuration of a radiation imaging system according to this embodiment. A radiation imaging system 1700 is provided with a control unit 1710, a radiation generating device 1701, a stage 1706, a radiation imaging device 1704, a robot arm 1705, and an imaging device support unit 1703. Note that the configurations of the radiation generating device 1701 and the radiation imaging device 1704 may be similar to the configurations of the X-ray tube 103 and FPD 106 according to Example 1, and therefore description thereof will be omitted.

[0118] Here, the radiation imaging device 1704 is supported by an imaging device support unit 1703, and is configured so that the radiation imaging device 1704 can also be moved by movement of the imaging device support unit 1703 and the robot arm 1705. An object to be inspected (hereinafter also referred to as "workpiece 1702") is placed on a stage 1706. The stage 1706 is configured to move to a position designated for radiation imaging or to stop at a predetermined position designated for radiation imaging in response to a control signal from a stage control unit 1718 of the control unit 1710.

[0119] The object to be inspected may include, for example, various articles and the human body. When various articles (for example, a circuit board) are inspected, the method can be applied to determining whether electronic components are mounted on the board or to analyzing the inside of the object to be inspected. When the object to be inspected is a human body, the method can be applied to diagnostic imaging using tomography.

[0120] The control unit 1710 is provided with a first acquisition unit 1711, a generation unit 1712, a second acquisition unit 1713, a third acquisition unit 1714, a display control unit 1715, a storage unit 1716, an imaging device control unit 1717, a stage control unit 1718, and a radiation control unit 1719. The first acquisition unit 1711 can acquire images captured by the radiation imaging device 1704 and images generated by the generation unit 1712. The first acquisition unit 1711 can also acquire various images from an external device (not shown) connected to the control unit 1710 via a network such as the Internet.

[0121] The generating unit 1712 can reconstruct a three-dimensional image from a plurality of radiation images (projection images) acquired by the acquiring unit. The generating unit 1712 can also reconstruct a tomographic image of an arbitrary cross section from the generated three-dimensional image. Details of the processing by the generating unit 1712 will be described later.

[0122] The second acquisition unit 1713 extracts and acquires chip regions and solder regions from the input image using a trained model for the tomographic image reconstructed by the generation unit 1712, similar to the first acquisition unit 121 of the image processing device 120 according to the first embodiment. The third acquisition unit 1714 acquires an analysis region, which is a target region for analyzing the presence or absence of solder, using the chip regions and solder regions acquired by the second acquisition unit 1713. Details of the processing by the second acquisition unit 1713 and the third acquisition unit 1714 will be described later. Note that, hereinafter, the acquisition processing of the solder regions, chip regions, and analysis regions includes the acquisition of images showing these regions and the identification of these regions.

[0123] In this embodiment, the second acquisition unit 1713 and the third acquisition unit 1714 are realized as functional components within the control unit 1710. However, similar to the first embodiment, a device (image processing device) separate from the control unit 1710 may be provided, and the second acquisition unit 1713 and the third acquisition unit 1714 may be configured within the separate device.

[0124] The control unit 1710 can analyze the presence of solder in the acquired analysis area and determine whether the electronic components are properly attached. If a separate image processing device is provided, the image processing device can also perform the analysis. Any known method can be used to analyze the presence of solder.

[0125] The display control unit 1715 controls the display of the display unit 1720. The display control unit 1715 can display, for example, various images generated by the generation unit 1712, each region acquired by the second acquisition unit 1713, analysis results of the analysis region acquired by the third acquisition unit 1714, patient information, etc. on the display unit 1720. The storage unit 1716 can function in the same way as the storage unit 109 according to the first embodiment.

[0126] Here, the control unit 1710 can be configured by a computer provided with a processor and a memory. The control unit 1710 may be configured by a general computer or a computer dedicated to the radiation control system. The control unit 1710 may be, for example, a personal computer, and a desktop PC, a notebook PC, a tablet PC (portable information terminal), or the like may be used. Furthermore, the control unit 1710 may be configured as a cloud-based computer in which some of the components are located in an external device.

[0127] Furthermore, each component of the control unit 1710 other than the storage unit 1716 may be configured by a software module executed by a processor such as a CPU or MPU. The processor may be, for example, a GPU or FPGA. Each component may be configured by a circuit that performs a specific function, such as an ASIC. The storage unit 1716 may be configured by any storage medium, such as an optical disk such as a hard disk, or a memory.

[0128] The control unit 1710 is also connected to a display unit 1720 and an operation unit 1750. The display unit 1720 and the operation unit 1750 may be the same as the display unit 107 and the operation unit 108 according to the first embodiment, and therefore a description thereof will be omitted.

[0129] The radiation control unit 1719 can function in the same manner as the X-ray generator control unit 102 according to Example 1. The radiation control unit 1719 can control imaging conditions such as the irradiation angle of radiation emitted by the radiation generating device 1701, the radiation focal position, the tube voltage, and the tube current, based on operations by an operator via the operation unit 1750, for example.

[0130] The radiation generating device 1701 outputs radiation with an axis passing through the radiation focal point as the central axis based on a control signal from the radiation control unit 1719. The radiation generating device 1701 can be configured as a radiation generating device that is movable in, for example, the X, Y, Z, and Φ directions. In this case, the radiation generating device 1701 includes a driving unit such as a motor, and can move to any position within a plane (within the XY plane) intersecting the rotation axis (Z axis) or stop at any position (for example, the position of the rotation axis (Z axis)) based on a control signal from the radiation control unit 1719. The radiation generating device 1701 irradiates radiation from a direction tilted with respect to the rotation axis while moving within a plane intersecting the rotation axis or while stopped at the position of the rotation axis.

[0131] In FIG. 17, the rotation axis is the axis (Z axis) in the vertical direction of the paper, and the angle θ indicates the tilt angle with respect to the rotation axis (Z axis). The angle Φ indicates the rotation angle around the Z axis. The X direction corresponds to, for example, the left-right direction of the paper, and the Y direction corresponds to the direction perpendicular to the paper. The Z direction corresponds to, for example, the up-down direction of the paper. The coordinate system setting in FIG. 17 is the same in FIGS. 18A and 18B.

[0132] The state in which the radiation generating device 1701 moves within a plane intersecting the rotation axis refers to a state in which the radiation generating device 1701 moves within a plane intersecting the rotation axis (Z axis) (within the XY plane) along a predetermined trajectory 1840, as shown in Fig. 18A, for example. The state in which the radiation generating device 1701 stops at the position of the rotation axis refers to a state in which the radiation generating device 1701 stops at the position of the rotation axis (Z axis), as shown in Fig. 18B, for example. The radiation irradiation from a direction tilted with respect to the rotation axis refers to radiation irradiation in a state tilted at an angle θ with respect to the rotation axis (Z axis), as shown in Figs. 18A and 18B, for example.

[0133] In the radiation imaging system shown in FIG. 18A, a radiation generating device 1701 is configured to be movable within a plane (XY plane) intersecting a rotation axis (Z axis) and irradiates radiation from a direction inclined with respect to the rotation axis. A stage 1706 (holding unit) that holds a workpiece 1702 holds the workpiece 1702 while stopped at the position of the rotation axis (Z axis). A radiation imaging device 1704 is configured to be movable within a plane (XY plane) intersecting the rotation axis (Z axis) and detects radiation that has passed through an object to be inspected. In FIG. 18A, based on a control signal from a radiation control unit 1719, the radiation generating device 1701 outputs radiation from a position 1820 of the radiation focal point of the radiation generating device 1701, with an axis 1820A passing through the radiation focal point as the central axis. Similarly, based on a control signal from the radiation control unit 1719, the radiation generating device 1701 outputs radiation from a position 1821 of the radiation focal point of the radiation generating device 1701, with an axis 1821A passing through the radiation focal point as the central axis. Here, the angle formed by axis 1820A (axis 1821A) and the rotation axis (Z axis) is the tilt angle (angle θ).

[0134] On the other hand, in the radiation imaging system shown in FIG. 18B, a radiation generating device 1701 is stopped at the position of the rotation axis (Z axis) and irradiates radiation from a direction tilted with respect to the rotation axis. A stage 1706 that holds a workpiece 1702 is configured to be movable within a plane (XY plane) that intersects with the rotation axis, and holds the workpiece 1702. A radiation imaging device 1704 is also configured to be movable within a plane (XY plane) that intersects with the rotation axis (Z axis), and detects radiation that has passed through the object to be inspected. In FIG. 18B, the radiation generating device 1701 outputs radiation from a position 1820 (the position of the rotation axis (Z axis)) of the radiation focal point of the radiation generating device 1701, with an axis 1820B passing through the radiation focal point as its central axis, based on a control signal from a radiation control unit 1719. The radiation generating device 1701 also changes the radiation irradiation angle based on a control signal from the radiation control unit 1719, and outputs radiation from the position 1820 of the radiation focal point of the radiation generating device 1701 with an axis 1820C passing through the radiation focal point as the central axis. Here, the angle formed by the axis 1820B (axis 1820C) and the rotation axis (Z axis) is the tilt angle (angle θ).

[0135] The stage control unit 1718 controls the position of the stage 1706 so as to move the stage 1706 to a position designated for radiography and stop it at a predetermined position for radiography. The stage control unit 1718 can control the position of the stage 1706 based on a program for a predetermined radiography operation or an operation by an operator.

[0136] A workpiece 1702, which is an object to be inspected, is held on the stage 1706. The stage 1706 is configured as a stage that is movable in, for example, the X, Y, Z, and Φ directions. The stage 1706 includes a drive unit such as a motor, and can move to any position within a plane (within the XY plane) that intersects with the rotation axis (Z axis) and stop at any position (for example, the position of the rotation axis (Z axis)) based on a control signal from a stage control unit 1718. The stage 1706 holds the workpiece 1702 and functions as a holder configured to be movable within a plane (within the XY plane) that intersects with the rotation axis (Z axis).

[0137] Stage 1706 is configured to be movable, for example, along a trajectory 1860 ( FIG. 18B ) in the Φ direction around a rotation axis (Z axis) or a linear trajectory in the XY plane. Stage 1706 can be positioned and stopped at a predetermined position in the XY plane based on a control signal from stage control unit 1718. In other cases, stage 1706 can be configured to move in one direction using a belt conveyor or the like to place workpiece 1702 in a position for inspection.

[0138] The imaging apparatus control unit 1717 controls the position and operation of the radiation imaging apparatus 1704. The imaging apparatus control unit 1717 also controls the movement positions of the robot arm 1705 and the imaging apparatus support unit 1703. The robot arm 1705 and the imaging apparatus support unit 1703 can move the radiation imaging apparatus 1704 to a specified position by moving to predetermined positions based on control signals from the imaging apparatus control unit 1717. For example, the robot arm 1705 and the imaging apparatus support unit 1703 can be configured as an operating mechanism that moves the radiation imaging apparatus 1704 with degrees of freedom in the X and Y directions and degrees of freedom in the rotational direction (Φ) around the Z axis (degrees of freedom in the X, Y, and Φ directions).

[0139] The radiation imaging device 1704 is held at a predetermined position on the imaging device support unit 1703. The imaging device control unit 1717 acquires position information of the imaging device control unit 1717 based on the movement positions of the robot arm 1705 and the imaging device support unit 1703. The imaging device control unit 1717 transmits the position information and rotation angle information of the radiation imaging device 1704 acquired based on the movement positions and rotation angles of the robot arm 1705 and the imaging device support unit 1703 to the generation unit 1712.

[0140] The radiation imaging device 1704 detects radiation output by the radiation generating device 1701 and transmitted through the workpiece 1702, and sends image information of a projected image of the workpiece 1702 to the control unit 1710. The radiation imaging device 1704 is configured to be movable within a plane intersecting the rotation axis (Z axis) by the operation of the robot arm 1705, which has degrees of freedom in the X, Y, and Φ directions, and the imaging device support unit 1703, and detects radiation that has transmitted through the inspection object (workpiece 1702). Here, movement within a plane intersecting the rotation axis refers to a state in which the radiation imaging device 1704 moves along a predetermined trajectory 1850 within a plane (within the XY plane) intersecting the rotation axis (Z axis), as shown in, for example, FIGS. 18A and 18B .

[0141] 18A and 18B, the radiation imaging process according to this embodiment will be described. In the radiation imaging process according to this embodiment, in order to generate a three-dimensional image of a workpiece 1702, which is an object to be inspected, radiation is irradiated obliquely onto the workpiece 1702 while changing the imaging position of the workpiece 1702, and a plurality of projection images are captured.

[0142] 18A and 18B are diagrams for explaining an example of radiation imaging processing according to this embodiment. FIG. 18A shows an example of radiation imaging processing in a state where a radiation generating device 1701 moves along a predetermined trajectory 1840 within a plane (in the XY plane) intersecting the rotation axis (Z axis). On the other hand, FIG. 18B shows an example of radiation imaging processing in a state where the radiation generating device 1701 is stopped at the position of the rotation axis (Z axis). Note that the radiation imaging processing according to this embodiment is not limited to the configurations shown in FIGS. 18A and 18B. The radiation imaging processing according to this embodiment only requires that at least two of the radiation generating device 1701, the stage 1706 that holds the object to be inspected, and the radiation imaging device 1704 are configured to be movable (for example, rotatable in conjunction with each other) within a plane intersecting the rotation axis. Note that at least two of the above may be configured to be movable within a plane intersecting the rotation axis so as to satisfy a positional relationship in which radiation irradiated from the radiation generating device 1701 passes through the object to be inspected in a direction inclined with respect to the rotation axis and can be detected by the radiation imaging device 1704. For example, the radiation generating device 1701, the stage 1706, and the radiation imaging device 1704 may be configured to be movable within a plane intersecting the rotation axis so as to satisfy the above positional relationship. Furthermore, for example, the radiation generating device 1701 and the stage 1706 may be configured to be movable within a plane intersecting the rotation axis while the radiation imaging device 1704 is stopped at the position of the rotation axis so as to satisfy the above positional relationship.

[0143] Next, image generation processing and the like in the control unit 1710 according to this embodiment will be described. The first acquisition unit 1711 acquires a radiographic image (projected image) sent from the radiation imaging device 1704 and sends it to the generation unit 1712. The first acquisition unit 1711 can also acquire generated three-dimensional images and tomographic images, which will be described later. The first acquisition unit 1711 may also acquire this image information and various images from an external device connected to the control unit 1710.

[0144] The radiographic operation according to this embodiment is performed for each imaging position of the workpiece 1702 by irradiating the workpiece 1702 with radiation at an angle as described above in order to reconstruct a three-dimensional image from the projection images. The generation unit 1712 can reconstruct a three-dimensional image from the multiple projection images thus captured. More specifically, the generation unit 1712 performs reconstruction processing using position information and rotation angle information of the radiographic apparatus 1704 received from the radiographic apparatus control unit 1717 and the projection images of the workpiece 1702 captured by the radiographic apparatus 1704, to generate a three-dimensional image. The generation unit 1712 can also reconstruct three-dimensional images of different energies using projection images based on radiation of different energies, such as images captured by X-rays at two different tube voltages using the kV switching method described above.

[0145] The generating unit 1712 can also reconstruct a tomographic image of any cross section from the generated three-dimensional image. Note that any known method may be used to reconstruct the three-dimensional image and the tomographic image. The cross section from which the tomographic image is extracted from the three-dimensional image may be set based on a predetermined initial setting, or may be set in response to an instruction from the operator. The control unit 1710 may also automatically set the cross section based on the detection result of the state of the object to be inspected detected based on the projected image or information from various sensors (not shown), or may also automatically set the cross section in response to the selection of the inspection purpose based on the operator's operation, etc. In this embodiment, the generating unit 1712 reconstructs tomographic images for, for example, three cross sections: an XY cross section, a YZ cross section, and an XZ cross section.

[0146] The second acquisition unit 1713 uses the reconstructed tomographic image as input data for the trained model to perform segmentation of the tomographic image. As a result of the segmentation, the second acquisition unit 1713 can extract and acquire solder areas in the tomographic image where solder is applied, and chip areas that include the solder areas and where a chip such as an IC chip is applied.

[0147] Here, as the training data for the trained model according to this embodiment, a reconstructed tomographic image may be used as input data, and an image of the reconstructed tomographic image with labeled chip regions and solder regions may be used as output data. A trained model for extracting chip regions and a trained model for extracting solder regions may be provided separately. In this case, the labeled images may be trained separately. Specifically, each trained model may be generated by training a combination of a tomographic image and an image in which the chip regions within that tomographic image are labeled, and a combination of a tomographic image and an image in which the solder regions within that tomographic image are labeled, as training data.

[0148] The third acquisition unit 1714 corrects the solder area acquired by the second acquisition unit 1713 using the relationship between the solder area and the chip area and the relationship between areas within the solder area. The correction process by the third acquisition unit 1714 may be performed in the same manner as the correction process according to the first embodiment. Here, the correction process according to this embodiment will be described in more detail with reference to Figures 19(a) to 19(c).

[0149] FIG. 19(a) shows an example of a cross-sectional image 1900 of a substrate including chip regions 1901, 1902, and 1903. While this example illustrates an example in which the substrate includes three chip regions, the number of chip regions is not limited to three, and one or more regions may be provided depending on the desired configuration. In this example, solder regions are provided in chip regions 1901, 1902, and 1903 using separate patterns (solder patterns 1910, 1920, and 1930). However, the solder patterns for each of chip regions 1901, 1902, and 1903 may be the same or different. The solder patterns can define not only the relationship between solder regions, but also the relationship between the chip regions and the solder regions. The shapes, number, and positions of the chip regions and solder regions may be arbitrarily set depending on the desired configuration.

[0150] The second acquisition unit 1713 acquires the extraction results of the chip region and the solder region from the trained model by using the tomographic image 1900 as input data for the trained model. Fig. 19(b) shows an example of each chip region 1901, 1902, and 1903 acquired by segmentation using the tomographic image 1900 as input data for the trained model. Fig. 19(c) shows an example of solder regions 1911 to 1919 in the chip region 1901, regarding the solder regions acquired by segmentation using the tomographic image 1900 as input data for the trained model.

[0151] 19(c), solder areas 1911-1915, 1918, and 1919 have been extracted, but solder areas 1916 and 1917 have not been extracted. In this case, the third acquisition unit 1714 corrects solder areas 1916 and 1917 based on the relationship between the solder areas and chip area 1901 defined in a predetermined solder pattern 1910 for solder areas 1911-1919, and the relationship between areas within the solder areas.

[0152] For example, with respect to solder area 1913, if the distance from solder area 1913 to the next solder area in the area below it is greater than a threshold, third acquisition unit 1714 sets an additional solder area between solder area 1913 and the next solder area. In the example of FIG. 19(c), if the distance from solder area 1913 to the next solder area 1919 in the area below solder area 1913 is greater than a threshold, third acquisition unit 1714 sets solder area 1916 at the midpoint between these solder areas. The size of the solder area to be set may be set according to the desired configuration, and may, for example, be the same as the size of the other solder areas, or may be the size of solder area 1916 defined in solder pattern 1910.

[0153] Furthermore, with regard to solder area 1914, when the distance from the bottom end of chip area 1901 to the solder area in the area above said bottom end is greater than a threshold value, third obtaining unit 1714 sets an additional solder area between the bottom end of chip area 1901 and said solder area. In the example of Fig. 19(c), when the distance from the bottom end of chip area 1901 to solder area 1914 in the area above said bottom end is greater than a threshold value, third obtaining unit 1714 sets solder area 1917 at an intermediate position between them. The size of the set solder area may be set in the same manner as above.

[0154] In the above example, the third acquisition unit 1714 corrected the solder areas based on the distance in the vertical direction of the image, but the solder areas may also be corrected based on the distance in the horizontal direction or the distance in a diagonal direction of the image. Also, the threshold value that serves as the basis for the correction process may be dynamically set based on the average distance between the acquired solder areas, as in the first embodiment, or may be set in advance based on the solder pattern 1910. The position of the solder areas may be determined based on the center of gravity of the solder areas.

[0155] Furthermore, since the solder pattern may be set arbitrarily, the solder areas may not be spaced at equal intervals, as shown in solder patterns 1920 and 1930, for example. Therefore, when correcting the solder areas, third acquisition unit 1714 can determine whether or not correction of the solder areas is necessary based on a predetermined solder pattern. For example, third acquisition unit 1714 can determine that the lower left area of ​​solder pattern 1920 and the center area of ​​solder pattern 1930 are areas where correction of the solder areas is not required.

[0156] In the above example, an example in which a solder region is missing has been described, but the third acquisition unit 1714 may exclude any excess solder regions that have been acquired, as in the first embodiment. In this case, the third acquisition unit 1714 can also identify and exclude excess solder regions using the relationships between solder regions and the relationships between solder regions and chip regions. The method for identifying such excess solder regions may be the same as in the first embodiment. For example, if the distance between the acquired solder regions is smaller than a threshold, one of the corresponding solder regions may be considered an outlier (or an inference error) and excluded from the solder regions.

[0157] The display control unit 1715 can cause the control unit 1710 to display on the display unit 1720 the analysis results of the analysis region, such as the presence or absence of solder in the solder region and the quality of the mounting of electronic components based on the presence or absence of solder. The analysis results may be displayed superimposed on the tomographic image, or the tomographic image and the tomographic image on which the analysis results are superimposed may be switched between displaying in response to an instruction from the operator. Furthermore, when analysis results have been acquired for tomographic images of multiple cross sections, the display control unit 1715 can cause the display unit 1720 to display the analysis results for each tomographic image side by side or switch between displaying them. In these cases, the display control unit 1715 may collectively switch between displaying the multiple tomographic images and the analysis results or the multiple tomographic images on which the analysis results are superimposed, in response to an instruction from the operator via the operation unit 1750.

[0158] The display control unit 1715 can also display the analysis region acquired for the tomographic image on the display unit 1720. The display control unit 1715 can also superimpose the analysis region on the tomographic image and display it on the display unit 1720. When analysis regions have been acquired for tomographic images of multiple cross sections, the display control unit 1715 can display images in which these analysis regions are superimposed on the corresponding tomographic images side by side on the display unit 1720, or switch between the images. The display control unit 1715 can also switch between displaying the image on which the analysis region is superimposed and the original tomographic image. In this case, the display control unit 1715 can collectively switch between displaying the multiple tomographic images and the multiple tomographic images on which the analysis region is superimposed, in response to an instruction from the operator via the operation unit 1750.

[0159] In this embodiment, the second acquisition unit 1713 acquires the solder region and the chip region by segmentation, and the third acquisition unit 1714 corrects the solder region, which is the analysis region. However, the region acquired by segmentation is not limited to these, and as described above, it may be any region related to an object in which similar structures are continuously connected. Therefore, as in Examples 1 to 4, regions such as the lumbar vertebrae, cervical vertebrae, and thoracic vertebrae may be used as the analysis region, and these regions and the spinal region may be acquired by segmentation, and the correction process described above may be applied. Even in such cases, by applying the image processing according to the present disclosure, it is possible to correct missing regions that should be acquired and excess regions that have been acquired, and ultimately acquire the analysis region that should be acquired with high accuracy.

[0160] Note that a series of processes according to this embodiment will be omitted because they are similar to those according to embodiment 1. However, it should be noted that in this embodiment, radiation imaging is performed by irradiating radiation obliquely onto the workpiece 1702 and capturing a plurality of projection images while changing the imaging position of the workpiece 1702 as described above.

[0161] As described above, the control unit 1710 according to this embodiment functions as an example of an image processing device including a first acquisition unit 1711, a second acquisition unit 1713, and a third acquisition unit 1714. The first acquisition unit 1711 functions as an example of a first acquisition means that acquires an image obtained by irradiating an object under inspection with radiation in an oblique direction. The second acquisition unit 1713 functions as an example of a second acquisition means that acquires a first region and a second region that includes the first region and is larger than the first region, using the image acquired by the first acquisition unit 1711 as input data for a trained model. Here, solder regions 1911 to 1915 and 1918 to 1919 are examples of the acquired first region, and the chip region 1901 is an example of the acquired second region. The third acquisition unit 1714 functions as an example of a third acquisition means that corrects the acquired first region using at least one of the relationship between the acquired first region and the acquired second region and the relationship between multiple regions within the acquired first region, and acquires an analysis region for performing image analysis processing using the corrected first region. Note that the analysis region can be the corrected first region.

[0162] More specifically, the third acquisition unit 1714 corrects the acquired first region using a predetermined pattern related to the first region and at least one of the relationship between the acquired first region and the acquired second region and the relationship between multiple regions within the acquired first region. Here, the solder pattern 1910 is an example of the predetermined pattern. Furthermore, the third acquisition unit 1714 can determine whether or not the acquired first region needs to be corrected using the predetermined pattern related to the first region.

[0163] The image processing device according to this embodiment can accurately obtain an analysis region in which similar structures are continuously connected, such as a solder region, a lumbar region, etc. Furthermore, because radiation is irradiated obliquely onto the object to be inspected, the object to be inspected can be brought closer to the radiation source, making it easier to adjust the magnification ratio of the image and allowing the size of the inspection device to be made compact.

[0164] The control unit 1710 according to this embodiment further includes a display control unit 1715, which functions as an example of a display control unit that displays the analysis region acquired by the third acquisition unit 1714 on the display unit 1720 (display means). Furthermore, the second acquisition unit 1713 can acquire a plurality of first regions and a plurality of second regions based on a plurality of tomographic images corresponding to at least two cross sections of the object to be inspected, using the trained model. The third acquisition unit 1714 can correct the acquired plurality of first regions and acquire a plurality of analysis regions using the corrected plurality of first images. In this case, the display control unit 1715 can display the acquired plurality of analysis regions or a plurality of tomographic images on which the corresponding analysis regions are superimposed, side by side, on the display unit 1720. This allows the analysis region relating to a plurality of cross sections to be confirmed, enabling more efficient inspection of the object to be performed.

[0165] Furthermore, in response to an instruction from the operator, the display control unit 1715 can simultaneously switch between a plurality of tomographic images corresponding to at least two cross sections and a plurality of analysis regions corresponding to the plurality of tomographic images or a plurality of tomographic images on which the corresponding analysis regions are superimposed, and display them on the display unit 1720. In this case, the acquired analysis regions and the reconstructed tomographic images can be easily compared. This allows for more efficient inspection of the object to be inspected.

[0166] The cross sections for the multiple tomographic images can be set according to at least one of the initial settings, instructions from the operator, the detection results of the state of the inspection object, and the selection of the inspection purpose. Therefore, a tomographic image of a cross section according to the desired setting can be generated, and an analysis region corresponding to the tomographic image can be acquired with high accuracy. This allows for more efficient inspection of the inspection object.

[0167] In this embodiment, the second acquisition unit 1713 acquires the first region and the second region based on a tomographic image using a trained model. Alternatively, the second acquisition unit 1713 may acquire the first region and the second region based on a projection image, a three-dimensional image, or an energy sub-image. In this case, the projection image, the three-dimensional image, or the energy sub-image may be used as input data for the training data of the trained model. Furthermore, an image in which the projection image, the three-dimensional image, or the energy sub-image is labeled with the first region and the second region may be used as output data for the training data. When a three-dimensional image is used as input data for the training data, an image in which the first region and the second region are labeled in a tomographic image of a predetermined cross section may be used as output data for the training data.

[0168] (Variation) The training data of the trained models according to the first to fifth embodiments and the modified examples is not limited to data obtained using the radiation imaging device itself that actually performs imaging. The image data may be data obtained using the same type of radiation imaging device or the same kind of radiation imaging device, depending on the desired configuration.

[0169] Furthermore, when multiple images are used as input to a trained model, the multiple input images can be input to multiple input channels of the trained model, respectively. Alternatively, the multiple input images can be combined into a single image and input to one channel of the trained model. In this case, the single image obtained by combining the input images can be used as input data for the training data of the trained model as well.

[0170] Furthermore, in the above-described first to fifth embodiments and modifications, the control unit 101, 1710 may superimpose segmentation results such as analysis regions on display images and display them on the display unit 107, 1720 in response to an operator's instruction, such as the operation of a button (not shown). When there are multiple display images and corresponding segmentation results, the control unit 101, 1710 may superimpose the segmentation results on each display image collectively and display them on the display unit 107, 1720 in response to an operator's instruction. The control unit 101, 1710 may also display the acquired results, such as the bone density analysis region and background region, or the solder region and chip region, in different colors. In this case, multiple images may be displayed collectively in different colors in response to an operator's instruction.

[0171] In addition, in the trained models according to the above-mentioned Examples 1 to 5 and the modified examples, it is considered that the magnitude of the brightness values ​​of the input data image, the order and gradient of the bright and dark areas, position, distribution, continuity, etc. are extracted as part of the features and used in the inference process.

[0172] Furthermore, the various trained models according to the first to fifth embodiments and the modified examples described above can be provided in the control unit 101, 1710. The trained model may be configured, for example, as a software module executed by a processor such as a CPU, MPU, GPU, or FPGA, or as a circuit performing a specific function such as an ASIC. The trained model may also be provided in a separate server device connected to the control unit 101, 1710. In this case, the control unit 101, 1710 can use the trained model by connecting to a server or the like having the trained model via any network such as the Internet. Here, the server having the trained model may be, for example, a cloud server, a fog server, or an edge server. Note that when configuring a network within a facility, a site including a facility, or a region including multiple facilities to enable wireless communication, the reliability of the network may be improved by configuring it to use radio waves in a dedicated wavelength band allocated exclusively to the facility, site, region, etc. Furthermore, the network may be configured using wireless communication that is capable of high speed, large capacity, low latency, and multiple simultaneous connections.

[0173] The present invention can also be realized by executing the following process: software (programs) that realize one or more functions of the various embodiments and modifications described above are supplied to a system or device via a network or various storage media, and the computer (or CPU, MPU, etc.) of the system or device reads and executes the programs.

[0174] The present invention can also be realized by providing software (programs) that implement one or more of the functions of the various embodiments and modifications described above to a system or device via a network or a storage medium, and having a computer in the system or device read and execute the programs. It can also be realized by a circuit (e.g., an ASIC) that implements one or more functions. A computer may have one or more processors or circuits, and may include multiple separate computers or a network of multiple separate processors or circuits to read and execute computer-executable instructions.

[0175] In this case, the processor or circuitry may include a central processing unit (CPU), a microprocessing unit (MPU), a graphics processing unit (GPU), an application specific integrated circuit (ASIC), or a field programmable gateway (FPGA), and may also include a digital signal processor (DSP), a data flow processor (DFP), or a neural processing unit (NPU).

[0176] The above disclosure includes the following configurations and methods. (Configuration 1) a first acquisition means for acquiring a spinal region and an intervertebral region in an input image; a second acquisition means for acquiring a bone density analysis region, which is a target region for bone density measurement of the lumbar spine, using the acquired spinal region and the acquired intervertebral region; An image processing device having: (Configuration 2) 2. The image processing device according to claim 1, wherein the first acquisition means acquires the spinal region and the intervertebral region in the input image using a trained model. (Configuration 3) 3. The image processing device according to claim 1, wherein the input image is an energy subtraction image. (Configuration 4) 4. The image processing device according to any one of configurations 1 to 3, wherein the input image is an image acquired using X-rays at two types of tube voltages. (Configuration 5) 4. The image processing device according to any one of configurations 1 to 3, wherein the input image is an image acquired using a sensor having two types of materials with different X-ray absorption coefficients. (Configuration 6) 3. The image processing device according to configuration 1 or 2, wherein the input image is a plain X-ray image. (Configuration 7) 3. The image processing device according to claim 2, wherein the first acquisition means separately includes a trained model for acquiring the spinal region and a trained model for acquiring the intervertebral region. (Configuration 8) 3. The image processing device according to claim 2, wherein the first acquisition means includes one trained model for acquiring the spinal region and one trained model for acquiring the intervertebral region. (Configuration 9) 2. The image processing device according to configuration 1, wherein the first acquisition means acquires the spine region in the input image, and acquires the intervertebral region from a spine image generated based on the acquisition result. (Configuration 10) the second acquisition means acquires, as a vertebral body, an area inside the acquired spinal region within a reference region sandwiched between straight lines passing through the acquired intervertebral region; 10. The image processing device according to any one of configurations 1 to 9, wherein the bone density analysis region of the lumbar vertebrae is acquired by counting the acquired vertebral bodies from the pelvis side. (Configuration 11) 11. The image processing device according to any one of configurations 1 to 10, wherein the second acquisition means performs processing to exclude an area outside the acquired spinal area from the acquired intervertebral area. (Configuration 12) 12. The image processing device according to any one of configurations 1 to 11, wherein the second acquisition means performs rotation correction on the input image based on the acquired shape of at least one of the spinal region and the intervertebral region. (Configuration 13) 11. The image processing device according to configuration 10, wherein the second obtaining means draws a straight line forming the reference region obliquely based on the obtained shape of at least one of the spinal region and the intervertebral region. (Configuration 14) 14. The image processing device according to any one of configurations 1 to 13, wherein the second obtaining means sets a background region based on a variance of brightness values ​​of the input image. (Configuration 15) The image processing device according to any one of configurations 1 to 14, wherein the second acquisition means, when the number of the acquired intervertebral regions is less than a predetermined number, acquires the bone density analysis region by dividing the acquired spinal regions into the predetermined number. (Configuration 16) 16. The image processing device according to any one of configurations 1 to 15, wherein the second acquisition means performs correction processing based on the distance between the upper end or lower end of the acquired spinal region and the next intervertebral region on the inside. (Configuration 17) The image processing device according to any one of configurations 1 to 16, wherein the first acquisition means acquires the spine region, the intervertebral region, and the pelvic region in the input image, and the second acquisition means acquires the lumbar bone density analysis region based on the acquisition results of the spine region, the intervertebral region, and the pelvic region. (Configuration 18) The image processing device according to any one of configurations 1 to 17, wherein the first acquisition means acquires the spine region, the intervertebral region, and the rib region in the input image, and the second acquisition means acquires the lumbar bone density analysis region based on the acquisition results of the spine region, the intervertebral region, and the rib region. (Configuration 19) The image processing device according to any one of configurations 1 to 18, wherein the second acquisition means performs a correction process to lengthen or shorten the spinal region based on at least one of the acquired distance from the upper end of the spinal region to the lower end of the rib region or the acquired distance from the lower end of the spinal region to the upper end of the pelvic region. (Configuration 20) 20. The image processing device according to any one of configurations 1 to 19, wherein the first acquisition means acquires an outer frame region that combines the spine region and at least one of a pelvic region and a rib region in the input image. (Configuration 21) the first acquisition means acquires the spinal region and the intervertebral region for at least two of three images, which are two images acquired using a DXA method and an energy subtraction image generated from the two images; 3. The image processing device according to configuration 1 or 2, wherein the second acquisition means acquires the bone density analysis region based on acquisition results of the spinal region and the intervertebral region for the at least two images. (Configuration 22) 22. The image processing device according to any one of configurations 1 to 21, further comprising a display unit that displays to an operator the results of acquisition by the second acquisition unit or the process of acquisition. (Configuration 23) 23. The image processing device according to any one of configurations 1 to 22, further comprising an operation means for an operator to operate the image processing device while viewing the results or the process of acquisition by the second acquisition means. (Configuration 24) a first acquisition means for acquiring an image by irradiating an object to be inspected with radiation in an oblique direction; A second acquisition means for acquiring a first region and a second region that includes the first region and is larger than the first region by using the acquired image as input data for a trained model; a third acquisition means for correcting the acquired first region using at least one of the relationship between the acquired first region and the acquired second region and the relationship between a plurality of regions within the acquired first region, and for acquiring an analysis region for performing image analysis processing using the corrected first region; An image processing device having: (Configuration 25) The image processing device described in configuration 24, wherein the third acquisition means corrects the acquired first area using a predetermined pattern related to the first area and at least one of the relationship between the acquired first area and the acquired second area and the relationship between multiple areas within the acquired first area. (Configuration 26) 26. The image processing device according to configuration 25, wherein the third obtaining means uses the predetermined pattern to determine whether or not correction of the obtained first area is necessary. (Configuration 27) 27. The image processing device according to any one of configurations 24 to 26, wherein the analysis region is the corrected first region. (Configuration 28) further comprising a display control means for displaying the acquired analysis region on a display means; the second acquisition means acquires the first regions and the second regions based on a plurality of tomographic images corresponding to at least two cross sections of the object to be inspected, using the trained model; the third acquisition means corrects the acquired first regions and acquires the analysis regions using the corrected first regions; 28. The image processing device according to any one of configurations 24 to 27, wherein the display control means causes the display means to display the acquired analysis regions or the acquired tomographic images on which the corresponding analysis regions are superimposed, in a line. (Configuration 29) 29. The image processing device according to configuration 28, wherein the display control means, in response to an instruction from an operator, collectively switches between the plurality of tomographic images and the plurality of analysis regions or the plurality of tomographic images on which the corresponding analysis regions are superimposed, and causes the display means to display them. (Configuration 30) 30. An image processing device according to any one of configurations 28 to 29, wherein the cross sections relating to the plurality of tomographic images are set in accordance with at least one of initial settings, instructions from an operator, detection results of the state of the object to be inspected, and selection of an inspection purpose. (Method 1) a first acquisition step of acquiring a spinal region and an intervertebral region in an input image; a second acquisition step of acquiring a bone density analysis region, which is a target region for bone density measurement of the lumbar spine, using the acquired spinal region and the acquired intervertebral region; A method of operating an image processing device comprising: (Method 2) a first acquisition step of acquiring an image by irradiating an object to be inspected with radiation in an oblique direction; A second acquisition step of acquiring a first region and a second region that includes the first region and is wider than the first region by using the acquired image as input data for a trained model; a third acquisition step of correcting the acquired first region using at least one of the relationship between the acquired first region and the acquired second region and the relationship between a plurality of regions within the acquired first region, and acquiring an analysis region for performing image analysis processing using the corrected first region; A method of operating an image processing device comprising: (Program 1) A program that, when executed by a computer, causes the computer to execute each step of the method for operating an image processing device according to Method 1 or 2.

[0177] Although the present invention has been described above with reference to examples and modifications, the present invention is not limited to the above examples and modifications. The present invention also includes inventions that have been modified within the scope of the present invention and inventions equivalent to the present invention. Furthermore, the above-described examples and modifications can be combined as appropriate within the scope of the present invention. [Explanation of symbols]

[0178] 120: Image processing device, 121: First acquisition unit, 122: Second acquisition unit

Claims

1. A first acquisition means for acquiring a spinal region and an intervertebral region in an input image using a trained model; a second acquisition means for acquiring a bone density analysis region, which is a target region for bone density measurement of the lumbar spine, by using the acquired spinal region and the acquired intervertebral region; having The second acquisition means acquires, as a vertebral body, an area inside the acquired spinal region within a reference region sandwiched between straight lines passing through the acquired intervertebral region.

2. The image processing apparatus according to claim 1 , wherein the input image is an energy subtraction image.

3. The image processing apparatus according to claim 1 , wherein the input image is an image acquired using X-rays with two types of tube voltages.

4. 2. The image processing apparatus according to claim 1, wherein the input image is an image acquired using a sensor having two types of materials with different X-ray absorption coefficients.

5. The image processing apparatus according to claim 1 , wherein the input image is a plain X-ray image.

6. The image processing device according to claim 1 , wherein the first acquisition means separately includes a trained model for acquiring the spinal region and a trained model for acquiring the intervertebral region.

7. The image processing device according to claim 1 , wherein the first acquisition means includes one trained model for acquiring the spinal region and the intervertebral region, respectively.

8. The image processing apparatus according to claim 1 , wherein the first acquisition means acquires the spinal region in the input image, and acquires the intervertebral region from a spinal image generated based on an acquisition result.

9. An image processing device as described in claim 1, wherein the bone density analysis area of ​​the lumbar vertebrae is obtained by counting the acquired vertebral bodies from the pelvic side.

10. The image processing apparatus according to claim 1 , wherein the second acquisition means performs processing for excluding an area outside the acquired spinal region from the acquired intervertebral region.

11. The image processing apparatus according to claim 1 , wherein the second acquisition means performs rotation correction on the input image based on the acquired shape of at least one of the spinal region and the intervertebral region.

12. The image processing apparatus according to claim 1 , wherein the second acquisition means draws a straight line constituting the reference region obliquely based on the shape of at least one of the acquired spinal region and intervertebral region.

13. The image processing apparatus according to claim 1 , wherein the second acquisition means sets a background region based on a variance of luminance values ​​of the input image.

14. The image processing device according to claim 1 , wherein the second acquisition means acquires the bone density analysis region by dividing the acquired spinal region into the predetermined number when the number of the acquired intervertebral regions is less than a predetermined number.

15. 2 . The image processing apparatus according to claim 1 , wherein the second acquisition means performs a correction process based on a distance between an upper end or a lower end of the acquired spinal region and an intervertebral region one space inward from the upper end or the lower end.

16. 2. The image processing device according to claim 1, wherein the first acquisition means acquires the spinal region, the intervertebral region, and the pelvic region in the input image, and the second acquisition means acquires the lumbar bone density analysis region based on the acquisition results of the spinal region, the intervertebral region, and the pelvic region.

17. 2. The image processing device according to claim 1, wherein the first acquisition means acquires the spine region, the intervertebral region, and the rib region in the input image, and the second acquisition means acquires the lumbar bone density analysis region based on the acquisition results of the spine region, the intervertebral region, and the rib region.

18. 2. The image processing device according to claim 1, wherein the second acquisition means performs a correction process to extend or shorten the spinal region based on at least one of the acquired distance from the upper end of the spinal region to the lower end of the rib region or the distance from the lower end of the spinal region to the upper end of the pelvic region.

19. 2 . The image processing apparatus according to claim 1 , wherein the first acquisition means acquires an outer frame region obtained by fusing the spine region and at least one of a pelvis region and a rib region in the input image.

20. the first acquisition means acquires the spinal region and the intervertebral region for at least two images out of three images, which are two images acquired using a DXA method and an energy subtraction image generated from the two images; The image processing apparatus according to claim 1 , wherein the second acquisition means acquires the bone density analysis region based on acquisition results of the spinal region and the intervertebral region for the at least two images.

21. 2. The image processing apparatus according to claim 1, further comprising a display unit for displaying to an operator the results or the process of acquisition by said second acquisition unit.

22. 2. The image processing apparatus according to claim 1, further comprising an operation means for allowing an operator to operate the image processing apparatus while viewing the result or the process of acquisition by the second acquisition means.

23. A first acquisition step of acquiring a spinal region and an intervertebral region in an input image using a trained model; a second acquisition step of acquiring a bone density analysis region, which is a target region for bone density measurement of the lumbar spine, using the acquired spinal region and the acquired intervertebral region; Including, A method for operating an image processing device, wherein the second acquisition step includes acquiring an area inside the acquired spinal region as a vertebral body within a reference region sandwiched between straight lines passing through the acquired intervertebral region.

24. A program which, when executed by a computer, causes the computer to execute each step of the method for operating an image processing apparatus according to claim 23.

Images