Image diagnostic device for hip joint disease, image diagnostic system, image diagnostic method, and program

Abstract

An image diagnostic device (100) comprises: a bony acetabular line detection unit (153) that detects, on the basis of a pixel value of each pixel of an ultrasonic image acquired by an acquisition unit (152), a bony acetabular line in the ultrasonic image; and a baseline detection unit (154) that detects, on the basis of the pixel value of each pixel of the ultrasonic image acquired by the acquisition unit (152), a straight line indicating an iliac outer wall in the ultrasonic image and detects a baseline parallel to the detected straight line indicating the iliac outer wall and passing through a bony acetabulum or a lower end of the iliac outer wall at an end portion of the bony acetabular line detected by the bony acetabular line detection unit (153).

Description

Imaging Diagnosis Apparatus, Imaging Diagnosis System, Imaging Diagnosis Method, and Program for Hip Joint Diseases

[0001] The present invention relates to an imaging diagnosis apparatus, an imaging diagnosis system, an imaging diagnosis method, and a program for hip joint diseases.

[0002] In the field of orthopedics, ultrasonic examinations are performed to grasp anatomical position abnormalities, deformations, injuries, etc. of the hip joint of a subject. Since ultrasonic examinations have the weakness that the diagnosis results are easily influenced by the examiner's examination technique, technological development for improving the accuracy of diagnosis has been promoted. For example, Patent Document 1 discloses an imaging diagnosis system that supports the diagnosis of whether there is hip dysplasia in the hip joint of a subject based on the angle formed by a first auxiliary line and a second auxiliary line set on an ultrasonic image.

[0003] Japanese Unexamined Patent Application Publication No. 2022-181902 [[ID=!0]

[0004] The imaging diagnosis system of Patent Document 1 can support accurate diagnosis by a doctor with little experience and is extremely useful. However, it takes a lot of time for the arithmetic processing of estimating feature points on the ultrasonic image and setting a pair of auxiliary lines. Therefore, there is a time lag from the acquisition of the ultrasonic image that fluctuates moment by moment to the calculation of the diagnosis result corresponding to the ultrasonic image, and there is room for improvement in that the diagnosis result based on the ultrasonic image cannot be quickly provided to a user such as a doctor.

[0005] [[ID=!5]] The present invention has been made based on such a background, and an object thereof is to provide an imaging diagnosis apparatus, an imaging diagnosis system, an imaging diagnosis method, and a program capable of quickly providing a user with a diagnosis result based on an ultrasonic image of the hip joint.

[0006] To achieve the above objective, an image diagnostic device according to a first aspect of the present invention includes: an acquisition unit that acquires an ultrasound image of the hip joint of a subject; a bony acetabular line detection unit that detects a bony acetabular line in the ultrasound image based on the pixel value of each pixel in the ultrasound image acquired by the acquisition unit; a baseline detection unit that detects a straight line indicating the outer wall of the iliac crest in the ultrasound image based on the pixel value of each pixel in the ultrasound image acquired by the acquisition unit, and detects a baseline that is parallel to the detected straight line indicating the outer wall of the iliac crest and passes through the bony acetabular beak or the lower end of the outer wall of the iliac crest at the end of the bony acetabular line detected by the bony acetabular line detection unit; and an output unit that outputs a detection image in which the bony acetabular line detected by the bony acetabular line detection unit and the baseline detected by the baseline detection unit are drawn on the ultrasound image acquired by the acquisition unit.

[0007] The baseline detection unit may extract a plurality of local maxima points in the ultrasound image acquired by the acquisition unit where the pixel value is locally at its maximum, set a polygon region surrounding the iliac wall in the ultrasound image based on the pixel value of each pixel in the ultrasound image acquired by the acquisition unit, and detect a straight line representing the iliac wall based on the set of local maxima points that are located within the polygon region from the set of extracted local maxima points.

[0008] The image diagnostic device further includes an angle calculation unit that calculates the α angle, which is the angle between the bony acetabular line detected by the bony acetabular line detection unit and the baseline detected by the baseline detection unit, and the output unit may output the α angle calculated by the angle calculation unit.

[0009] The image diagnostic device further includes a determination unit that determines whether or not the subject has a hip joint disease based on the α angle calculated by the angle calculation unit, and the output unit may output the determination result determined by the determination unit.

[0010] The determination unit may determine that the subject does not have acetabular dysplasia if the angle calculated by the angle calculation unit is 60° or more.

[0011] The output unit draws a rectangular target area on the detected image, which is set on the iliac outer wall side with respect to the bony acetabular beak or the lower end of the iliac outer wall detected by the bony acetabular line detection unit. The target area may be set so that when the baseline, whose inclination changes due to the user's operation of the ultrasound probe, intersects with a horizontal line in the target area's frame that is on the opposite side of the bony acetabular beak or the lower end of the iliac outer wall, the angle calculated by the angle calculation unit can be evaluated.

[0012] The target area may extend upward from the bony acetabular canusa or the lower end of the iliac outer wall detected by the bony acetabular line detection unit and may be set in a band shape.

[0013] The bony acetabular line detection unit may output an output image in which a bounding box surrounding the bony acetabular line is drawn on the ultrasound image acquired by the acquisition unit, if a bony acetabular line is present in the ultrasound image acquired by the acquisition unit.

[0014] The acquisition unit may acquire the ultrasound images of the subject's hip joint at regular intervals, and the output unit may output detection images based on the ultrasound images acquired by the acquisition unit in real time.

[0015] The image diagnostic apparatus further comprises: a cartilaginous acetabular line detection unit that detects a cartilaginous acetabular line in the ultrasound image based on the pixel value of each pixel of the ultrasound image acquired by the acquisition unit; and an angle calculation unit that calculates a β angle, which is the angle between the cartilaginous acetabular line detected by the cartilaginous acetabular line detection unit and the baseline detected by the baseline detection unit, wherein the output unit may output the β angle calculated by the angle calculation unit.

[0016] The image diagnostic device further includes an FHC calculation unit that detects the outermost and innermost points of the bony acetabular canal and femoral head in the ultrasound image based on the pixel value of each pixel of the ultrasound image acquired by the acquisition unit, and calculates the FHC of the femoral head, which indicates the ratio of the portion of the femoral head covered by the hip acetabulum, based on the positions of the outermost and innermost points of the bony acetabular canal and femoral head in the ultrasound image, and the output unit may output the FHC calculated by the FHC calculation unit.

[0017] To achieve the above objective, an image diagnostic system according to a second aspect of the present invention comprises: an image diagnostic device; and an ultrasound examination device which is communicably connected to the image diagnostic device and transmits data of an ultrasound image of the hip joint of a subject to the image diagnostic device.

[0018] To achieve the above objective, an image diagnostic method according to a third aspect of the present invention is an image diagnostic method performed by an image diagnostic device, comprising: acquiring an ultrasound image of the hip joint of a subject; detecting a bony acetabular line in the ultrasound image based on the pixel value of each pixel in the acquired ultrasound image; detecting a straight line indicating the outer wall of the iliac crest in the ultrasound image based on the pixel value of each pixel in the acquired ultrasound image, and detecting a baseline that is parallel to the detected straight line indicating the outer wall of the iliac crest and passes through the bony acetabular beak or the lower end of the outer wall of the iliac crest at the end of the detected bony acetabular line; and outputting a detection image in which the detected bony acetabular line and the detected baseline are drawn on the acquired ultrasound image.

[0019] To achieve the above objective, the program according to the fourth aspect of the present invention causes the computer to function as: an acquisition means for acquiring an ultrasound image of the hip joint of a subject; a bony acetabular line detection means for detecting a bony acetabular line in the ultrasound image based on the pixel value of each pixel in the ultrasound image acquired by the acquisition means; a baseline detection means for detecting a straight line indicating the outer wall of the iliac crest in the ultrasound image based on the pixel value of each pixel in the ultrasound image acquired by the acquisition means, and detecting a baseline that is parallel to the detected straight line indicating the outer wall of the iliac crest and passes through the bony acetabular beak or the lower end of the outer wall of the iliac crest at the end of the bony acetabular line detected by the bony acetabular line detection means; and an output means for outputting a detection image in which the bony acetabular line detected by the bony acetabular line detection means and the baseline detected by the baseline detection means are drawn on the ultrasound image acquired by the acquisition means.

[0020] According to the present invention, it is possible to provide an image diagnostic device, an image diagnostic system, an image diagnostic method, and a program that can quickly provide users with diagnostic results based on ultrasound images of the hip joint.

[0021] This is a schematic diagram showing the configuration of an image diagnostic system according to Embodiment 1 of the present invention. This is a diagram for explaining the α angle in an ultrasound image of a subject's hip joint. This is a block diagram showing the hardware configuration of an image diagnostic device according to Embodiment 1 of the present invention. This is a diagram showing an example of a first training dataset according to Embodiment 1 of the present invention. This is a conceptual diagram of a neural network used by an image diagnostic device according to Embodiment 1 of the present invention. This is a diagram showing an example of a data table for an image data storage unit according to Embodiment 1 of the present invention. This is a diagram showing how the bony acetabular line is detected by the image diagnostic device according to Embodiment 1 of the present invention. This is a diagram showing how the local maximum point is screened by the image diagnostic device according to Embodiment 1 of the present invention. This is a diagram showing how a straight line representing the outer wall of the iliac crest is generated by the image diagnostic device according to Embodiment 1 of the present invention. This is a diagram showing how a baseline is generated by the image diagnostic device according to Embodiment 1 of the present invention. (a) and (b) are both diagrams showing examples of diagnostic result screens displayed on the display of the diagnostic device according to Embodiment 1 of the present invention. This is a flowchart showing the flow of the learning process according to Embodiment 1 of the present invention. This is a flowchart showing the flow of the image diagnostic process according to Embodiment 1 of the present invention. This is a flowchart showing the flow of the baseline detection process according to Embodiment 1 of the present invention. This is a diagram showing an example of a second training dataset according to Embodiment 2 of the present invention. This is a block diagram showing the hardware configuration of an image diagnostic device according to Embodiment 3 of the present invention. This is a diagram showing an example of a third training dataset according to Embodiment 3 of the present invention. This figure shows the position of the bounding box in the output image of the third training dataset according to Embodiment 3 of the present invention. This figure illustrates the β angle in an ultrasound image of the subject's hip joint. This figure shows an example of the fourth training dataset according to Embodiment 3 of the present invention. This figure shows the position of the bounding box in the output image of the fourth training dataset according to Embodiment 3 of the present invention. This figure shows an example of the data table of the image data storage unit according to Embodiment 3 of the present invention. This figure shows an example of the data table of the parameter storage unit according to Embodiment 3 of the present invention. This flowchart shows the flow of the image diagnostic processing according to Embodiment 3 of the present invention.This figure shows a method for diagnosing the presence or absence of hip joint disease using an image diagnostic device according to Embodiment 4 of the present invention. This figure shows an example of a first training dataset according to Embodiment 4 of the present invention.

[0022] Hereinafter, an image diagnostic apparatus, image diagnostic system, image diagnostic method, and program according to embodiments of the present invention will be described in detail with reference to the drawings. In each drawing, the same or equivalent parts are denoted by the same reference numerals. In the embodiments, a two-dimensional coordinate system is used in which the horizontal axis of the ultrasound image is the X-axis and the vertical axis is the Y-axis.

[0023] (Embodiment 1) First, an image diagnostic apparatus, image diagnostic method, and program according to Embodiment 1 will be described with reference to Figures 1 to 14. As shown in Figure 1, the image diagnostic system 1 comprises an ultrasound examination apparatus 2 and an image diagnostic apparatus 100. The ultrasound examination apparatus 2 and the image diagnostic apparatus 100 are connected to each other via a wired or wireless communication line.

[0024] The ultrasound examination device 2 is an imaging device that transmits and receives ultrasound waves to and from the area of ​​the patient to be examined and acquires ultrasound images of the patient. The ultrasound examination device 2 is equipped with an ultrasound probe that emits ultrasound waves toward the patient and captures the ultrasound waves reflected by the area of ​​the patient to be examined. A user such as a physician applies the ultrasound probe to the area of ​​the patient to be examined and takes an image, and the ultrasound examination device 2 transmits the ultrasound image data obtained from the image to the diagnostic imaging device 100.

[0025] The diagnostic imaging device 100 performs image processing on ultrasound images acquired by the ultrasound examination device 2 to provide information useful for diagnosing whether a patient has hip joint disease, particularly acetabular dysplasia in infants. Specifically, as shown in Figure 2, the diagnostic imaging device 100 detects the bony acetabular line and baseline in the ultrasound image and displays the detected image, in which the bony acetabular line and baseline are drawn on the ultrasound image, on the display of the ultrasound examination device 2 in real time. The diagnostic imaging device 100 may also display the detected image on its own display in real time.

[0026] In Embodiment 1, the term "real-time" means that each frame of the ultrasound video is processed immediately. For example, it means that the detection images corresponding to each frame are displayed on the display one after another without any delay from the moment a user, such as a doctor, captures each frame of the ultrasound video.

[0027] The ultrasound image in Figure 2 was obtained by applying an ultrasound probe to the ilium from the ventral to the dorsal side while the infant was lying on a bed. The bony roof line is the straight line connecting the bony rim point and the lower limb point of the ilium. The bony rim point is the point where the shape of the ilium changes from concave to convex. The baseline is a straight line that extends parallel to the outer wall of the ilium and passes through the bony rim point. Users can determine that the subject's hip joint is normal, and if the subject is an infant, that there is no developmental abnormality, if the α angle, which is the angle between the bony roof line and the baseline, is above a threshold. For example, in a screening for acetabular dysplasia, it can be determined that: if the α angle is 60° or greater, the subject does not have acetabular dysplasia; if the α angle is 50° or greater but less than 60°, there is a delay in ossification of the bony acetabulum; if the α angle is 43° or greater but less than 50°, there is a risk of dislocation or the femoral head has lost its center of gravity; and if the α angle is less than 43°, there is dislocation or complete dislocation. Such a method is suitable for quickly checking, for example, whether there is any developmental abnormality in the hip joint of an infant, such as acetabular dysplasia.

[0028] Next, the hardware configuration of the medical imaging device 100 will be described with reference to Figure 3. The medical imaging device 100 is, for example, a general-purpose computer. The medical imaging device 100 includes an operation unit 110, a display unit 120, a communication unit 130, a storage unit 140, and a control unit 150. Each part of the medical imaging device 100 is interconnected via an internal bus (not shown).

[0029] The operation unit 110 receives user instructions and supplies operation signals corresponding to the received operations to the control unit 150. The operation unit 110 includes, for example, a mouse and a keyboard.

[0030] The display unit 120 displays various images to the user based on control signals supplied from the control unit 150. For example, the display unit 120 displays an ultrasound image of the hip joint taken by the ultrasound examination device 2 and the α angle calculated by the control unit 150.

[0031] The communication unit 130 is a communication interface for the medical imaging device 100 to communicate with external equipment. The communication unit 130 includes, for example, an antenna and input / output terminals, and communicates with external equipment via a communication circuit. For example, the communication unit 130 receives ultrasound image data from the ultrasound examination device 2 and transmits the detected image to the ultrasound examination device 2.

[0032] The memory unit 140 includes, for example, RAM (Random Access Memory), ROM (Read Only Memory), flash memory, and a hard disk. The memory unit 140 stores programs and various data executed by the control unit 150. The memory unit 140 also temporarily stores various information and functions as work memory for the control unit 150 to execute processing. Furthermore, the memory unit 140 includes a training data storage unit 141, a trained model storage unit 142, and an image data storage unit 143.

[0033] The learning data storage unit 141 stores a first learning data set that includes multiple datasets used as training data for machine learning. As shown in Figure 4, the first learning data set includes one ultrasound image of a subject obtained by an ultrasound probe (input image) and one labeled image (output image) in which data labeling is performed by manually drawing a bounding box around the bony acetabular line on the ultrasound image.

[0034] The first training data uses ultrasound images acquired from different subjects. The ultrasound images are, for example, grayscale images with pixel values ​​(hierarchy) from 0 to 255. Both the ultrasound images and label images consist of m x n pixels, for example, 256 x 256 pixels. The bounding box is a rectangular box with the ends of the bony acetabular line as diagonal vertices, and its position and size are determined by the pixel positions of these diagonal vertices.

[0035] Furthermore, in the first training dataset, data augmentation may be performed as a preprocessing step for machine learning to increase the number of data points. Data augmentation is a process that increases the amount of data in the first training dataset by treating the data originally in the first training dataset as images and transforming or combining them. For example, data augmentation can be performed by randomly flipping the sample images horizontally, changing their brightness, or rotating them.

[0036] Returning to Figure 3, the trained model storage unit 142 stores the first trained model generated by machine learning based on the first training data stored in the training data storage unit 141. The first trained model is a model that outputs the pixel value of each pixel in the output image in response to the pixel value of each pixel in the input image, and is pre-generated based on the first training data. As the first trained model, a multi-layered neural network consisting of multiple interconnected nodes is used. The trained model storage unit 142 stores optimized weights that indicate the strength of the connections between the nodes constituting the neural network. As the first trained model, for example, YOLO (You Only Look Once) is used.

[0037] As shown in Figure 5, the neural network comprises an input layer into which the pixel values ​​of each pixel in an input image are input, an output layer into which the pixel values ​​of each pixel in an output image are output, and at least one hidden layer positioned between the input and output layers. The arrows between nodes in each layer represent the connections between parameters in the input and output layers. The number of nodes in the input layer corresponds to the number of input data, and the number of nodes in the output layer corresponds to the number of output data. The arrows between nodes represent the connections between parameters in the input and output layers.

[0038] The input data consists of the pixel values ​​of each pixel in the input image, and can be represented as I(1,1), I(1,2), ..., I(m,n). Similarly, the output data consists of the pixel values ​​of each pixel in the output image, and can be represented as O(1,1), O(1,2), ..., O(m,n). Each node in the input layer corresponds to each pixel (m x n) in the input image, and each node in the output layer corresponds to each pixel (m x n) in the output image.

[0039] The image data storage unit 143 is created for each ultrasound examination and stores the ultrasound image data repeatedly captured by the ultrasound examination device 2, as shown in Figure 6, by associating it with the α angle, which is the angle between the bony acetabular line and the baseline, and the pixel positions of the diagonal vertices that indicate the positions of the boundary box surrounding the bony acetabular line, for example, the upper left vertex (bony acetabular canusa) and the lower right vertex (lower end of the ilium). The ultrasound image data is assigned an image ID (Identification) that is generated in the order in which the data was acquired. The image ID is identification information for identifying each image data.

[0040] Returning to Figure 3, the control unit 150 includes a processor and controls each part of the image diagnostic device 100. The processor includes, for example, a CPU (Central Processing Unit) and a GPU (Graphics Processing Unit). The control unit 150 executes the learning process in Figure 12, the image diagnostic process in Figure 13, and the baseline detection process in Figure 14 by executing the program stored in the storage unit 140. Functionally, the control unit 150 includes a learning unit 151, an acquisition unit 152, a bony acetabular line detection unit 153, a baseline detection unit 154, an angle calculation unit 155, and an output unit 156.

[0041] The learning unit 151 performs machine learning using the first training data stored in the training data storage unit 141, and stores the first trained model generated by machine learning in the trained model storage unit 142. The learning unit 151 generates the first trained model by performing supervised learning using all or part of the multiple datasets included in the first training data as training data.

[0042] More specifically, the learning unit 151 uses multiple datasets included in the first training data as training data to adjust the weights that represent the connection state of each layer in the neural network. The procedure involves first inputting the pixel values ​​of each pixel in the input image from the training data into each node of the input layer, and then comparing the pixel values ​​of each pixel in the output image output from each node of the output layer with the pixel values ​​of each pixel in the label image from the training data. The weights are then optimized so that the difference between the pixel values ​​of each pixel in the output image output from the output layer and the pixel values ​​of each pixel in the label image from the training data is as small as possible. For example, backpropagation is used to optimize the weights.

[0043] The acquisition unit 152 periodically acquires ultrasound image data transmitted from the ultrasound examination device 2 and stores it in the image data storage unit 143, associating it with an image ID. The acquisition of image data by the acquisition unit 152 includes receiving image data from the ultrasound examination device 2 and reading data stored in the storage unit 140. The acquisition unit 152 also determines whether it has received a command from the user to stop the diagnosis, and stops acquiring ultrasound image data if it determines that it has received such a command.

[0044] The bony acetabular line detection unit 153 uses a first trained model stored in the trained model storage unit 142 to detect the bony acetabular line in the ultrasound image of the subject acquired by the acquisition unit 152. Specifically, when the pixel value of each pixel in the ultrasound image is input to each node of the input layer of the first trained model, the pixel value of each pixel in the output image is output from each node of the output layer. If a bony acetabular line is present in the ultrasound image, a bounding box surrounding the bony acetabular line is drawn on the output image. If a bony acetabular line is not present in the ultrasound image, no bounding box surrounding the bony acetabular line is drawn on the output image. When a bony acetabular line is detected, as shown in Figure 7, the upper left vertex of the bounding box surrounding the bony acetabular line corresponds to the bony acetabular beak (symbol ★), so the pixel position of the bony acetabular beak can also be identified at the same time. The lower right vertex of the bounding box corresponds to the lower end of the ilium.

[0045] The baseline detection unit 154 detects a baseline in the ultrasonic image of the subject acquired by the acquisition unit 152 based on the bony acetabular rim detected by the bony acetabular rim detection unit 153. Specifically, a plurality of local maximum points are extracted from the ultrasonic image of the subject, and the baseline is detected using the extracted local maximum points. Since the characteristics of the shading pattern representing the bone transparency are reflected in the local maximum points of the ultrasonic image, the detection accuracy of the baseline can be improved by using the local maximum points. Hereinafter, a specific procedure for detecting the baseline in the ultrasonic image will be described with reference to FIGS. 8 to 10.

[0046] First, as shown in FIG. 8, a plurality of local maximum points are extracted in the ultrasonic image. Specifically, first, the pixel value of each pixel in the ultrasonic image is filtered with the maximum value of the pixel values of the adjacent N pixels. The adjacent N pixels are the pixels above, below, left, and right of the target pixel and are the Nth pixel counted from the target pixel. For example, it is preferably set to N = 3. In the filtering, first, the original ultrasonic image is processed by a maximum value filter to generate a filter image in which the pixel value of each pixel in the original ultrasonic image is replaced with the maximum value of the pixel values of the adjacent N pixels. Next, the pixel values of the pixels at the same coordinates in the original image and the filter image are compared, and a masking process is executed to set the pixel values of each pixel whose pixel values are not the same to zero.

[0047] Next, the pixel values of the pixels with small pixel values among the pixels remaining after the masking are set to zero. The pixels whose pixel values are set to zero in this process are pixels with pixel values less than or equal to the order times of the maximum pixel value for all pixels. For example, it is preferably set to order = 0.5. The pixels with pixel values remaining after the above series of processes are local maximum points.

[0048] Next, local maximum points that are unnecessary for detecting the outer wall of the ilium are deleted from the set of extracted local maximum points. Specifically, taking the top left vertex of the ultrasonic image as the origin and the pixel position of the bony acetabular rim as (pX, pY), the detection area is set as follows, and local maximum points outside the detection area are deleted. The detection area is rectangular as shown in FIG. 8. Since the outer wall of the ilium exists near the bony acetabular rim, the detection area is set in the area where the outer wall of the ilium is presumed to exist based on the bony acetabular rim. X direction: pX - Cx × ΔX ≤ x ≤ pX + Cx × ΔX Y direction: Cy × pY ≤ y ≤ pY

[0049] Here, ΔX is the width of the detection area in the X-axis direction and may be set considering the user's skill level and the characteristics of the subject. If the horizontal width of the ultrasonic image is t, ΔX may be set to, for example, ΔX = 0.3 × t so that it is smaller than the horizontal width t. Cx and Cy are adjustment coefficients and are both positive values. For example, Cx = 0.7 and Cy = 0.4 may be set.

[0050] Next, as shown in FIG. 9, linear approximation is performed on a plurality of local maximum points within the detection area to obtain a straight line corresponding to the outer wall of the ilium. In linear approximation, for example, a regression line can be obtained using the least squares method. The regression line obtained at this time is the straight line corresponding to the outer wall of the ilium. Since the outer wall of the ilium exists on the ilium with a bulge like a mountain ridge line, and each local maximum point corresponds to one of the peaks constituting the mountain ridge line, the outer wall of the ilium can be detected based on the set of local maximum points within the detection area.

[0051] Next, as shown in FIG. 10, a baseline is generated based on the straight line corresponding to the outer wall of the ilium and the bony acetabular rim (the end of the bony acetabular line). Specifically, a straight line parallel to the outer wall of the ilium and passing through the bony acetabular rim may be drawn as the baseline. The above is the specific procedure for detecting the baseline.

[0052] Returning to Figure 3, the angle calculation unit 155 calculates the α angle based on the bony acetabular line detected by the bony acetabular line detection unit 153 and the baseline detected by the baseline detection unit 154. The user can determine that the subject's hip joint is normal if the α angle is above the threshold, and if the α angle is below the threshold, the subject, if an infant, may have a developmental disorder in the hip joint, such as acetabular dysplasia.

[0053] The output unit 156 generates a detection image by plotting the bony acetabular line detected by the bony acetabular line detection unit 153 and the baseline detected by the baseline detection unit 154 on the ultrasound image of the subject acquired by the acquisition unit 152, and outputs a diagnostic result screen to the outside, which includes the generated detection image and the α angle calculated by the angle calculation unit 155. The output unit 156 can, for example, display the diagnostic result screen in real time on the display unit 120 and also on the display of the ultrasound examination device 2 to assist the user in making a diagnosis. The diagnostic result screen is updated in real time each time the latest ultrasound image is acquired.

[0054] Figures 11(a) and (b) are examples of diagnostic result screens. The detected image on the diagnostic result screen is a drawing of the bony acetabular line and baseline superimposed on the ultrasound image, but in Figures 11(a) and (b), the illustration of the ultrasound image is omitted for ease of understanding.

[0055] In the detection image, a rectangular target area is drawn on the iliac wall side, relative to the bony acetabular beak. The target area extends in a band along the Y-axis from the position pixels (pX, pY) of the bony acetabular beak, serving as a guide to assist the user in manipulating the ultrasound probe. Such a guide is necessary because the orientation of the resulting ultrasound image differs depending on the tilt of the ultrasound probe relative to the diagnostic site. The target guide assists the user in manipulating the ultrasound probe so that the baseline drawn on the ultrasound image is oriented vertically (in the Y-axis direction). Images with a vertically oriented baseline are suitable for accurate diagnosis because the iliac wall is also oriented vertically in the image, allowing for threshold evaluation of the alpha angle.

[0056] The target area is defined by a pair of opposing horizontal lines extending in the X-axis direction and a pair of opposing vertical lines extending in the Y-axis direction. The lower horizontal line of the target area is set so that its midpoint is located at the pixel position (pX, pY) of the bony acetabular canusa, and the upper horizontal line L of the target area is set so that its Y coordinate value is zero. The length d in the Y-axis direction and the width t in the X-axis direction of the target area can be set to arbitrary values ​​considering the appearance of the maximum points of the iliac wall. For example, the length d can be set to d = pY, and the width t can be set to 20% of the width in the X-axis direction of the bony acetabular line in order to ensure the number of maximum points necessary for baseline calculation.

[0057] The following describes how to use the target area in the detection image. First, the user focuses on the upper transverse line L extending from the bony acetabular canusa in the target area, and operates the ultrasound probe while checking whether the baseline intersects with the transverse line L. Specifically, as shown in Figure 11(a), if the baseline does not intersect with the transverse line L of the target area, threshold evaluation of the α angle using the ultrasound image at that time is not possible. On the other hand, as shown in Figure 11(b), if the baseline intersects with the transverse line L of the target area, threshold evaluation of the α angle using the ultrasound image at that time is possible.

[0058] The user should operate the ultrasound probe to obtain an ultrasound image that allows for threshold evaluation of the alpha angle, as shown in Figure 11(b). Once an ultrasound image like the one shown in Figure 11(b) is obtained, the user should operate the control unit 110 or the ultrasound probe to store the image in the storage unit 140 and provide it for final diagnosis by a physician. The final diagnosis should be made by a physician, taking into account other findings besides the alpha angle. The above describes the configuration of the diagnostic imaging device 100.

[0059] (Learning Process) Next, with reference to the flowchart in Figure 12, the flow of the learning process performed by the control unit 150 of the image diagnostic device 100 according to Embodiment 1 will be explained. The learning process is the process of generating a first trained model based on first training data. The learning process starts when the user stores the first training data in the training data storage unit 141 and operates the operation unit 110 to instruct the start of the learning process.

[0060] First, the learning unit 151 selects the unlearned first learning data from the first learning data dataset stored in the learning data storage unit 141 (step S11).

[0061] Next, the learning unit 151 adjusts the weights between each node of the neural network shown in Figure 5 based on the first training data selected in step S11 (step S12). Specifically, it inputs the pixel values ​​of each pixel of the input image in the selected first training data dataset to the input node of the input layer, thereby outputting the pixel values ​​(estimated values) of each pixel of the output image output from the output node of the output layer.

[0062] Next, the Mean Squared Error (MSE) is calculated based on the estimated value and the pixel value of each pixel in the label image from the first training data, where the bounding box was drawn manually. MSE is an example of an evaluation function. If the MSE is above a threshold, the weights between each node are adjusted so that the MSE falls below the threshold. A suitable update algorithm for reducing the MSE is, for example, gradient descent.

[0063] Next, the learning unit 151 determines whether the learning using the first learning data has been completed (step S13). Specifically, it determines whether there is any unlearned first learning data remaining. If it is determined that the learning using the first learning data has been completed (step S13; Yes), the process moves to step S14. On the other hand, if it is determined that the learning using the first learning data has not been completed (step S13; No), the process returns to step S11.

[0064] If the result in step S13 is Yes, the learning unit 151 stores the weights between each node optimized in step S12 in the trained model storage unit 142 (step S14), and terminates the process. This concludes the flow of the learning process.

[0065] (Image Diagnosis Processing) The flow of image diagnosis processing performed by the control unit 150 of the image diagnosis device 100 according to Embodiment 1 will be described below with reference to the flowchart in Figure 13. Image diagnosis processing is a process that applies image processing to ultrasound images taken by the ultrasound examination device 2 to provide information useful for diagnosing whether or not a hip joint disease exists in the subject. Image diagnosis processing is started, for example, when the user operates the operation unit 110 to instruct the start of image diagnosis processing.

[0066] First, the acquisition unit 152 acquires the latest ultrasound image data from the ultrasound examination device 2 and stores it in the image data storage unit 143 in association with the image ID (step S21). The image ID is an identification number generated in the order in which the image data is acquired.

[0067] Next, the bony acetabular line detection unit 153 uses the first trained model stored in the trained model storage unit 142 to detect the bony acetabular line present in the ultrasound image acquired in step S21 (step S22). Specifically, when the pixel value of each pixel in the ultrasound image is input to each node of the input layer of the first trained model, the pixel value of each pixel in the output image is output from each node of the output layer. If a bony acetabular line is present in the ultrasound image, a bounding box surrounding the bony acetabular line is drawn on the output image. When a bony acetabular line is detected, as shown in Figure 7, the upper left vertex of the bounding box surrounding the bony acetabular line corresponds to the bony acetabular beak (symbol ★), so the pixel position of the bony acetabular beak can also be identified at the same time. The pixel positions of the upper left and lower right vertices of the drawn bounding box are stored in the image data storage unit 143 in Figure 6, respectively.

[0068] Next, the baseline detection unit 154 performs a baseline detection process (step S23) to detect the baseline by applying image processing to the ultrasound image acquired in step S21. The flow of the baseline detection process will be explained below with reference to Figure 14.

[0069] (Baseline detection process) First, the baseline detection unit 154 extracts local maxima in the ultrasound image acquired in step S21 (step S31). The local maxima are distributed on the upper side of the bony acetabular beak, as shown in Figure 8.

[0070] Next, the baseline detection unit 154 removes local maxima points that are unnecessary for detecting the iliac wall from the set of local maxima points extracted in step S31 (step S32). Specifically, local maxima points that are outside the detection area set based on the pixel position (pX, pY) of the bony acetabular canusa should be removed.

[0071] Next, the baseline detection unit 154 performs a linear approximation based on the set of local maxima points remaining from the processing in step S32, and generates a straight line corresponding to the outer wall of the iliac crest shown in Figure 9 (step S33). For the linear approximation, for example, the least squares method may be used.

[0072] Next, the baseline detection unit 154 generates a baseline (step S34) that is parallel to the line corresponding to the outer wall of the iliac crest shown in Figure 10 and passes through the bony acetabular beak, based on the bony acetabular beak detected in step S22 and the line corresponding to the outer wall of the iliac crest obtained in step S33, and returns the process. This completes the baseline detection process.

[0073] Returning to Figure 13, the angle calculation unit 155 calculates the α angle based on the bony acetabular line detected in step S22 and the baseline detected in step S23 (step S24). The calculated α angle is stored in the image data storage unit 143 in Figure 6.

[0074] Next, the output unit 156 generates a diagnostic result screen based on the ultrasound image of the subject acquired in step S21, the bony acetabular line detected in step S22, the baseline detected in step S23, and the α angle calculated in step S24, and displays it on the display unit 120 and transmits it to the ultrasound examination device 2 (step S25). Specifically, first, a detection image is generated on the ultrasound image of the subject with the bony acetabular line, baseline, and target area drawn on it, and then a diagnostic result screen including the generated detection image and the α angle is generated.

[0075] Next, the acquisition unit 152 determines whether it has received an instruction from the user to stop the diagnosis (step S26). If it is determined that an instruction to stop the diagnosis has been received (step S26; Yes), the process ends. On the other hand, if it is determined that an instruction to stop the diagnosis has not been received (step S26; No), the process returns to step S21.

[0076] When the user starts imaging with the ultrasound probe, they refer to the diagnostic results screen displayed on the display unit 120 or the display of the ultrasound examination device 2 in step S25, which is updated repeatedly at regular intervals. The user adjusts the tilt of the ultrasound probe from a state where the baseline does not intersect with the horizontal line L on the opposite side of the bony acetabular canusa in the target area's frame, as shown in Figure 11(a), to a state where the baseline intersects with the horizontal line L of the target area, as shown in Figure 11(b). This allows the user, such as a physician, to quickly determine whether they have obtained an appropriate ultrasound image necessary for the final diagnosis. The above is the flow of the image diagnostic processing.

[0077] As described above, the image diagnostic device 100 according to Embodiment 1 includes a bony acetabular line detection unit 153 that detects the bony acetabular line in the ultrasound image based on the pixel value of each pixel in the ultrasound image acquired by the acquisition unit 152, and a baseline detection unit 154 that detects a straight line indicating the outer wall of the iliac crest in the ultrasound image based on the pixel value of each pixel in the ultrasound image acquired by the acquisition unit 152, and detects a baseline that is parallel to the detected straight line indicating the outer wall of the iliac crest and passes through the bony acetabular beak at the end of the bony acetabular line detected by the bony acetabular line detection unit 153. Therefore, the bony acetabular line and baseline on the ultrasound image can be detected quickly, and as a result, diagnostic results can be provided to the user quickly. Furthermore, the bony acetabular beak can be detected with high accuracy, which can improve the accuracy of the diagnostic results.

[0078] (Embodiment 2) Next, with reference to Figure 15, an image diagnostic apparatus, image diagnostic method, and program according to Embodiment 2 will be described. In Embodiment 1, a rectangular detection area based on the bony acetabular canusa is set on the ultrasound image during baseline detection processing, but in Embodiment 2, a polygonal region surrounding the outer wall of the iliac crest is set instead of a rectangular detection area. The differences between the two will be explained below.

[0079] The training data storage unit 141 stores a second training data set containing multiple datasets used as training data for machine learning. As shown in Figure 15, the second training data set contains one ultrasound image of a subject obtained by an ultrasound probe (input image) and one labeled image (output image) that has been manually segmented to surround the outer wall of the iliac crest. In segmentation, data labeling is performed to define a polygon region surrounding the outer wall of the iliac crest. The polygon region is represented by the pixel positions of the vertices that make up the polygon in the ultrasound image. Data augmentation may also be performed on the second training data as a preprocessing step for machine learning in order to increase the number of datasets.

[0080] The trained model storage unit 142 stores a second trained model generated by machine learning based on the second training data. The second trained model is a model that outputs the pixel value of each pixel in the output image in response to the pixel value of each pixel in the input image, and uses a multi-layered neural network consisting of multiple interconnected nodes as shown in Figure 5. The trained model storage unit 142 stores optimized weights that indicate the strength of the connections between the nodes constituting the neural network. For example, YOLO can be used as the second trained model.

[0081] The image data storage unit 143 stores the pixel positions of multiple vertices that identify the polygon region drawn using the second trained model, associating them with the image data.

[0082] The learning unit 151 performs machine learning using the second training data stored in the training data storage unit 141, and stores the generated second trained model in the trained model storage unit 142.

[0083] The baseline detection unit 154 extracts multiple local maxima points in the ultrasound image acquired by the acquisition unit 152 where the pixel value is locally at its maximum, and deletes local maxima points that are unnecessary for detecting the iliac wall from the extracted set of local maxima points. In this process of deleting unnecessary local maxima points, a polygon region set using a second trained model is used.

[0084] The procedure for deleting unnecessary local maxima will be explained in detail. First, a polygon region is set on the ultrasound image using the second trained model. At this time, when the pixel value of each pixel in the ultrasound image is input to each node of the input layer of the second trained model, the pixel value of each pixel in the output image is output from each node of the output layer. If the iliac wall is present in the ultrasound image, a polygon region surrounding the iliac wall is drawn in the output image. Next, local maxima outside the polygon region are deleted as local maxima that are unnecessary for detecting the iliac wall. The multiple local maxima remaining within the polygon region can be used to obtain a straight line corresponding to the iliac wall using the same procedure as in Embodiment 1. By using this procedure, the local maxima necessary for detecting the iliac wall can be narrowed down more appropriately.

[0085] As described above, the image diagnostic apparatus 100 according to Embodiment 2 includes a baseline detection unit 154 that extracts a plurality of local maxima points in the ultrasound image acquired by the acquisition unit 152 where the pixel value is locally at its maximum, sets a polygon region surrounding the iliac wall in the ultrasound image based on the pixel value of each pixel in the ultrasound image acquired by the acquisition unit 152, and detects a straight line representing the iliac wall based on the set of local maxima points within the polygon region from the set of extracted local maxima points. Therefore, the baseline can be detected more accurately, and as a result, the accuracy of the α angle can be improved.

[0086] (Embodiment 3) Next, with reference to Figures 16 to 24, the image diagnostic apparatus, image diagnostic method, and program according to Embodiment 3 will be described. In Embodiments 1 and 2, the α angle was calculated as a parameter useful for diagnosing hip joint diseases, but in Embodiment 3, in addition to the α angle, the β angle and FHC (Femoral Head Coverage) are also calculated. The differences between the two will be explained below.

[0087] As shown in Figure 16, the storage unit 140 of the medical image device 100 further includes a parameter storage unit 144 in addition to the training data storage unit 141, the trained model storage unit 142, and the image data storage unit 143. The training data storage unit 141 further stores a third training data set and a fourth training data set, each containing multiple datasets used as training data for machine learning.

[0088] As shown in Figure 17, the third training data set includes one ultrasound image of the subject obtained by an ultrasound probe (input image) and one labeled image (output image) in which a bounding box is manually drawn around the cartilage roof line on the ultrasound image. As shown in Figure 18, this bounding box is a rectangular box with the ends of the cartilage roof line as its diagonal vertices, and its position and size are determined, for example, by the pixel positions of these diagonal vertices. The cartilage roof line is a straight line connecting the bony acetabular canusa and the center of the labrum. As shown in Figure 19, the angle between the cartilage roof line and the baseline is the β angle.

[0089] As shown in Figure 20, the fourth training data set includes one ultrasound image of a subject obtained by an ultrasound probe (input image) and one labeled image (output image) on which two bounding boxes BA and BB are manually drawn. As shown in Figure 21, bounding box BA is a rectangular box with the bony acetabular canusa and the innermost point of the femoral head (medial point of the femoral head) as its diagonal vertices. Bounding box BB is a rectangular box with the bony acetabular canusa and the outermost point of the femoral head (lateral point of the femoral head) as its diagonal vertices. One side of each bounding box BA and BB overlaps with the other. The position and size of each bounding box BA and BB are determined by the pixel position of the diagonal vertex.

[0090] By estimating the boundaries of each boundary box BA and BB on the ultrasound image, the lengths Xa and Xb necessary for calculating the FHC can be obtained. FHC, also known as femoral head coverage, represents the ratio of the femoral head covered by the hip acetabulum. A lower FHC indicates greater hip joint instability. As shown in Figure 20, length Xa is the length of one side extending in the X-axis direction of boundary box BA, and length Xb is the sum of the lengths of the two sides extending in the X-axis direction in the two boundary boxes BA and BB, respectively. Using lengths Xa and Xb, FHC can be expressed by the following formula: FHC = Xa / Xb × 100 …(1)

[0091] The trained model storage unit 142 stores a third trained model generated by machine learning based on the third training data, and a fourth trained model generated by machine learning based on the fourth training data. Both the third and fourth trained models are models that output the pixel value of each pixel in the output image in response to the pixel value of each pixel in the input image, and use a multi-layered neural network consisting of multiple interconnected nodes as shown in Figure 5. The trained model storage unit 142 stores optimized weights that indicate the strength of the connections between the nodes constituting the neural network. For example, YOLO can be used as the third and fourth trained models.

[0092] As shown in Figure 22, the image data storage unit 143 stores, in association with the ultrasound image, an image ID unique to the ultrasound image, and the pixel positions of a pair of vertices on the diagonal of each bounding box drawn using the first, third, and fourth trained models. Each bounding box drawn using the first, third, and fourth trained models uses the bony acetabular beak as its vertex. Therefore, the minimum pixel positions required to identify the position of each bounding box are five: the upper left vertex (bony acetabular beak) and the lower right vertex (inferior end of the ilium) of the bounding box used by the first trained model, the lower left vertex (center point of the labrum) of the bounding box used by the third trained model, and the lower right vertex (medial point of the femoral head) of bounding box BA and the lower left vertex (lateral point of the femoral head) of bounding box BB used by the fourth trained model.

[0093] As shown in Figure 23, the parameter storage unit 144 stores the α angle, β angle, and FHC calculated by the control unit 150 based on each ultrasound image, associating them with the image ID corresponding to each ultrasound image.

[0094] Returning to Figure 16, the control unit 150 functionally further comprises a cartilaginous acetabular line detection unit 157 and an FHC calculation unit 158. The learning unit 151 performs machine learning using the third and fourth learning data stored in the learning data storage unit 141, respectively, and stores the generated third and fourth trained models in the trained model storage unit 142.

[0095] The cartilaginous acetabular line detection unit 157 uses a third trained model stored in the trained model storage unit 142 to detect the cartilaginous acetabular line in the ultrasound image of the subject acquired by the acquisition unit 152. Specifically, when the pixel value of each pixel in the ultrasound image is input to each node of the input layer of the third trained model, the pixel value of each pixel in the output image is output from each node of the output layer. If a cartilaginous acetabular line is present in the ultrasound image, a bounding box with the cartilaginous acetabular line as the diagonal is drawn in the output image, as shown in Figure 17.

[0096] The angle calculation unit 155 calculates the α angle and also calculates the β angle based on the cartilaginous acetabular line detected by the cartilaginous acetabular line detection unit 157 and the baseline detected by the baseline detection unit 154. The calculated α angle and β angle are stored in the parameter storage unit 144. The user can determine that if the β angle is below the threshold, the infant's hip joint is normal, and if the β angle is above the threshold, the infant's hip joint has a developmental defect, such as acetabular dysplasia. The threshold for the β angle is, for example, 55°.

[0097] The FHC calculation unit 158 ​​uses the fourth trained model stored in the trained model storage unit 142 to calculate the FHC of the femoral head in the ultrasound image of the subject acquired by the acquisition unit 152. The procedure for calculating the FHC will be described in detail below.

[0098] First, two bounding boxes BA and BB are drawn on the ultrasound image using the fourth pre-trained model. Specifically, the pixel values ​​of each pixel in the ultrasound image are input to each node in the input layer of the fourth pre-trained model, and the pixel values ​​of each pixel in the output image are output from each node in the output layer. If the femoral head is present in the ultrasound image, two bounding boxes BA and BB are drawn on the output image as shown in Figure 20.

[0099] Next, the lengths Xa and Xb of the femoral head are calculated based on the drawn boundary boxes BA and BB. Then, the FHC is calculated based on the lengths Xa and Xb calculated using the above formula (1) and stored in the parameter storage unit 144. The user can determine that if the FHC is above the threshold, the hip joint of the infant being examined is normal, and if the FHC is below the threshold, there is a developmental defect in the infant's hip joint, such as acetabular dysplasia. The threshold for FHC is, for example, 50%.

[0100] The output unit 156 generates a detection image in real time, which, in addition to the bony acetabular line and baseline, also displays the cartilaginous acetabular line detected by the cartilaginous acetabular line detection unit 157 on the ultrasound image of the subject acquired by the acquisition unit 152. It also outputs a diagnostic result screen in real time to the outside, which includes the detection image, the α angle and β angle calculated by the angle calculation unit 155, and the FHC calculated by the FHC calculation unit 158. The above is the configuration of the diagnostic imaging device 100.

[0101] (Image diagnostic processing) Next, with reference to Figure 24, the flow of image diagnostic processing performed by the image diagnostic device 100 according to Embodiment 3 will be explained. First, the control unit 150 sequentially executes the processes of steps S21 to S23.

[0102] Next, the cartilaginous acetabular line detection unit 157 uses the third trained model stored in the trained model storage unit 142 to detect the cartilaginous acetabular line in the ultrasound image acquired in step S21 (step S23A). Specifically, when the pixel value of each pixel in the ultrasound image is input to each node of the input layer of the third trained model, the pixel value of each pixel in the output image is output from each node of the output layer. If a cartilaginous acetabular line is present in the ultrasound image, a bounding box surrounding the cartilaginous acetabular line is drawn on the output image as shown in Figure 17. The pixel position of the center point of the labrum, which is the lower left vertex of the bounding box, is stored in the image data storage unit 143 as shown in Figure 22.

[0103] Next, the angle calculation unit 155 calculates the α angle and β angle in the ultrasound image based on the bony acetabular line calculated in step S22, the baseline calculated in step S23, and the cartilaginous acetabular line calculated in step S23A (step S24). The calculated α angle and β angle are stored in the parameter storage unit 144 shown in Figure 23.

[0104] Next, the FHC calculation unit 158 ​​uses the fourth trained model stored in the trained model storage unit 142 to calculate the FHC of the femoral head in the ultrasound image acquired in step S21 (step S24A). Specifically, when the pixel value of each pixel in the ultrasound image is input to each node of the input layer of the fourth trained model, the pixel value of each pixel in the output image is output from each node of the output layer. If a femoral head is present in the ultrasound image, two bounding boxes BA and BB are drawn on the output image as shown in Figure 20. Next, the lengths Xa and Xb of the femoral head are calculated based on each bounding box BA and BB, and the FHC based on the lengths Xa and Xb is calculated using the above formula (1). The pixel positions of the medial and lateral points of the femoral head, which are the vertices of each bounding box BA and BB, are stored in the image data storage unit 143 in Figure 22, and the FHC is stored in the parameter storage unit 144 in Figure 23.

[0105] Next, the output unit 156 generates a diagnostic result screen based on the ultrasound image of the subject acquired in step S21, the bony acetabular line detected in step S22, the baseline detected in step S23, the cartilaginous acetabular line detected in step S23A, the α angle and β angle calculated in step S24, and the FHC calculated in step S24A, and displays it on the display unit 120 and transmits it to the ultrasound examination device 2 (step S25).

[0106] Next, the process in step S26 is executed, and the process is either terminated or returned to step S21. This concludes the flow of the image diagnostic processing.

[0107] As described above, the image diagnostic device 100 according to Embodiment 3 includes: a cartilaginous acetabular line detection unit 157 that detects the cartilaginous acetabular line in the ultrasound image based on the pixel value of each pixel of the ultrasound image acquired by the acquisition unit 152; an angle calculation unit 155 that calculates the β angle, which is the angle between the cartilaginous acetabular line detected by the cartilaginous acetabular line detection unit 157 and the baseline detected by the baseline detection unit 154; and an FHC calculation unit 158 ​​that calculates the FHC of the femoral head in the ultrasound image based on the pixel value of each pixel of the ultrasound image acquired by the acquisition unit 152. Therefore, physicians can understand the condition of the patient's hip joint by considering the α angle, β angle, and FHC, further supporting accurate diagnosis by physicians.

[0108] (Embodiment 4) Next, with reference to Figures 25 and 26, the imaging diagnostic device, imaging diagnostic method, and program according to Embodiment 4 will be described. In Embodiments 1 to 3, the straight line connecting the bony acetabular beak and the lower end of the ilium was defined as the bony acetabular line, and the straight line parallel to the outer wall of the ilium and passing through the bony acetabular beak was defined as the baseline. In Embodiment 4, as shown in Figure 25, the straight line connecting the lower end of the outer wall of the ilium (bony roof point) and the lower end of the ilium is defined as the bony acetabular line, and the straight line parallel to the outer wall of the ilium and passing through the lower end of the outer wall of the ilium is defined as the baseline. The lower end of the outer wall of the ilium is the lowest point of the outer wall (outer edge) of the ilium. The reason for defining the bony acetabular line and baseline in this way is that in newborn infants, the bony acetabular beak and the lower end of the outer wall of the ilium are separated. The differences between the two will be explained below.

[0109] As shown in Figure 26, the first training data set includes one ultrasound image of the subject obtained by an ultrasound probe (input image) and one labeled image (output image) in which a boundary box is manually drawn around the bony acetabular line on the ultrasound image. Here, the bony acetabular line is a straight line connecting the lower end of the outer wall of the ilium and the lower end of the ilium.

[0110] The bony acetabular line detection unit 153 uses a first trained model stored in the trained model storage unit 142 to detect the bony acetabular line in the ultrasound image of the subject acquired by the acquisition unit 152. When the bony acetabular line is detected, the pixel position from the upper left vertex of the boundary box to the lower end of the iliac outer wall is obtained.

[0111] The baseline detection unit 154 detects the baseline in the ultrasound image of the subject acquired by the acquisition unit 152, based on the lower end of the iliac outer wall detected by the bony acetabular line detection unit 153. Here, the baseline is a straight line parallel to the iliac outer wall and passing through the lower end of the iliac outer wall. The procedure for detecting the baseline can be the same as that used in any of embodiments 1 to 3.

[0112] The output unit 156 generates a detection image. The detection image draws a rectangular target area set on the iliac outer wall side, with the lower end of the iliac outer wall as the reference point. The target area is defined by a pair of opposing horizontal lines extending in the X-axis direction and a pair of opposing vertical lines extending in the Y-axis direction. The lower horizontal line of the target area is set so that its midpoint is located at the pixel position (pX, pY) of the lower end of the iliac outer wall.

[0113] As described above, the image diagnostic device 100 according to Embodiment 4 includes a bony acetabular line detection unit 153 that detects the bony acetabular line in the ultrasound image based on the pixel value of each pixel in the ultrasound image acquired by the acquisition unit 152, and a baseline detection unit 154 that detects a straight line indicating the outer wall of the iliac crest in the ultrasound image based on the pixel value of each pixel in the ultrasound image acquired by the acquisition unit 152, and detects a baseline that is parallel to the detected straight line indicating the outer wall of the iliac crest and passes through the lower end of the outer wall of the iliac crest at the end of the bony acetabular line detected by the bony acetabular line detection unit 153. Therefore, the bony acetabular line can be accurately detected even in newborn infants, and as a result, the accuracy of the α angle can be improved.

[0114] The present invention is not limited to the embodiments described above, and the following modifications are also possible.

[0115] (Modification) In the above embodiment, the position and size of the bounding box were determined from the pixel positions of the opposing vertices, but the present invention is not limited thereto. For example, the position and size may be characterized by the position of the center point of the bounding box and the width and height of the frame.

[0116] In the above embodiment, the optimized weights obtained by the learning process using the training data were stored directly in the trained model storage unit 142, but the present invention is not limited to this. For example, after performing the learning process using a portion of the training data, the validity of the optimized weights may be evaluated using the remaining training data, and if the weights are evaluated as valid, they may be stored in the trained model storage unit 142.

[0117] In the above embodiment, the medical image diagnostic device 100 was equipped with the functions of the learning unit 151, but the present invention is not limited thereto. For example, a separate learning device from the medical image diagnostic device 100 may be equipped with the functions of the learning unit 151. In this case, the learning device generates a trained model by learning the relationship between input images and output images based on training data, and the medical image diagnostic device 100 can acquire the trained model generated by the learning device.

[0118] In the above embodiment, the weights of the neural network were optimized by determining whether the evaluation function, MSE, was below a threshold; however, the present invention is not limited to this. A function other than MSE may be used as the evaluation function. Furthermore, the weights may be determined to be optimized when the number of training iterations reaches an upper limit.

[0119] In the above embodiment, the learning unit 151 generated a trained model using a neural network, but the present invention is not limited to this. As long as the relationship between the input image and the output image is defined and input data is input to the input layer and output data is obtained from the output layer, models and algorithms constructed by other methods may be used. For example, a model using HoG (Histgrams of Oriented Gradients) features may be used.

[0120] In the above embodiment, the bony acetabular line, the cartilaginous acetabular line, and the baseline were drawn on the ultrasound image, but the present invention is not limited thereto. The bony acetabular line, the cartilaginous acetabular line, and the baseline may be set in a computer so that the α angle and β angle can be calculated. In this case, it is not necessarily required to illustrate the bony acetabular line, the cartilaginous acetabular line, and the baseline on the ultrasound image.

[0121] In the above embodiment, a diagnostic result screen including a detection image in which the bony acetabular line and baseline are drawn on the ultrasound image, and the α angle, was displayed on the display unit 120 of the image diagnostic device 100 and the display of the ultrasound examination device 2. However, the present invention is not limited thereto. For example, in order to simplify the screen configuration, the display of at least one of the α angle, β angle, and FHC may be omitted on at least one of the display units 120 and the display of the ultrasound examination device 2. Alternatively, the control unit 150 may functionally include a determination unit, which determines the presence or absence of hip joint disease in the subject based on at least one of the α angle, β angle, and FHC, and the determination result may be displayed on at least one of the display units 120 and the display of the ultrasound examination device 2.

[0122] In the above embodiment 3, the α angle and β angle are calculated after detecting the cartilaginous acetabular line in the ultrasound image, but the present invention is not limited thereto. The α angle may be calculated before detecting the cartilaginous acetabular line.

[0123] In the above embodiment 3, the FHC is calculated after the α angle and β angle are calculated, but the present invention is not limited thereto. The FHC may be calculated at any point after the bony acetabular line has been detected, for example, the α angle and β angle may be calculated after the FHC has been calculated.

[0124] In the above embodiment 3, the α angle, β angle, and FHC are calculated, but the present invention is not limited thereto. For example, in order to reduce the computational load on the medical imaging device 100, the calculation process of at least one of the α angle, β angle, and FHC may be omitted.

[0125] In the above embodiment, the ultrasound examination device 2 and the image diagnostic device 100 were configured as separate units, but the present invention is not limited thereto. For example, the image diagnostic device 100 may be omitted, and the functions of the image diagnostic device 100 may be incorporated into the ultrasound examination device 2.

[0126] In the above embodiment, various types of data were stored in the storage unit 140 of the medical imaging device 100, but the present invention is not limited thereto. For example, all or part of the various types of data may be stored in an external computer or data logger via a communication network.

[0127] In the above embodiment, the medical imaging device 100 operated based on a program stored in the storage unit 140, but the present invention is not limited thereto. For example, a functional configuration realized by a program may be realized by hardware.

[0128] In the above embodiment, the medical imaging device 100 was a general-purpose computer, but the present invention is not limited thereto. For example, the medical imaging device 100 may be implemented as a computer located on the cloud.

[0129] In the above embodiment, the processing performed by the image diagnostic device 100 was realized by the device having the above-described physical configuration executing a program stored in the storage unit. However, the present invention may be realized as a program, or as a storage medium on which that program is recorded.

[0130] Alternatively, a device that performs the above-mentioned processing operations may be configured by distributing a program for executing the above-mentioned processing operations on a computer-readable non-temporary recording medium such as a flexible disk, CD-ROM (Compact Disk Read-Only Memory), DVD (Digital Versatile Disk), or MO (Magneto-Optical Disk), and then installing that program on a computer.

[0131] The embodiments described above are illustrative, and the present invention is not limited thereto. Various embodiments are possible without departing from the spirit of the invention as described in the claims. The components described in the embodiments and modifications can be freely combined. Furthermore, inventions equivalent to the invention described in the claims are also included in the present invention.

[0132] The present invention will be specifically described below with reference to examples. However, the present invention is not limited to these examples.

[0133] (Example 1) In Example 1, we verified whether the α angle could be calculated using the method according to Embodiment 1, and compared the results with the calculation results using the method in Patent Document 1. The same computer was used for all calculations. As a result, the method according to Embodiment 1 was able to reduce the processing time by about 100 msec to 200 msec compared to the method in Patent Document 1, and succeeded in calculating the α angle almost instantly. This is because the method according to Embodiment 1 reduces the number of calculation steps by one compared to the method in Patent Document 1.

[0134] Furthermore, verification was conducted using a phantom by 10 examiners, and the measurement errors in both methods were compared. As a result, the method according to Embodiment 1 improved the error of the iliac outer plate from 2.3° to 0.0° and the acetabular angle error from 3.6° to 0.4°, and as a result, the disease detection rate was improved from 92.5% (37 / 40) to 100% (40 / 40).

[0135] (Example 2) In Example 2, we verified whether the baseline detection accuracy could be further improved using the method according to Embodiment 2. The same computer used in Example 1 was also used in this verification. As a result, it was found that when using the method according to Embodiment 2, the baseline could be detected accurately even when the outer wall of the iliac crest was curved and inclined, and consequently the accuracy of the α angle and β angle also improved. Furthermore, compared to the case where the method according to Embodiment 1 was used, the sensitivity of image extraction and the accuracy of diagnostic suitability image extraction increased by 8% to 23% (81% to 87%). The reason for this is that in the method according to Embodiment 1, the rectangular target area is set independently of the shape of the iliac crest. Therefore, if the iliac crest is curved or has a large inclination with respect to the Y-axis, local maxima indicating areas other than the iliac crest will fall within the target area, reducing the accuracy of baseline detection. However, in the method according to Embodiment 2, the polygon area represents the shape contour of the iliac crest. Therefore, regardless of the shape or inclination of the iliac crest, local maxima indicating areas other than the iliac crest will not fall within the target area.

[0136] This application is based on Japanese Patent Application No. 2024-157479, filed on 11 September 2024, and includes its specification, claims, drawings, and abstract. The disclosures in the aforementioned Japanese Patent Application are incorporated herein by reference in their entirety.

[0137] The present invention provides useful diagnostic imaging devices, diagnostic imaging systems, diagnostic imaging methods, and programs because they can quickly provide users with diagnostic results based on ultrasound images of the hip joint.

[0138] 1. Image diagnostic system 2. Ultrasound examination device 100. Image diagnostic device 110. Operation unit 120. Display unit 130. Communication unit 140. Storage unit 141. Learning data storage unit 142. Trained model storage unit 143. Image data storage unit 144. Parameter storage unit 150. Control unit 151. Learning unit 152. Acquisition unit 153. Bony acetabular line detection unit 154. Baseline detection unit 155. Angle calculation unit 156. Output unit 157. Cartilaginous acetabular line detection unit 158. FHC calculation unit

Claims

1. An imaging diagnostic device comprising: an acquisition unit that acquires an ultrasound image of the hip joint of a subject; a bony acetabular line detection unit that detects a bony acetabular line in the ultrasound image based on the pixel value of each pixel in the ultrasound image acquired by the acquisition unit; a baseline detection unit that detects a straight line indicating the outer wall of the iliac crest in the ultrasound image based on the pixel value of each pixel in the ultrasound image acquired by the acquisition unit, and detects a baseline that is parallel to the detected straight line indicating the outer wall of the iliac crest and passes through the bony acetabular beak or the lower end of the outer wall of the iliac crest at the end of the bony acetabular line detected by the bony acetabular line detection unit; and an output unit that outputs a detection image in which the bony acetabular line detected by the bony acetabular line detection unit and the baseline detected by the baseline detection unit are drawn on the ultrasound image acquired by the acquisition unit.

2. The diagnostic imaging apparatus according to claim 1, wherein the baseline detection unit extracts a plurality of local maximum points in the ultrasound image acquired by the acquisition unit in which the pixel value is locally maximum, sets a polygon region surrounding the outer wall of the iliac crest in the ultrasound image based on the pixel value of each pixel in the ultrasound image acquired by the acquisition unit, and detects a straight line representing the outer wall of the iliac crest based on the set of local maximum points that are located within the polygon region from the set of extracted local maximum points.

3. The image diagnostic apparatus according to claim 1, further comprising an angle calculation unit that calculates the α angle, which is the angle between the bony acetabular line detected by the bony acetabular line detection unit and the baseline detected by the baseline detection unit, and the output unit outputs the α angle calculated by the angle calculation unit.

4. The image diagnostic apparatus according to claim 3, further comprising a determination unit that determines whether or not a hip joint disease exists in the subject based on the α angle calculated by the angle calculation unit, and the output unit outputs the determination result determined by the determination unit.

5. The diagnostic imaging apparatus according to claim 4, wherein the determination unit determines that the subject does not have acetabular dysplasia when the α angle calculated by the angle calculation unit is 60° or more.

6. The image diagnostic apparatus according to claim 3, wherein the output unit draws a rectangular target area on the detected image, which is set on the iliac outer wall side with respect to the bony acetabular beak or the lower end of the iliac outer wall detected by the bony acetabular line detection unit, and the target area is set so that when the baseline whose inclination changes due to the operation of the ultrasound probe by the user intersects with a horizontal line in the frame of the target area that is on the opposite side of the bony acetabular beak or the lower end of the iliac outer wall, the α angle calculated by the angle calculation unit can be evaluated.

7. The image diagnostic apparatus according to claim 6, wherein the target area extends upward from the bony acetabular canusa or the lower end of the iliac outer wall detected by the bony acetabular line detection unit and is set in a band shape.

8. The image diagnostic apparatus according to claim 1, wherein the bony acetabular line detection unit outputs an output image in which a boundary box surrounding the bony acetabular line is drawn on the ultrasound image acquired by the acquisition unit when a bony acetabular line is present in the ultrasound image acquired by the acquisition unit.

9. The imaging diagnostic apparatus according to claim 1, wherein the acquisition unit acquires the ultrasound image of the subject's hip joint at regular intervals, and the output unit outputs a detection image based on the ultrasound image acquired by the acquisition unit in real time.

10. The image diagnostic apparatus according to claim 1, further comprising: a cartilaginous acetabular line detection unit that detects a cartilaginous acetabular line in the ultrasound image based on the pixel value of each pixel of the ultrasound image acquired by the acquisition unit; and an angle calculation unit that calculates a β angle, which is the angle between the cartilaginous acetabular line detected by the cartilaginous acetabular line detection unit and the baseline detected by the baseline detection unit, wherein the output unit outputs the β angle calculated by the angle calculation unit.

11. The image diagnostic apparatus according to claim 1, wherein the image diagnostic apparatus detects the outermost point and the innermost point of the bony acetabular cana, the femoral head, and the bony acetabular cana, and the femoral head, in the ultrasound image based on the pixel value of each pixel of the ultrasound image acquired by the acquisition unit, and further comprises an FHC calculation unit that calculates the FHC of the femoral head, which indicates the ratio of the portion of the femoral head covered by the hip acetabulum, based on the positions of the outermost point and the innermost point of the bony acetabular cana, the femoral head, and the bony acetabular cana, the femoral head, and the output unit outputs the FHC calculated by the FHC calculation unit.

12. An image diagnostic system comprising: an image diagnostic device according to any one of claims 1 to 11; and an ultrasound examination device which is communicably connected to the image diagnostic device and transmits data of an ultrasound image of the hip joint of a subject to the image diagnostic device when an ultrasound image of the hip joint of the subject is acquired.

13. An imaging diagnostic method performed by an imaging diagnostic device, comprising: acquiring an ultrasound image of the hip joint of a subject; detecting a bony acetabular line in the ultrasound image based on the pixel value of each pixel in the acquired ultrasound image; detecting a straight line indicating the outer wall of the iliac crest in the ultrasound image based on the pixel value of each pixel in the acquired ultrasound image, and detecting a baseline that is parallel to the detected straight line indicating the outer wall of the iliac crest and passes through the bony acetabular beak or the lower end of the outer wall of the iliac crest at the end of the detected bony acetabular line; and outputting a detection image in which the detected bony acetabular line and the detected baseline are drawn on the acquired ultrasound image.

14. A program to cause a computer to function as: an acquisition means for acquiring an ultrasound image of a subject's hip joint; a bony acetabular line detection means for detecting a bony acetabular line in the ultrasound image based on the pixel value of each pixel in the ultrasound image acquired by the acquisition means; a baseline detection means for detecting a straight line indicating the outer wall of the iliac crest in the ultrasound image based on the pixel value of each pixel in the ultrasound image acquired by the acquisition means, and detecting a baseline that is parallel to the detected straight line indicating the outer wall of the iliac crest and passes through the bony acetabular beak or the lower end of the outer wall of the iliac crest at the end of the bony acetabular line detected by the bony acetabular line detection means; and an output means for outputting a detection image in which the bony acetabular line detected by the bony acetabular line detection means and the baseline detected by the baseline detection means are drawn on the ultrasound image acquired by the acquisition means.

Images