Re-imaging support device, program, and re-imaging determination support method

Abstract

To accurately determine the deviation of positioning of a joint.SOLUTION: According to a control unit of a console, a re-imaging support device determines first deviation of a predetermined area of a radiation image, determines second deviation of a predetermined area of the radiation image, and outputs re-imaging determination support information for supporting determination on whether or not to capture the radiation image again, on the basis of at least the determination of the first deviation and the determination of the second deviation.SELECTED DRAWING: Figure 6

Description

[Technical field]

[0001] The present invention relates to a re-photographing decision support device, a learning device, a re-photographing support device, a program, and a re-photographing decision support method. [Background technology]

[0002] In recent years, there has been a growing emphasis on optimizing medical radiation exposure and managing radiation doses, and there is a demand for reducing the radiation dose when taking images using radiography devices. In order to avoid unnecessary radiation exposure, there is a need to find ways to minimize the number of retakes. If the patient's posture during radiography is not appropriate, deviations in the patient's positioning will occur during radiography, making it necessary to retake the image. Although there is known technology that determines whether or not to retake the image by determining deviations in the patient's positioning, there is a need for a more accurate determination of deviations in positioning in order to avoid having to retake the image multiple times.

[0003] As a technique for determining deviation in positioning of a patient, a technique for determining deviation in positioning of a joint has been conventionally known (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent Publication No. 2021-097864 Summary of the Invention [Problem to be solved by the invention]

[0005] In the knee, for example, when imaging in a supine position, the degree of freedom of the joint is low. The positioning on the thigh side is difficult to change and is hard to alter. For this reason, in the past, the deviation in positioning at the knee was determined by determining the deviation in positioning of one area on the lower leg side. Similarly, in the ankle, for example, when imaging in a supine position, the degree of freedom of the joint is low. The positioning on the lower leg side is difficult to change and is hard to alter. For this reason, in the past, the deviation in positioning at the ankle was determined by determining the deviation in positioning of one area on the toe side.

[0006] On the other hand, since the elbow has a high degree of freedom of movement, positioning deviations are likely to occur in both the forearm and upper arm directions when determining deviations in positioning at the elbow. Therefore, even if positioning deviations in only one direction (the forearm or upper arm) are determined by determining positioning deviations in only one area as in the past, it is not possible to determine deviations in elbow positioning with high accuracy.

[0007] In addition, even though the degree of freedom of the joints of the knee and ankle is low, there is a possibility that deviations in positioning may occur in both directions (for example, the thigh side and the lower leg side for the knee, and the lower leg side and the toe side for the ankle). Therefore, in determining deviations in positioning of joints such as the knee and ankle, determining only deviations in positioning in one direction by determining deviations in positioning in only one area does not allow for highly accurate determination of deviations in positioning.

[0008] An object of the present invention is to determine deviations in joint positioning with high accuracy. [Means for solving the problem]

[0009] In order to solve the above problems, the re-photographing decision support device of the present invention comprises: a first determination means for determining a first deviation in a predetermined region of a radiographic image; a second determination means for determining a second deviation of a predetermined region of the radiation image; an output unit that outputs re-imaging decision support information that supports a decision as to whether or not to re-imaging the radiographic image based on at least the first deviation determination and the second deviation determination; has.

[0010] Further, the learning device of the present invention comprises: At least a radiographic image, information regarding a first deviation in a predetermined area of ​​the radiographic image, and information regarding a second deviation in a predetermined area of ​​the radiographic image are learned as training data.

[0011] Further, the re-photographing decision support device of the present invention comprises: a trained model in which at least a radiographic image, information regarding a first deviation in a predetermined region of the radiographic image, and information regarding a second deviation in a predetermined region of the radiographic image are trained as training data; An acquisition means for acquiring a radiation image; an output means for outputting re-imaging decision support information for supporting a decision as to whether or not to re-imaging the radiographic image based on the trained model and the radiographic image acquired by the acquisition means; has.

[0012] Further, the re-photographing support device of the present invention comprises: a first determination means for determining a first deviation in a predetermined region of a radiographic image; a second determination means for determining a second deviation of a predetermined region of the radiation image; an output unit that outputs re-imaging support information for supporting re-imaging of the radiological image based on at least the first deviation determination and the second deviation determination; has.

[0013] In addition, the program of the present invention is Computer, a first determination means for determining a first deviation of a predetermined region from the radiation image; A second determination means for determining a second deviation of a predetermined region of the radiation image; an output means for outputting re-imaging decision support information for supporting a decision as to whether or not to re-imaging the radiographic image based on at least the first deviation determination and the second deviation determination; Function as.

[0014] Further, the re-photographing decision support method of the present invention includes the steps of: a first determination step of determining a first deviation in a predetermined region of the radiographic image; a second determination step of determining a second deviation in a predetermined region of the radiographic image; an output step of outputting re-imaging decision support information for supporting a decision as to whether or not to re-imaging the radiographic image based on at least the first deviation determination and the second deviation determination; Includes. Effect of the Invention

[0015] According to the present invention, deviation in joint positioning can be determined with high accuracy. [Brief description of the drawings]

[0016] [Figure 1] 1 is a block diagram showing an example of a radiation imaging system according to an embodiment of the present invention. [Diagram 2] FIG. 2 is a schematic diagram showing a side view of an elbow joint. [Diagram 3] FIG. 2 is a schematic diagram showing a front view of an elbow joint. [Figure 4] FIG. 2A is a diagram showing an example of a radiological image used as training data for input data, and FIG. 2B is a diagram showing an example of a region map used as training data for output data. [Diagram 5] FIG. 2 is a block diagram showing a functional configuration of the console of FIG. [Figure 6] 6 is a flowchart showing the flow of a shooting control process executed by the control unit in FIG. 5. [Figure 7] FIG. 13 is a diagram showing a display example of re-photographing decision support information. [Figure 8] FIG. 13 is a diagram showing a display example of re-photographing support information. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0017] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. However, the technical scope of the present invention is not limited to the following embodiment and illustrated examples.

[0018] <1. Radiography system> First, a schematic configuration of a radiation imaging system according to this embodiment (hereinafter, referred to as a system 100) will be described. FIG. 1 is a block diagram illustrating a system 100 .

[0019] As shown in FIG. 1, the system 100 includes a radiation image capturing apparatus (hereinafter, referred to as the capturing apparatus 1) and a console 2. The system 100 according to this embodiment further includes a radiation generating device (hereinafter, a generating device 3), an image management device 4, and a learning device 5. The devices 1 to 5 are capable of communicating with each other via, for example, a communication network N (such as a local area network (LAN), a wide area network (WAN), or the Internet).

[0020] The system 100 may be installed in an imaging room, or may be configured to be movable (for example, as a medical cart). Furthermore, the system 100 may be capable of communicating with a hospital information system (HIS), a radiology information system (RIS), and the like, which are not shown.

[0021] [1-1. Radiation Generator] The generating device 3 includes a generator 31 , an irradiation instruction switch 32 , and a radiation source 33 .

[0022] The generator 31 applies a voltage corresponding to preset imaging conditions to a radiation source 33 (tube) based on the operation of an irradiation instruction switch 32 .

[0023] When a voltage is applied from the generator 31, the radiation source 33 generates radiation R (for example, X-rays) with a dose corresponding to the applied voltage.

[0024] Moreover, the generating device 3 according to this embodiment is adapted to generate radiation R in a manner that corresponds to the form of the radiation image to be generated (a still image, a dynamic image having a plurality of frames). In the case of a still image, radiation R is irradiated only once each time the irradiation instruction switch 32 is pressed. In the case of dynamic images, each time the irradiation instruction switch 32 is pressed, irradiation of pulsed radiation R is repeated multiple times per predetermined time (for example, 15 times per second), or irradiation of radiation R is continued for a predetermined time.

[0025] [1-2. Radiation imaging device] The imaging device 1 generates digital data of a radiation image showing an imaging region of a subject. The imaging device 1 according to this embodiment is a portable FPD (Flat Panel Detector). Specifically, although not shown in the figures, the imaging device 1 according to this embodiment includes a sensor substrate on which image pickup elements that generate an electric charge according to the radiation dose upon receiving radiation R and switch elements that store and release the electric charge are arranged two-dimensionally (in a matrix), a scanning unit that switches each switch element on / off, a readout unit that reads out the amount of electric charge released from each pixel as a signal value, a control unit that controls each unit and generates a radiographic image from the multiple signal values ​​read out by the readout unit, and a communication unit that transmits data on the generated radiographic image and various signals to other devices (the console 2, the generator 3, the image management device 4, etc.) and receives various information and various signals from the other devices.

[0026] The photographing device 1 is configured to generate image data of a still image (hereinafter, still image data) or image data of a dynamic image (hereinafter, dynamic image data) by storing and releasing electric charges and reading out signal values ​​in synchronization with the timing at which radiation R is irradiated from the generating device 3. When still image data is to be generated, a radiation image is generated only once for each depression of the irradiation instruction switch 32. When dynamic image data is generated, the generation of frames constituting a dynamic image is repeated multiple times per predetermined time (for example, 15 times per second) for each depression of the irradiation instruction switch 32.

[0027] The image capturing device 1 may be integrated with the generating device 3 .

[0028] [1-3. Console] The console 2 sets various imaging conditions for at least one of the imaging device 1 and the generating device 3. Moreover, the console 2 is composed of a PC, a dedicated device, etc. The shooting conditions include, for example, conditions related to the subject S (area to be shot, shooting direction, physique, etc.), conditions related to the irradiation of radiation R (tube voltage, tube current, irradiation time, current-time product (mAs value), etc.), and conditions related to image reading by the shooting device 1. The console 2 may set the imaging conditions automatically based on examination order information obtained from other systems (HIS, RIS, etc.), or may set the imaging conditions (manually) based on operations performed on the operation unit 25 by a user (e.g., a technician, etc.).

[0029] Moreover, the console 2 according to this embodiment also serves as a re-photographing decision support device and a re-photographing support device. That is, the console 2 has a function as a rephotographing decision support device that outputs rephotographing decision support information at the timing of deciding whether or not to rephotograph (at the timing when it has not yet been decided whether or not to rephotograph) to support the user in deciding whether or not to rephotograph the radiographic image generated in the radiographing device 1. The console 2 also has a function as a rephotographing support device that outputs rephotographing support information at the timing of rephotographing (at the timing when it has been decided to rephotograph) to support the user in deciding whether or not to rephotograph the radiographic image generated in the radiographing device 1.

[0030] [1-4. Image management device] The image manager 4 manages the image data generated by the image capture device 1 . The image management device 4 is a Picture Archiving and Communication System (hereinafter, PACS), an image diagnosis workstation (hereinafter, IWS), or the like.

[0031] [1-5. Learning device] The learning device 5 is a device that learns a plurality of pieces of teacher data and generates a learned model M that is used when the console 2 performs a re-photographing determination support information generation process, which will be described later. The learning device 5 is configured, for example, with a CPU (Central Processing Unit), a memory unit, a communication unit, an operation unit, a display unit, etc., and the generation of a trained model M based on teacher data is realized through cooperation between the CPU and a program stored in the memory unit. The learning device 5 generates a plurality of types of trained models M according to, for example, the imaging part and the imaging direction.

[0032] In this embodiment, the learning device 5 performs machine learning on a set of a radiographic image of an imaged joint (e.g., elbow joint, knee joint, ankle joint, etc.), information on a first deviation in a predetermined area of ​​the radiographic image, and information on a second deviation in a predetermined area of ​​the radiographic image, as teacher data, to generate a trained model M (a trained model M that estimates (determines) information on the first deviation and information on the second deviation in the predetermined area from the input radiographic image). The machine learning method is not particularly limited, and for example, deep learning or the like can be used.

[0033] Here, in the present invention, deviation refers to deviation from a reference correct position or state. The reference correct position or state is, for example, a position or state in ideal positioning that is optimal for diagnosis. For example, in a radiographic image of the lateral side of the elbow joint, in ideal positioning, the outer edge 71 of the trochlea of ​​the humerus and the capitellum edge 72 of the humerus (see FIG. 2) are in a state where they almost overlap (match). A radiographic image in which the outer edge 71 of the trochlea of ​​the humerus and the capitellum edge 72 do not overlap indicates that deviation has occurred.

[0034] The predetermined area in the present invention is an area for determining the deviation (first deviation, second deviation) in the radiographic image, and is an area determined based on the imaging part information. This area is called a deviation area in this embodiment. The predetermined area for determining the first deviation is called the first deviation area, and the predetermined area for determining the second deviation is called the second deviation area. The first displacement region and the second displacement region are the same joint region. The same joint region may be, for example, the same elbow joint region. It may also be the same ankle joint region or the same knee joint region. For example, the first displacement region in a radiographic image of the lateral side of an elbow joint may be the region on the inside of the elbow surrounded by the trochlea lateral edge 71 of the humerus and the capitellum edge 72 of the humerus (the region shown by low-density dots in FIG. 2), and the second displacement region may be the region on the outside of the elbow surrounded by the trochlea lateral edge 71 of the humerus and the capitellum edge 72 of the humerus (the region shown by high-density dots in FIG. 2) in the same elbow joint as the first displacement region.

[0035] In the present invention, the first deviation may be, for example, a deviation in a first direction, and the second deviation may be, for example, a deviation in a second direction different from the first direction. For example, the first deviation in a predetermined region in a radiographic image of the lateral surface of an elbow joint may be a deviation in the forearm direction between the outer edge 71 of the trochlea of ​​the humerus and the capitellum edge 72 of the humerus in the elbow joint region (shown by reference symbol 241k in FIG. 7), and the second deviation may be a deviation in the upper arm direction between the outer edge 71 of the trochlea of ​​the humerus and the capitellum edge 72 of the humerus in the elbow joint region (shown by reference symbol 241j in FIG. 7).

[0036] An example of an index representing the first deviation is the amount of deviation in a first direction, and an example of an index representing the second deviation is the amount of deviation in a second direction. Here, the deviation amount in the present invention refers to the amount by which the position of a specific part deviates from the standard when ideal positioning is used as the standard. When the deviation amount is zero, ideal positioning is achieved. For example, the deviation amount in the first direction of a specific area in a radiographic image of the lateral side of an elbow joint is the amount by which the outer edge 71 of the trochlea of ​​the humerus and the capitellum edge 72 of the humerus deviate toward the forearm (shown by reference symbol D1 in FIG. 2), and the deviation amount in the second direction is the amount by which the outer edge 71 of the trochlea of ​​the humerus and the capitellum edge 72 of the humerus deviate toward the upper arm (shown by reference symbol D2 in FIG. 2).

[0037] Hereinafter, the generation of the trained model M in the learning device 5 will be described using an example in which a radiological image of the lateral surface of an elbow joint is input. Fig. 2 is a schematic side view of an elbow joint, and Fig. 3 is a schematic front view of an elbow joint. As described above, in a radiographic image of the side of an elbow joint, the state where the outer edge 71 of the trochlea of ​​the humerus and the capitellum edge 72 of the humerus coincide is an ideal positioning state, but there are cases where the outer edge 71 of the trochlea of ​​the humerus and the capitellum edge 72 do not coincide and are shifted in the forearm direction (forearm side) and / or upper arm direction (upper arm side). In this embodiment, when the radiographic image is an image of the side of an elbow joint, the console 2 determines the first deviation and the second deviation by determining the deviation in the forearm direction between the outer edge 71 of the trochlea of ​​the humerus and the capitellum edge 72 of the humerus as a first deviation, and the deviation in the upper arm direction between the outer edge 71 of the trochlea of ​​the humerus and the capitellum edge 72 of the humerus as a second deviation, and generates re-photographing decision support information and re-photographing support information. The learning device 5 uses the captured radiographic image as an input and generates a trained model M for acquiring information on the first deviation and information on the second deviation used in the console 2.

[0038] The learning device 5 receives one or more input data including at least a radiological image, and output data including information regarding the deviation of a specific region of the radiological image (information regarding the first deviation and information regarding the second deviation), and by learning the set of input data and output data as teacher data, generates a learned model M that estimates the output data when input data with unknown output data is input. 4(a), the input data in the training data may include, for example, imaging site information (e.g., whether the image is of the elbow joint), left / right information, imaging conditions such as the dose at the time of imaging, image information such as pixel size, images of the affected area learned by an optical camera, etc. The more input data in the training data, the higher the estimation accuracy of the trained model M.

[0039] Examples of output data in the teacher data include a displacement area map (coordinate information of the displacement area) for each of the first displacement and the second displacement shown in FIG. 4(b), the amount of displacement, information for changing the position of the part related to the displacement (described in detail later), a rank of the degree of displacement, information on the presence or absence of an alert indicating the occurrence of a displacement, and coordinate information of the pulley shaft (73 in FIG. 2) used for displacement determination. The displacement area map may be for an area that combines the first displacement area and the second displacement area. In this case, information separating the first displacement area and the second displacement area (information on a line segment (e.g., L3 in FIG. 2) separating the two areas or information indicating whether each coordinate of the displacement area belongs to the first displacement area or the second displacement area) is also required. In addition, information on whether the radiographic image is an image of the target (e.g., whether it is an image of an elbow, whether a joint is included, etc.) may be included. Furthermore, information indicating the direction in which the center of irradiation at the time of shooting deviates from the ideal position may be included in the output data, so that information indicating the direction in which the center of irradiation at the time of shooting deviates from the ideal position (the accuracy of the position for each direction) can be output from the trained model M. The displacement area map and other information regarding the displacement are determined by an expert (radiologist or physician) for the radiological image.

[0040] In this embodiment, when a radiographic image of the side of an elbow joint is input, the learning device 5 generates a trained model M that outputs information on the first displacement area (displacement area map or information dividing the first displacement area and the second displacement area) and information on the second displacement area (displacement area map or information dividing the first displacement area and the second displacement area). The trained model M also outputs the coordinates of the pulley shaft 73. Here, the range of the displacement area (the combined area of ​​the first displacement area and the second displacement area) in the radiographic image of the lateral side of the elbow joint can be, for example, the area surrounded by the outer edge 71 of the trochlea of ​​the humerus, the capitellum edge 72 of the humerus, a line L1 drawn from the trochlear axis 73 in the uncinate fossa direction, and a line L2 drawn through the trochlear axis 73 in the longitudinal direction of the humerus 76 (toward the ulna 77), as shown in Figures 2 and 3. Of these, the area on the inside of the elbow of the line L3 drawn from the trochlear axis 73 to the end of the radius 75 (area shown by low-density dots in Figure 2) is the displacement area in the forearm direction (first displacement area), and the area on the outside of the elbow (area shown by high-density dots in Figure 2) is the displacement area in the upper arm direction (second displacement area). In this embodiment, the first and second deviations are determined based on the deviation between the outer edge 71 of the trochlea of ​​the humerus and the capitellum edge 72 of the humerus, but this is not limited to this, and the first and second deviations may also be determined based on deviations from ideal positioning between other two parts.

[0041] In addition to the above, the trained model M may be one that, when a radiological image or the like is input, outputs at least one of the amount of deviation, information for changing the position of the part related to the deviation, a judgment rank of the degree of deviation, or the presence or absence of an alert.

[0042] <2. Console details> Next, the console 2 will be described in detail. FIG. 5 is a block diagram showing the functional configuration of the console 2, and FIG. 6 is a flowchart showing the flow of processing in the console 2.

[0043] [2-1. Console configuration] As shown in FIG. 5, the console 2 is configured to include a control unit 21, a memory unit 22, a communication unit 23, a display unit 24, and an operation unit 25, and each of the units 21 to 25 is electrically connected via a bus or the like.

[0044] The control unit 21 includes a CPU (Central Processing Unit), a RAM (Random Access Memory ), ROM (Read Only Memory), etc. The ROM stores various programs executed by the CPU as well as parameters necessary for executing the programs. The CPU reads out various programs stored in the ROM, loads them into the RAM, executes various processes in accordance with the loaded programs, and centrally controls the operations of each part of the console 2. The control unit 21 executes a shooting control process, which will be described later, and thereby functions as a first determination unit, a second determination unit, an output unit, and a decision unit.

[0045] The storage unit 22 is composed of a non-volatile memory, a hard disk, or the like. The storage unit 22 is capable of storing image data of radiation images acquired from other devices (the imaging device 1, the image management device 4, etc.).

[0046] Moreover, the storage unit 22 according to this embodiment stores a plurality of trained models M. The plurality of trained models M includes, for example, a trained model M generated in the learning device 5. The storage unit 22 also stores a plurality of types of algorithms used when executing a re-photographing decision support information generation process, which will be described later. The memory unit 22 also stores, in association with the imaging part and imaging direction, information indicating the type of re-photographing decision support information generation processing to be executed on the radiation image of the imaging part and imaging direction (e.g., algorithm name), and information indicating the type of trained model M used in the processing (e.g., trained model name).

[0047] Moreover, the storage unit 22 stores the examination order information transmitted from the RIS or the like.

[0048] The communication unit 23 is composed of a communication module and the like. The communication unit 23 transmits and receives various signals and various data to and from other devices (the photographing device 1, the generating device 3, the image management device 4, and the learning device 5) connected by wire or wirelessly via the communication network N. The communication unit 23 functions as an acquisition unit.

[0049] The display unit 24 is configured with, for example, an LCD (Liquid Crystal Display), a CRT (Cathode Ray Tube), etc. The display unit 24 displays a radiation image or the like in accordance with an image signal received from the control unit 21.

[0050] The operation unit 25 includes a keyboard (cursor keys, numeric input keys, various function keys, etc.), a pointing device (mouse, etc.), a touch panel laminated on the surface of the display unit 24, etc. The operation unit 25 outputs a control signal to the control unit 21 in response to an operation performed by the user.

[0051] In addition, the console 2 may not be equipped with a display unit 24 and an operation unit 25, and may be configured to receive control signals from an input device provided separately from the console 2, for example via a communication unit 23, or output image signals to a display device (monitor) provided separately from the console 2. In addition, if the other device (such as the image management device 4) has a display unit and an operation unit, it may be configured to receive control signals from the operation unit of the other device and output image signals to the display unit of the other device (the display unit and operation unit may be shared with the other device).

[0052] [2-2. Console Operation] Next, the operation of the console 2 will be described with reference to FIG. The console 2 executes the imaging control process shown in Fig. 6. The imaging control process is executed by the CPU of the control unit 21 in cooperation with a program stored in the ROM when, for example, examination order information is selected by the operation unit 25 from the examination list screen displayed on the display unit 24.

[0053] First, the control unit 21 causes the display unit 24 to display the examination screen 241 for the selected examination order information (step S1). The examination screen 241 (see FIG. 7, for example) is provided with an imaging selection button 241a displaying the details of each imaging (imaging site, imaging direction, etc.) included in the examination order information, a setting area 241b for setting image reading conditions and image processing conditions for the selected imaging, an image display area 241c for displaying the captured radiographic image, a rejected image button 241d, an output button 241e, etc. At the stage of step S1, no radiographic image is yet displayed in the image display area 241c. Also, re-imaging decision support information (indicated by symbols 241f to 241n) described later is not displayed.

[0054] When the shooting selection button 241a is pressed on the operation unit 25 to select the shooting to be performed (shooting area, shooting direction) (step S2), the control unit 21 sets the shooting conditions (image reading conditions, radiation irradiation conditions) in the shooting device 1 and the generator 3 (step S3). For example, the control unit 21 automatically sets the imaging conditions (image reading conditions, e.g., pixel size, image size, frame rate, etc.) in the imaging device 1 based on the imaging part, imaging direction, etc. of the pressed imaging selection button 241a, and also sets the imaging conditions (radiation irradiation conditions, e.g., tube voltage (kV), tube current (mA), irradiation time (ms), etc.) in the generator 3. Alternatively, the imaging conditions (image reading conditions) for imaging to be performed may be set in the imaging device 1 in response to the user's operation of the operation unit 25 on the examination screen 241. Moreover, the radiation irradiation conditions may be set by the user from the operation panel of the generator 3.

[0055] After pressing the radiography selection button 241a and setting the radiography conditions, the user (technologist) places the subject S between the radiation source 33 of the generating device 3 and the radiography device 1, and performs positioning. Here, positioning refers to, for example, how the patient is positioned during radiography. When the user operates the irradiation instruction switch 32, the radiation generator 3 irradiates the region of the subject S to be imaged with the radiation R. The imaging device 1 generates a radiation image (still image, dynamic image) showing the imaging region when it receives radiation R from the generation device 3, and transmits the image data (still image data, dynamic image data) to the console 2.

[0056] When image data of a radiographic image is received (acquired) by the communication unit 23 (step S4), the control unit 21 displays a preview of the received radiographic image in the image display area 241c of the examination screen 241 (step S5).

[0057] Next, the control unit 21 determines the type of re-photographing decision support information generation process to be applied to the received radiographic image based on the radiographic region and the radiographic direction of the radiographic image (step S6). As described above, the memory unit 22 stores information indicating the type of re-shooting decision support information generation processing to be executed on the radiographic image of the radiographic image captured in the captured body part and the captured direction, in association with the captured body part and the captured direction. The control unit 21 determines the re-shooting decision support information generation processing to be applied to the received radiographic image based on the captured body part and the captured direction of the radiographic image. The re-photographing decision support information generation process to be applied to the received radiographic image may be determined when the photography selection button 241a is pressed and the body part to be photographed, the photography direction, etc. are set. Alternatively, the control unit 21 may not automatically determine the process, but the user may select the process by operating the operation unit 25.

[0058] Next, the control unit 21 executes the determined re-photographing decision support information generation process (step S7).

[0059] In the re-photographing decision support information generation process, the control unit 21 first inputs the received radiographic image into a trained model M corresponding to the imaging area and imaging direction of the received radiographic image, out of the multiple trained models M stored in the memory unit 22, and generates re-photographing decision support information based on the information output from the trained model M. The re-photographing decision support information is not particularly limited as long as it is information that supports a decision on whether or not to perform re-photographing. For example, the re-photographing decision support information includes information on a first deviation and information on a second deviation in a predetermined area of ​​a radiographic image. The information on deviation includes at least one of information on the direction of deviation, information on the distance (amount) of deviation, information on the area of ​​deviation, a judgment rank of the degree of deviation, and information for changing the position of a part related to the deviation. The information on the direction of deviation may include information on the angle of deviation.

[0060] The re-photographing decision support information generating process will be described below by taking as an example a case where the received radiographic image is a radiographic image of the lateral surface of an elbow joint. First, the control unit 21 inputs the received radiographic image to the trained model M corresponding to the side of the elbow joint. When the radiographic image of the side of the elbow joint is input, the trained model M determines the first and second deviation areas of the input radiographic image based on the imaging site (elbow joint in this case), determines the coordinates of the pulley shaft 73, and outputs information on the first and second deviation areas (for example, the deviation area map shown in FIG. 4(b) or information dividing the first and second deviation areas) and information on the coordinates of the pulley shaft 73. The coordinates of the pulley shaft 73 may be determined by the control unit 21 through image processing.

[0061] Next, the control unit 21 generates re-photographing decision support information based on the information output from the trained model M as follows. First, the control unit 21 determines (measures) the amount of deviation of the elbow joint in the forearm direction (the amount of deviation in the first direction) and the amount of deviation in the upper arm direction (the amount of deviation in the second direction). For example, as shown in Fig. 2, in a region on the inside of the elbow from a straight line L3 drawn from the trochlear shaft 73 to the end of the radius 75, the control unit 21 determines the maximum width D1 at which a straight line is drawn from the trochlear shaft 73 in the radial direction and perpendicularly intersects with the first deviation region (region shown by low-density dots in Fig. 2) as the deviation amount in the forearm direction. Also, in a region on the outside of the elbow from the straight line L3 drawn from the trochlear shaft 73 to the end of the radius 75, the control unit 21 determines the maximum width D2 at which a straight line is drawn from the trochlear shaft 73 in the ulna 77 direction and perpendicularly intersects with the second deviation region (region shown by high-density dots in Fig. 2) as the deviation amount in the upper arm direction.

[0062] The amount of deviation may be measured directly as described above, or indirectly. For example, the amount of deviation in the first direction and the amount of deviation in the second direction may be measured based on the width (e.g., maximum width) of the area surrounded by the capitellum edge 72 of the humerus and the medial edge 78 of the trochlea of ​​the humerus. In this case, a trained model M that outputs the area surrounded by the capitellum edge 72 of the humerus and the medial edge 78 of the trochlea of ​​the humerus using a radiographic image of the lateral surface of the elbow joint as input may be generated by the learning device 5 and stored in the storage unit 22, or the area to be measured may be determined by image processing.

[0063] Next, the control unit 21 determines the rank of the degree of deviation (determination rank) based on the determined amount of deviation and a preset threshold value. For example, the control unit 21 classifies the amount of deviation in the forearm direction as "Rank A (good)" when it is less than the threshold TH1 (lowest possibility of imaging failure), as "Rank B (tolerable)" when it is equal to or greater than the threshold TH1 and less than the threshold TH2, and as "Rank C (retake)" when it is equal to or greater than the threshold TH2 (highest possibility of imaging failure). In addition, the control unit 21 classifies the amount of deviation in the upper arm direction as "Rank A (good)" when it is less than the threshold TH11 (lowest possibility of imaging failure), as "Rank B (tolerable)" when it is equal to or greater than the threshold TH11 and less than the threshold TH12, and as "Rank C (retake)" when it is equal to or greater than the threshold TH12 (highest possibility of imaging failure). Then, the lower of the judgment ranks for the forearm direction and the upper arm direction (Rank A>Rank B>Rank C) is used as the overall judgment rank.

[0064] Furthermore, when the determination rank of the amount of deviation in the forearm direction is C, the control unit 21 generates information for changing the position of a first portion (here, for example, the outer edge 71 of the trochlea of ​​the humerus or the capitellum edge 72 of the humerus) related to the first deviation (deviation in the forearm direction) based on the amount of deviation. When the determination rank of the amount of deviation in the upper arm direction is C, the control unit 21 generates information for changing the position of a second portion (here, the outer edge 71 of the trochlea of ​​the humerus or the capitellum edge 72 of the humerus) related to the second deviation (deviation in the upper arm direction) based on the amount of deviation.

[0065] Here, the information for changing the position of the part related to the deviation is information indicating what, in which direction, and how much to move in order to move the part related to the deviation from its current position on the radiation image to the correct reference position.

[0066] For example, information for changing the position of a part related to the deviation may include information indicating the distance and direction (including the angle) to move the part from its current position to a correct reference position. For example, if the deviation in the upper arm direction is 2 cm, An example of information for changing the position of a part related to the misalignment is notification information such as "When taking the photograph again, move the outer edge of the trochlea of ​​the humerus 2 cm upward."

[0067] In addition, if there is another object that can be moved from its current position to move the area related to the deviation on the radiation image from its current position to a correct reference position, the distance and direction (including the angle) of movement of that object may be used as information for changing the position of the area related to the deviation. For example, if the part related to the deviation can be moved from the current position to a correct reference position by moving the position of the shoulder relative to the panel (imaging device 1) 2 cm upward, notification information such as "When re-imaging, please raise the height of the shoulder relative to the panel (FPD) by 2 cm" may be generated as information for changing the position of the part related to the deviation. Also, if the part related to the deviation can be moved from the current position to a correct reference position by internally rotating the irradiation center of the radiation source 33 by 5 degrees and abducting it by 2 degrees, notification information such as "When re-imaging, please internally rotate the irradiation center by 5 degrees and abduct it by 2 degrees" may be generated as information for changing the position of the part related to the deviation.

[0068] Information for changing the position of a part related to the deviation can be generated, for example, by creating a table or the like by experimentally determining in advance the relationship between the direction and amount of deviation of a part related to the deviation, the object to be moved, and the distance and direction of movement of that object, and storing the table or the like in memory unit 22. Although it is ideal for the deviation to be zero, if there is a deviation but it is within the range that is acceptable for diagnosis, there is no need to generate information for changing the position related to the deviation.

[0069] Furthermore, the control unit 21 creates a displacement area map by superimposing a predetermined color, which indicates that the displacement area, on the displacement area of ​​the radiation image, and also superimposes a marker on the position where the amount of displacement is measured.

[0070] When the generation of the re-photographing decision support information is completed, the control unit 21 outputs the generated re-photographing decision support information (step S8). For example, the control unit 21 displays the generated re-photographing decision support information on the display unit 24 (examination screen 241).

[0071] FIG. 7 is a diagram showing an example of an examination screen 241 on which a preview image of a radiation image received from the radiographing apparatus 1 and re-photographing decision support information are displayed. As shown in Fig. 7, in step S8, the received radiographic image is previewed in the image display area 241c of the examination screen 241. Also, re-imaging decision support information is displayed. In Fig. 7, as the re-imaging decision support information, a judgment rank 241f, a deviation amount in a first direction 241h, a deviation amount in a second direction 241g, a deviation area map 241i, a measurement position of the deviation amount in the first direction 241k, a measurement position of the deviation amount in the second direction 241j, and information for changing the position of a part related to the deviation (information for changing the position of a first part related to the first deviation and information for changing the position of a second part related to the second deviation) 241m are displayed. Also, when the judgment rank is C, an alert 241n is displayed indicating that re-imaging is necessary. Note that, as the re-photographing decision support information, it is also possible to generate and display only a part of the above information, rather than generating and displaying all of it.

[0072] The user (photographer) checks the radiographic image and the re-photographing decision support information, and ultimately determines whether or not re-photographing is necessary. If it is determined that re-photographing is necessary, the user presses the rejected image button 241d. Here, the rejected image refers to attaching a label to the failed image so that it is not used for diagnosis when re-photographing due to a failed image.

[0073] The control unit 21 determines whether or not the rejected image button 241d has been pressed by the operation unit 25 to instruct re-shooting (step S9). When it is determined that the defective image button 241d has been pressed by the operation unit 25 and that a command to retake the image has been issued (step S9; YES), the control unit 21 causes the radiological image determined to require retaking to be stored in the memory unit 22 in association with a flag indicating that the image is a defective image, information such as the judgment rank, the deviation area, the deviation amount, information on the body part, and information on the technician in charge (step S10). By storing and storing information such as the judgment rank, deviation area, deviation amount, body part information, and information about the technician in charge, in association with radiation images that are determined to require re-shooting, this information can be useful for training future photographers.

[0074] Next, the control unit 21 outputs re-photographing support information (step S11). For example, the control unit 21 displays the re-imaging support information on the display unit 24 (examination screen 241). The re-photographing support information is not particularly limited as long as it is information that supports re-photographing. For example, among the information generated as the re-photographing decision support information, information for changing the position of a part related to a deviation is useful re-photographing support information. In addition, information on the direction of deviation, information on the distance (amount of deviation), information on the deviation area, etc. may be used as the re-photographing support information. In addition, new information that has not been generated as the re-photographing decision support information may be generated and output as the re-photographing support information.

[0075] For example, when information indicating the direction of deviation from the ideal position of the irradiation center at the time of shooting (the likelihood of each direction) is output from the trained model M, a heat map 241p indicating the direction of deviation of the irradiation center may be generated and displayed as re-shooting support information, as shown in FIG. 8. In the heat map 241p shown in FIG. 8, the center of the nine divided regions indicates the ideal position of the irradiation center, and among the directions of deviation of the irradiation center output from the trained model M, the direction with higher likelihood is indicated with a darker color. FIG. 8 shows that the irradiation center at the time of shooting was rotated outward by about 5 degrees and adducted by about 2 degrees from the ideal position. Internal rotation / external rotation indicates rotation in the left-right direction, and internal rotation / abduction indicates rotation in the up-down direction. Together with the heat map 242p, information indicating the direction and amount to which the irradiation center should be shifted may be displayed as information 241m for changing the position of the part related to the deviation. The heat map 241p may be generated and output as re-photographing decision support information.

[0076] The user refers to the displayed preview image of the radiation image and the re-photographing support information, resets the settings of the photographing conditions and resets the positioning, and performs re-photographing. In response to an operation of the operation unit 25 or reception of a radiological image by the communication unit 23, the control unit 21 returns to step S4 (S3) and repeats steps S4 (S3) to S8.

[0077] On the other hand, in step S9, if it is determined that the rejected image button 241d has not been pressed by the operation unit 25 (step S9; NO), the control unit 21 performs a predetermined image processing on the radiation image and displays it in the image display area 241c as a final image (step S12). When the user operates the image processing conditions in the setting area 241b, the control unit 21 performs image processing in accordance with the operation.

[0078] The control unit 21 judges whether or not the output button 241e has been pressed by the operation unit 25, and when it judges that the output button 241e has not been pressed (step S13; NO), the process returns to step S9. When it is determined that the output button 241e has been pressed by the operation unit 25 (step S13; YES), the control unit 21 associates a flag indicating that the image is not a shooting error, body part information, information of the technician in charge, etc. with the radiation image generated as the final image, and stores the image in the storage unit 22. In addition, the control unit 21 associates the patient information and examination information (examination ID, examination date, shooting body part, shooting direction, etc.) with the radiation image generated as the final image, and transmits the associated image to the image management device 4 by the communication unit 23 (step S14), and ends the shooting control process.

[0079] As described above, the control unit 21 of the console 2 determines a first deviation in a specified area of ​​a radiological image, determines a second deviation in a specified area of ​​the radiological image, and outputs re-shooting decision support information that supports a decision on whether or not to re-shoot the radiological image based on at least the first deviation determination and the second deviation determination. Therefore, deviation in the positioning of the joint can be determined with high accuracy, and effective re-photographing decision support information can be output.

[0080] Moreover, the control unit 21 outputs information regarding the first deviation and information regarding the second deviation as re-photographing decision support information. Therefore, it is possible to assist the radiographer in easily understanding what type of deviation has occurred in a specific area of ​​the radiographic image.

[0081] For example, the control unit 21 outputs, as the re-photographing decision support information, information for changing the position of a first part related to a first deviation and information for changing the position of a second part related to a second deviation. Therefore, it is possible to assist the photographer in easily understanding how to change the positions of the first and second regions to achieve correct positioning.

[0082] For example, control unit 21 outputs the movement distance of the first part as information for changing the position of the first part, and outputs the movement distance of the second part as information for changing the position of the second part. Therefore, it is possible to assist the photographer in easily understanding how much the positions of the first and second regions should be moved to achieve correct positioning.

[0083] Also, for example, control unit 21 outputs the movement direction of the first part as information for changing the position of the first part, and outputs the movement direction of the second part as information for changing the position of the second part. Therefore, it is possible to assist the photographer in easily understanding in what direction the positions of the first and second regions should be moved to achieve correct positioning.

[0084] Furthermore, for example, the control unit 21 outputs re-photographing decision support information together with information indicating that re-photographing is necessary. Therefore, it is possible to assist the photographer in easily determining whether or not re-shooting is necessary.

[0085] In addition, the control unit 21 determines a first deviation in a specified area of ​​the radiographic image, determines a second deviation in a specified area of ​​the radiographic image, and outputs re-imaging support information that supports re-imaging of the radiographic image based on at least the determination of the first deviation and the determination of the second deviation. Therefore, deviation in the positioning of the joint can be determined with high accuracy, and effective re-photographing support information can be output.

[0086] The console 2 also has a trained model M in which at least a radiological image, information regarding a first deviation in a specified area of ​​the radiological image, and information regarding a second deviation in a specified area of ​​the radiological image are trained as teacher data, and the control unit 21 outputs decision support information that supports the decision of whether or not to re-take the radiological image based on the trained model M and the acquired radiological image. Therefore, deviation in the positioning of the joint can be determined with high accuracy, and effective re-photographing decision support information can be output.

[0087] Furthermore, the learning device 5 learns at least a radiographic image, information on a first deviation in a predetermined region of the radiographic image, and information on a second deviation in a predetermined region of the radiographic image as training data, and generates a trained model. Therefore, it is possible to generate a trained model that outputs information on the first deviation and information on the second deviation from an input radiographic image.

[0088] It goes without saying that the present invention is not limited to the above-described embodiment, and can be modified as appropriate without departing from the spirit of the present invention.

[0089] For example, in the above embodiment, the present invention has been described as being applied to support for deciding whether to reshoot or to support reshoot when capturing a Radiation image of the lateral side of an elbow joint. However, the present invention can also be applied to support for deciding whether to reshoot or to support reshoot when capturing a Radiation image of other joints, such as a knee joint or an ankle joint.

[0090] Furthermore, in the above embodiment, the description has been given of extracting the misalignment area from the radiation image using machine learning, but the misalignment area may be extracted by image processing. In addition to the deviation area, the re-photographing decision support information and the re-photographing support information may also be generated by machine learning.

[0091] In the above embodiment, the functions of the re-photographing decision support device and the re-photographing support device of the present invention are mounted on the console 2. However, the functions of the re-photographing decision support device and the re-photographing support device may be mounted on a device other than the console 2, or may be a dedicated device. In addition, the function of the learning device 5 may be provided in the console 2.

[0092] In addition, in the above embodiment, the control unit 21 of the console 2 is described as displaying the re-photographing decision support information and the re-photographing support information on the display unit 24, but the information may also be displayed on a display device separate from the console 2.

[0093] Further, the re-photographing decision support information and the re-photographing support information may be outputted not only as a display but also as a sound by a sound output device (not shown).

[0094] The control unit 21 of the console 2 may output the result of the misalignment determination (information about the misalignment), the radiographic image used for the misalignment determination (generation of information about the misalignment), threshold information, the type of algorithm, and other contents to an external device so that the contents can be confirmed on the external device. Also, the misalignment determination function (for example, a program for executing the above-mentioned re-photographing determination support information generation process or the used trained model M) itself may be output to the external device so that the misalignment can be determined by the external device.

[0095] Also, for example, in the above description, a hard disk or a non-volatile semiconductor memory is used as a computer-readable medium for the program according to the present invention, but the present invention is not limited to this example. As other computer-readable media, a portable recording medium such as a CD-ROM can be applied. Furthermore, a carrier wave can be applied as a medium for providing data of the program according to the present invention via a communication line. [Explanation of symbols]

[0096] 100 Radiography system 1 Radiography equipment 2 Console 21 Control section 22 Memory section 23 Communications Department 24 Display section 25 Control section 3. Radiation Generator 31 Generator 32 Irradiation switch 33 Radiation source 4. Image management device 5 Learning Device 71 Lateral border of trochlea of ​​humerus 72 Capitellum of the Humerus 73 Pulley shaft 75 Radius 76 Humerus 77 Ulna 78 Medial trochlea of ​​humerus N Communication Network R Radiation S Subject M trained models

Claims

1. a first determination means for determining a first deviation of a predetermined region of a radiographic image; a second determination means for determining a second deviation of a predetermined region of the radiation image; an output unit that outputs re-imaging decision support information that supports a decision as to whether or not to re-imaging the radiation image based on at least the first deviation determination and the second deviation determination; A re-photographing decision support device having the same.

2. the first determination means determines a deviation in a first direction of a predetermined region of the radiation image as the determination of the first deviation; the second determination means determines a deviation in a second direction of a predetermined region of the radiation image as the second deviation determination; 2 . The re-photographing decision support device according to claim 1 , wherein the output means outputs the re-photographing decision support information based on the determination of the deviation in the first direction and the determination of the deviation in the second direction.

3. the first determination means determines an amount of deviation in a first direction of a predetermined region of the radiation image as the determination of the first deviation; the second determination means determines an amount of deviation in a second direction of a predetermined region of the radiation image as the determination of the second deviation; 3. The re-photographing decision support device according to claim 1, wherein the output means outputs the re-photographing decision support information based on the determination of the amount of deviation in the first direction and the determination of the amount of deviation in the second direction.

4. 4. The re-photographing decision support device according to claim 1, wherein the output unit outputs information about the first deviation and information about the second deviation as the re-photographing decision support information.

5. 5. The re-photographing determination support device according to claim 1, wherein the output unit outputs, as the re-photographing determination support information, information for changing a position of a first portion associated with the first deviation and information for changing a position of a second portion associated with the second deviation.

6. 4. The re-photographing decision support device according to claim 3, wherein the output unit outputs, as the re-photographing decision support information, information for changing a position of a first portion associated with the first deviation, based on the amount of deviation in the first direction, and outputs information for changing a position of a second portion associated with the second deviation, based on the amount of deviation in the second direction.

7. 7. The re-photographing determination support device according to claim 5, wherein the output means outputs a moving distance of the first part as information for changing a position of the first part, and outputs a moving distance of the second part as information for changing a position of the second part.

8. 8. The re-photographing determination support device according to claim 5, wherein the output unit outputs a moving direction of the first part as information for changing a position of the first part, and outputs a moving direction of the second part as information for changing a position of the second part.

9. The re-photographing decision support device according to claim 1 , wherein the output unit outputs the re-photographing decision support information together with information indicating that re-photographing is necessary.

10. The re-photographing determination support device according to claim 1 , wherein the predetermined area for determining the first deviation and the predetermined area for determining the second deviation are the same joint area.

11. The re-photographing decision support device according to claim 1 , further comprising a determination unit that determines the predetermined region based on information on the body part to be photographed.

12. 12. The re-photographing decision support device according to claim 1, wherein the predetermined area for determining the first deviation and the predetermined area for determining the second deviation include an area surrounded by the outer edge of the trochlea of ​​the humerus and the capitellum edge of the humerus on the lateral side of the elbow joint, or the capitellum edge of the humerus and the inner edge of the trochlea of ​​the humerus.

13. A learning device that learns at least a radiographic image, information regarding a first deviation in a predetermined region of the radiographic image, and information regarding a second deviation in a predetermined region of the radiographic image as teacher data.

14. a trained model in which at least a radiographic image, information regarding a first deviation in a predetermined region of the radiographic image, and information regarding a second deviation in a predetermined region of the radiographic image are trained as training data; An acquisition means for acquiring a radiation image; an output means for outputting re-imaging decision support information for supporting a decision as to whether or not to re-imaging the radiographic image based on the trained model and the radiographic image acquired by the acquisition means; A re-photographing decision support device having the same.

15. a first determination means for determining a first deviation of a predetermined region of a radiographic image; a second determination means for determining a second deviation of a predetermined region of the radiation image; an output unit that outputs re-imaging support information for supporting re-imaging of the radiological image based on at least the first deviation determination and the second deviation determination; A re-photographing support device having the same.

16. Computer, a first determination means for determining a first deviation of a predetermined region from the radiation image; a second determination means for determining a second deviation of a predetermined region of the radiation image; an output unit for outputting re-imaging decision support information for supporting a decision as to whether or not to re-imaging the radiation image based on at least the first deviation determination and the second deviation determination; A program to function as a

17. the first determination means determines a deviation in a first direction of a predetermined region of the radiation image as the determination of the first deviation; the second determination means determines a deviation in a second direction of a predetermined region of the radiation image as the second deviation determination; The program according to claim 16 , wherein the output means outputs the re-photographing decision support information based on the determination of the deviation in the first direction and the determination of the deviation in the second direction.

18. the first determination means determines an amount of deviation in a first direction of a predetermined region of the radiation image as the determination of the first deviation; the second determination means determines an amount of deviation in a second direction of a predetermined region of the radiation image as the determination of the second deviation; 18. The program according to claim 16, wherein the output means outputs the re-photographing decision support information based on the determination of the amount of deviation in the first direction and the determination of the amount of deviation in the second direction.

19. 19. The program according to claim 16, wherein the output means outputs information about the first deviation and information about the second deviation as the re-photographing decision support information.

20. 20. The program according to claim 16, wherein the output means outputs, as the re-photographing decision support information, information for changing a position of a first portion associated with the first deviation and information for changing a position of a second portion associated with the second deviation.

21. 19. The program according to claim 18, wherein the output means outputs, as the re-photographing decision support information, information for changing a position of a first portion associated with the first deviation based on the amount of deviation in the first direction, and outputs information for changing a position of a second portion associated with the second deviation based on the amount of deviation in the second direction.

22. 22. The program according to claim 20 or 21, wherein the output means outputs a distance traveled by the first part as information for changing the position of the first part, and outputs a distance traveled by the second part as information for changing the position of the second part.

23. 23. The program according to claim 20, wherein the output means outputs a moving direction of the first part as information for changing the position of the first part, and outputs a moving direction of the second part as information for changing the position of the second part.

24. 24. The program according to claim 16, wherein the output means outputs the re-photographing decision support information together with information indicating that re-photographing is necessary.

25. 25. The program according to claim 16, wherein the predetermined area for determining the first deviation and the predetermined area for determining the second deviation are the same joint area.

26. The computer, a determining means for determining the predetermined region based on information on an imaging region; The program according to any one of claims 16 to 25,

27. A program according to any one of claims 16 to 26, characterized in that the specified area for determining the first deviation and the specified area for determining the second deviation include an area surrounded by the outer edge of the trochlea of ​​the humerus and the capitellum edge of the humerus on the side of the elbow joint, or the capitellum edge of the humerus and the inner edge of the trochlea of ​​the humerus.

28. a first determination step of determining a first deviation of a predetermined region of the radiographic image; a second determination step of determining a second deviation of a predetermined region of the radiation image; an output step of outputting re-imaging decision support information for supporting a decision as to whether or not to re-imaging the radiographic image based on at least the first deviation determination and the second deviation determination; A re-photographing decision support method including:

Images