Rejected image management device, operation method and operation program for rejected image management device, and radiation image capturing system

Abstract

Provided is a failed photograph image management device, wherein a processor executes a dataset acquisition process, a display control process, a correction process, and a learning process. In the dataset acquisition process, a dataset is acquired, the dataset including a radiographic image input into a first failed photograph determiner (LM1) configured by a machine learning model and mounted to a radiographic image capturing device (11), and a determination result output from the first failed photograph determiner (LM1). In the display control process, control is implemented to display the dataset on a display screen (19A, 12A). In the correction process, the determination result is corrected if an instruction to correct the determination result of the displayed dataset has been input. In the learning process, a second failed photograph determiner (LM2) that can be substituted with the first failed photograph determiner (LM1) is learned, and the second failed photograph determiner (LM2) is learned using a dataset selected from among displayed datasets as learning data.

Description

[Technical Field]

[0001] The technology disclosed herein relates to a failed image management device, an operating method and operating program for the failed image management device, and a radiographic image capturing system. [Background technology]

[0002] Radiation imaging devices that capture radiation images of a subject are known in the medical field. In the field of radiation imaging, a failed capture is generally called a "missing image," and a failed capture of a radiation image is called a "failed image." Reasons for a failure include improper positioning of the imaging area and an insufficient radiation dose. In a radiation imaging device, the captured radiation image is displayed on the console immediately after radiation imaging. A technician checks the radiation image displayed on the console and determines whether a failure has occurred. If a failure has occurred, the technician performs re-imaging.

[0003] JP 2020-025781 A describes a radiographic imaging device equipped with a defect determiner (corresponding to the determination unit in JP 2020-025781 A) configured using a machine learning model. The defect determiner receives a radiographic image captured by the radiographic imaging device as input and outputs a determination result as to whether the input radiographic image is a defect image. The radiographic imaging device described in JP 2020-025781 A is provided with a function that allows an operator of the radiographic imaging device, such as a technician, to correct the determination result output by the defect determiner and perform additional training on the defect determiner using learning data that uses the corrected determination result. Summary of the Invention [Problem to be solved by the invention]

[0004] Since the defect detector is constructed using a machine learning model, the more training data there is, the higher the detection accuracy will generally be. Therefore, even if a defect detector has already completed training, it may be possible to improve its detection accuracy by additionally training it using new training data.

[0005] However, in the radiographic imaging device described in JP 2020-025781 A, the defect determiner is additionally trained by using the judgment result corrected by the judgment of the operator of the radiographic imaging device as learning data. Therefore, if the operator's judgment is incorrect, the defect determiner will be additionally trained using learning data with incorrect judgment results, which may reduce the judgment accuracy of the defect determiner.

[0006] The technology disclosed herein provides a failed image management device that can suppress a decrease in the judgment accuracy of a failed image determiner installed in a radiological image capture device, an operating method and operating program for the failed image management device, and a radiological image capture system. [Means for solving the problem]

[0007] In order to achieve the above object, a failed image management device manages at least failed images among radiographic images of a subject captured by a radiographic imaging device, and includes a processor and a memory built into or connected to the processor. The processor is configured using a machine learning model and is a first failure determiner that performs failure determination on radiographic images, and executes the following: a dataset acquisition process that acquires a dataset including a radiographic image input to the first failure determiner mounted on the radiographic imaging device and a determination result output from the first failure determiner; a display control process that controls the display of the dataset on a display; a correction process that corrects the determination result when a correction instruction is input for the determination result of the displayed dataset; and a learning process that trains a second failure determiner that can be replaced with the first failure determiner and is not mounted on the radiographic imaging device, and trains the second failure determiner using a dataset selected from the displayed datasets as learning data.

[0008] If the data set includes both a primary judgment result, which is the judgment result output by the first defect judger, and a secondary judgment result, which is the judgment result by the operator of the radiographic imaging device, the processor may display the primary judgment result and the secondary judgment result in a distinguishable manner in the display control process.

[0009] In the dataset acquisition process, the processor may acquire a dataset that includes a failure determination result indicating that the radiological image may be a failure image in at least one of the primary determination result and the secondary determination result.

[0010] When the user inputs an approval instruction for the judgment result of the displayed dataset, the processor may record approval information indicating that approval has been given in association with the dataset to be approved.

[0011] When the determination result is reviewed by the user, the processor may record information indicating that the review has been performed in association with the dataset to be reviewed.

[0012] The processor may be capable of accumulating training data in a store.

[0013] The processor may not execute the learning process while the accumulated number of learning data is less than a preset number, and may execute the learning process when the accumulated number reaches the preset number.

[0014] When the processor has executed the learning process for the second defect determiner, the processor may transmit an update notification to the radiographic image capturing apparatus indicating that the first defect determiner can be updated by the second defect determiner.

[0015] The update notification may be a trigger for the radiographic imaging device to start the update process for the first defect determiner, or may be a notification that can be displayed on the console of the radiographic imaging device.

[0016] The processor may store the second defect determiner that has undergone the learning process in a storage unit that is accessible by the radiographic image capturing apparatus.

[0017] The processor may be capable of performing control to execute a test process using test data on the second defect determiner on which the learning process has been executed, and to display the test results on a display.

[0018] The second defect determiner immediately before the learning process is executed may be the same as the first defect determiner, and the processor may execute additional learning on the second defect determiner.

[0019] The operating method of the failed image management device according to the technology of the present disclosure is a method for operating a failed image management device that manages at least failed images among radiographic images of a subject captured by a radiographic imaging device, and includes: a first failure determiner configured using a machine learning model to perform a failure determination for radiographic images; a dataset acquisition process that acquires a dataset including a radiographic image input to the first failure determiner mounted on the radiographic imaging device and a determination result output from the first failure determiner; a display control process that controls the display of the dataset on a display; a correction process that corrects the determination result when a correction instruction for the determination result of the displayed dataset is input; and a learning process that trains a second failure determiner that can be replaced by the first failure determiner and is not mounted on the radiographic imaging device, the learning process training the second failure determiner using a dataset selected from the displayed datasets as learning data.

[0020] The operating program according to the technology of the present disclosure is configured by a machine learning model and causes a computer to execute the following operations: a dataset acquisition process for acquiring a dataset including a radiographic image input to the first defect determiner mounted on a radiographic imaging device and a determination result output from the first defect determiner; a display control process for controlling the display of the dataset on a display; a correction process for correcting the determination result when a correction instruction is input for the determination result of the displayed dataset; and a learning process for training a second defect determiner that can be replaced with the first defect determiner and is not mounted on the radiographic imaging device, the learning process training the second defect determiner using a dataset selected from the displayed datasets as learning data.

[0021] The operating program may cause the computer to function as a failed image management device that manages at least failed images among the radiation images of a subject captured by the radiation image capturing device.

[0022] A radiographic image capturing system according to the technique of the present disclosure includes any of the failed image management devices described above and a radiographic image capturing device. [Effects of the Invention]

[0023] According to the technology of the present disclosure, it is possible to suppress a decrease in the determination accuracy of the defect determiner installed in the radiographic image capturing device. [Brief explanation of the drawings]

[0024] [Figure 1] 1 is a diagram showing the overall configuration of a radiation image capturing system; [Figure 2] FIG. 10 is a diagram illustrating an example of a failed image. [Figure 3] FIG. 2 is a diagram illustrating the configuration of a console. [Figure 4] FIG. [Figure 5] FIG. 10 is a diagram showing classification of the reject determination results. [Figure 6]FIG. 10 is a diagram illustrating an example of an unauthorized data set. [Figure 7] FIG. 2 is a hardware configuration diagram of the failed image management device. [Figure 8] FIG. 2 is a functional block diagram of the failed image management device. [Figure 9] FIG. 10 is a diagram showing a conference screen. [Figure 10] FIG. 10 is a diagram showing a conference screen when the reason for the rejection is corrected. [Figure 11] FIG. 10 illustrates a conference screen displaying approved data sets. [Figure 12] FIG. 10 is a diagram showing a conference screen when there is no correction to the reason for the rejection. [Figure 13] FIG. 10 illustrates another example of a conference screen displaying approved data sets. [Figure 14] FIG. 2 is a diagram illustrating a learning unit. [Figure 15] FIG. 10 illustrates a test process. [Figure 16] FIG. 10 illustrates an update notification. [Figure 17] 10 is a flowchart of the entire system when updating the defect determiner. [Figure 18] 10 is a flowchart showing a procedure for generating and outputting an unauthorized data set. [Figure 19] 10 is a flowchart showing the procedure for generating and outputting an approved data set. [Figure 20] 10 is a flowchart showing the procedure of a learning process. [Figure 21] FIG. 10 is an explanatory diagram for narrowing down review targets. DETAILED DESCRIPTION OF THE INVENTION

[0025] The radiographic image capturing system 10 shown in FIG. 1 includes a radiographic image capturing device 11 and a failed image management device 12. The radiographic image capturing system 10 is installed in a medical facility, for example. In this example, the radiographic image capturing system 10 includes a failed image determiner update function that updates the failed image determiner installed in the radiographic image capturing device 11. The failed image determiner is configured by a machine learning model and performs a failed image determination for a radiographic image P captured by the radiographic image capturing device 11. When a radiographic image P is input as input data, the failed image determiner outputs a failed image determination result RS as output data. The failed image determination result RS is an example of a "determination result" according to the technology of the present disclosure.

[0026] The radiographic imaging device 11 of this example is equipped with a first defect determiner LM1 as a defect determiner. By being equipped with the first defect determiner LM1, the radiographic imaging device 11 has a function of presenting a defect determination result RS for the captured radiographic image P to an operator OP immediately after radiography is performed. On the other hand, the rejected image management device 12 is equipped with a second defect determiner LM2 as a defect determiner. The second defect determiner LM2 is replaceable with the first defect determiner LM1, but is an defect determiner that is not installed in the radiographic imaging device 11. The rejected image management device 12 has a learning function for training the second defect determiner LM2. The radiographic imaging system 10 can update the first defect determiner LM1 installed in the radiographic imaging device 11 by using the second defect determiner LM2 trained in the rejected image management device 12.

[0027] As is well known, the radiographic imaging device 11 obtains a radiographic image P of the subject H by imaging the subject H using radiation such as X-rays. The subject H is an example of a subject. The radiographic imaging device 11 is installed in an imaging room, for example. The radiographic imaging device 11 includes a radiation source 16, a radiation source control device 17, a radiographic image detection device 18, and a console 19.

[0028] The radiation source 16 irradiates radiation. The radiation source control device 17 controls the radiation source 16. The radiation source control device 17 is provided with an operation panel (not shown). An operator OP operates the operation panel to set radiation irradiation conditions. The radiation irradiation conditions include the tube voltage (unit: kV), tube current (unit: mA), and radiation irradiation time (unit: mS) applied to the radiation source 16. The radiation source 16 is also provided with an irradiation field limiter that limits the radiation irradiation field. The irradiation field is adjusted manually or electrically. The radiation source 16 is also attached to a moving mechanism (not shown), which makes it possible to adjust the relative distance, height, and irradiation direction of the radiation source 16 relative to the subject H.

[0029] The radiation image detection device 18 detects a radiation image P of the subject H by receiving radiation irradiated from the radiation source 16 and transmitted through the subject H. The radiation image detection device 18 is, for example, a flat panel detector in which pixels that convert radiation into electrical signals are arranged two-dimensionally. The radiation image detection device 18 may be of a direct conversion type or an indirect conversion type, depending on the method for converting radiation into electrical signals. The radiation image detection device 18 may take the form of, for example, a stationary type that can be attached to a supine position radiography table or an upright position radiography table, or a portable electronic cassette. The electronic cassette may be used alone or in combination with an radiography table. Even in the stationary type, the height of the radiation image detection device 18 may be adjusted.

[0030] When radiography is performed using the radiographic imaging device 11, an operator OP such as a radiological technician adjusts the relative positional relationship between the radiation source 16, the radiographic image detection device 18, and the subject H, thereby positioning the subject H with respect to the radiographic imaging device 11. The operator OP also sets irradiation conditions and the like according to the imaging site and the body thickness of the subject H, etc.

[0031] The console 19 acquires the radiographic image P detected by the radiographic image detection device 18 and displays it on the display 19A. For example, a captured image confirmation screen 19B is displayed on the display 19A. The first loss determiner LM1 is mounted on the console 19, and the loss determination using the first loss determiner LM1 is performed on the console 19. In addition to the radiographic image P, the captured image confirmation screen 19B can display the loss determination result RS by the first loss determiner LM1.

[0032] The reasons for rejection that a radiographic image P should be deemed a rejected image include improper positioning of the imaging region due to a positioning error, and an insufficient radiation dose. The operator OP sets the positioning and irradiation conditions of the subject H when radiographing. Because the radiographic image capturing device 11 has a reject determination function, the operator OP can immediately check whether or not the radiographic image P is likely to be a rejected image after capturing the radiographic image P. If re-capture is required, the operator OP can correct the positioning and irradiation conditions by checking the reject determination result RS.

[0033] The failure determination is not limited to determining whether the radiological image P is a failure image, but may include at least a determination of whether there is a possibility that the radiological image P is a failure image. The determination of whether there is a possibility that the radiological image P is a failure image includes at least one of determining whether there is a failure reason that is the basis for determining that the image is a failure image, whether there is a possible failure reason, determining the details of the failure reason, and determining whether re-imaging is necessary. The failure determination result RS indicating that there is a possibility that the image is a failure image includes the failure determination results RS such as "it is a failure image," "it may be a failure image," "the existence of a failure reason and its details," and "re-imaging is necessary."

[0034] In this example, the first reject determiner LM1 determines whether or not there is a reject reason for the radiological image P and the details of that reject reason. In this example, the reject determination result RS outputs a reject reason such as "insufficient dose" only if there is a reject reason, and if there is no reject reason, nothing is output. In other words, the reject determination result RS indicates whether or not there is a reject reason depending on whether or not a specific reject reason is included. Of course, in addition to the specific reject reason, the reject determination result RS may also be output, such as "reject reason present or absent" and "appropriate or inappropriate."

[0035] The console 19 is equipped with a defect determination function using the first defect determiner LM1, and is also capable of accepting input of a defect determination result RS by an operator OP if the operator OP determines that the defect determination result RS by the first defect determiner LM1 is incorrect. Here, the defect determination result RS by the first defect determiner LM1 is referred to as a primary determination result RS1, and the defect determination result RS by the operator OP is referred to as a secondary determination result RS2.

[0036] The console 19 transmits a data set DS1 including a radiographic image P and a failure determination result RS to the failed image management device 12. In this example, a conference regarding the data set DS1 is held using the failed image management device 12, as described below. The conference is a meeting in which users review the validity of the failure determination result RS included in the data set DS1. In this example, the users are medical staff ST, such as doctors and technicians, at the medical facility where the failed image management device 12 is installed. When the failure determination result RS of the data set DS1 is approved by the user at the conference, the data set DS1 becomes approved. Here, to distinguish data sets based on whether they have been approved by the user, an unapproved data set is referred to as an unapproved data set DS1, and an approved data set is referred to as an approved data set DS2. The console 19 transmits the unapproved data set DS1 to a failed image DB (Data Base) 12C of the failed image management device 12.

[0037] The failed image management device 12 manages at least failed images among the radiographic images P captured by the radiographic image capturing device 11. A failed image refers to a radiographic image that has failed to be captured and is not used for diagnosis. Whether or not the radiographic image has failed is generally determined by the judgment of the operator OP. A typical failed image management device collects such failed images and manages the collected failed images. The failed image management device 12 of this example manages not only failed images but also radiographic images P that may be failed. The reason for this, as will be described later, is that in addition to failed images, radiographic images P that may be failed are also used as learning data for training the second failure determiner LM2. Of course, the failed image management device 12 can also manage radiographic images P other than failed images, such as re-photographed images PR related to failed images, in addition to failed images. A re-photographed image PR related to a failed image is a radiographic image P that is re-photographed when the initially captured radiographic image P is a failed image.

[0038] The failed image management device 12 has a function of managing failed images generated in a medical facility, for example, and performing statistical processing of failed images and reasons for failure, etc. The statistical processing is, for example, processing of calculating statistical data such as the number of failed images, the number of failed images by reason for failure, and the number of failed images for each imaging technique (described later). The failed image management device 12 is installed, for example, in a room separate from the imaging room.

[0039] In addition to these functions, the failed image management device 12 also has a conference function. The conference function is used, for example, in a conference where multiple medical staff members ST exchange opinions about failed images. In the conference, the validity of the failed image determination result RS included in the unapproved data set DS1 to be reviewed is verified through the exchange of opinions among the multiple medical staff members ST. For example, it is verified whether the determination of whether or not there is a reason for the failed image in the failed image determination result RS is valid, or whether the specific content of the reason for the failed image is valid. If the failed image determination result RS of the unapproved data set DS1 is determined to be valid in the conference, the failed image determination result RS is approved as is without modification. If the failed image determination result RS is determined to be invalid, the failed image determination result RS is modified as necessary, and the modified failed image determination result RS is approved.

[0040] Furthermore, the unapproved data set DS1 may also include re-captured images PR (see Figure 6, etc.). In such cases, the re-captured images PR are also subject to similar verification. Through such a conference, the validity of the reject determination result RS is verified, and the verified reject determination result RS is approved.

[0041] The conference function of the failed image management device 12 not only displays the unapproved data set DS1 to be reviewed on a display screen such as the conference screen 12B of the display 12A, but also, as described below, makes it possible to correct the failed image determination result RS included in the unapproved data set DS1 and record approval information indicating that the uncorrected or corrected failed image determination result RS whose validity has been verified has been approved.

[0042] For example, the purpose of the conference is to educate less experienced medical staff STs. The medical staff STs who participate in the conference include experienced doctors or technicians in leadership positions. Because the validity of the defect determination result RS is verified by these multiple medical staff STs, the reliability of the defect determination result RS for the approved dataset DS2 that has been reviewed by the conference is considered to be higher than the sole judgment of the operator OP.

[0043] When the unapproved data set DS1 is approved, the failed image management device 12 stores it in the failed image DB 12C as an approved data set DS2. The failed image management device 12 also has a function of training a second failure determiner LM2 that can replace the first failure determiner LM1. The failed image management device 12 additionally trains the second failure determiner LM2 using the approved data set DS2. In this example, the second failure determiner LM2 immediately before the additional training learning process is performed is identical to the first failure determiner LM1. Additional training refers to additional training that is performed during actual operation after the second failure determiner LM2 is generated by programming and training processes. When additional training is performed, the first failure determiner LM1 is updated by the second failure determiner LM2 that has completed the additional training. In this way, the radiographic image capturing system 10 has a function of updating the first reject determiner LM1 installed in the radiographic image capturing device 11 by the radiographic image capturing device 11 and the rejected image management device 12.

[0044] FIG. 2 shows samples SP1 to SP3 of failed images. Sample SP1 is a radiographic image P with the imaging technique set to "chest / back," i.e., a radiographic image P captured from the back with the chest as the imaging region. The reason for the failure of sample SP1 is an insufficient radiation dose. Insufficient dose is mainly caused by an error in setting the irradiation conditions. An imaging technique is an imaging method defined by at least a combination of an imaging region and an imaging direction. In an imaging order, a doctor specifies an imaging technique including an imaging region and an imaging direction according to the diagnostic purpose of subject H.

[0045] Sample SP2 is a radiological image P obtained using the same "chest / back" imaging technique as sample SP1. The reason for the rejection of sample SP2 is a partial loss of the imaging area. In this example, there is a partial loss of the left side of the chest. The partial loss is mainly caused by a positioning error. Sample SP3 is a radiological image P obtained using the "knee / flexed position / side" imaging technique, i.e., the imaging area is the knee, and the image was obtained from the side with the knee bent. The reason for the rejection of sample SP3 is internal rotation of the knee joint. Since the area of ​​interest in the knee joint differs depending on the diagnostic purpose, the way the knee joint is bent, etc., is important in positioning. Therefore, internal or external rotation of the knee joint may be a reason for rejection depending on the diagnostic purpose.

[0046] Next, the configuration and functions of the console 19 of the radiographic image capturing apparatus 11 will be described in more detail with reference to FIGS.

[0047] The console 19 is configured by installing a console program (shown as a CSL program in the figure) 36 on a computer such as a personal computer or a workstation. The console 19 includes a display 19A, an input device 31, a CPU (Central Processing Unit) 32, a storage device The system includes a processor 34, a memory 33, and a communication unit 35. These are interconnected via a data bus.

[0048] The display 19A is a display unit that displays various operation screens having operation functions using a GUI (Graphical User Interface), and the radiation image P. The input device 31 is an input operation unit that includes a touch panel, a keyboard, or the like.

[0049] The storage device 34 is, for example, a hard disk drive (HDD) or a solid state drive (SSD), and is either built into the console 19 or externally connected to the console 19. The storage device 34 stores control programs such as an operating system, various application programs, and various data associated with these programs.

[0050] The storage device 34 also stores a console program 36 for causing the computer to function as the console 19, and a first defect determiner LM1. The storage device 34 also stores radiological images P received from the radiological image detecting device 18.

[0051] The memory 33 is a work memory for the CPU 32 to execute processing. The CPU 32 loads a console program 36 stored in a storage device 34 into the memory 33 and executes processing in accordance with the console program 36, thereby comprehensively controlling each section of the console 19. The communication section 35 transmits and receives various data such as radiographic images P to and from the radiographic image detection device 18. The communication section 35 also communicates with the radiation source control device 17 and the failed image management device 12. The communication section 35 is connected to a network such as a LAN (Local Area Network), for example. Which network will the communication take place over?

[0052] The image DB 37 is, for example, a storage device of a PACS (Picture Archiving and Communication System). The unapproved data set DS1 is transmitted to the PACS as well as to the failed image management device 12 and stored in the image DB 37.

[0053] The captured image confirmation screen 19B is an example of a display screen displayed on the display 19A of the console 19, and is a screen on which the operator OP can confirm the radiographic image P that has been radiographed. The operator OP checks the radiographic image P on the captured image confirmation screen 19B and determines whether the radiographic image P is a defective image, whether there is a possibility that it is a defective image, what the specific reason for the defect is, and whether re-imaging is necessary. The defective image determination function of the console 19 supports such work by the operator OP.

[0054] The captured image confirmation screen 19B includes an AI (Artificial Intelligence) judgment button 41, Operation buttons are provided, such as a next judgment input button 42, a re-shoot button 43, and an image output button 44. The AI ​​judgment button 41 is an operation button for executing the first defect judgment process using the first defect judger LM1.

[0055] When the AI ​​determination button 41 is operated, the CPU 32 executes a first defect determination process using the first defect determiner LM1.

[0056] 4, the first defect determiner LM1 may be, for example, a convolutional neural network (CNN) suitable for image analysis. The first defect determiner LM1 may include, for example, an encoder 46 and a classifier 47.

[0057] The encoder 46 is configured with a CNN, which performs convolution and pooling on the input radiographic image P to extract multiple types of feature maps representing the features of the radiographic image P. As is well known, in a CNN, convolution is a spatial filtering process using multiple types of filters, each having a size of, for example, 3 × 3. In the case of a 3 × 3 filter, a filter coefficient is assigned to each of the nine squares. In convolution, the center square of the filter is aligned with a pixel of interest in the radiographic image P, and the sum of products of the pixel values ​​of a total of nine pixels, including the pixel of interest and its eight surrounding pixels, is output. The output sum of products represents the feature amount of the region of interest to which the filter is applied. Then, for example, by shifting the pixel of interest by one pixel at a time while applying the filter to all pixels in the radiographic image P, a feature map having feature amounts equivalent to the number of pixels in the radiographic image P is output. By applying multiple types of filters with different filter coefficients, multiple types of feature maps are output. The number of feature maps corresponding to the number of filters is also called the number of channels.

[0058] This convolution is repeated while reducing the size of the radiographic image P. The process of reducing the radiographic image P is called pooling. Pooling is performed by thinning out adjacent pixels and averaging them. The pooling process reduces the size of the radiographic image P in stages, such as by half, one-quarter, and one-eighth. A multi-channel feature map is output for each size of the radiographic image P. When the size of the radiographic image P is large, the radiographic image P depicts detailed morphological features of the subject, whereas when the size of the radiographic image P is small (i.e., when the resolution is low), the detailed morphological features are ignored from the radiographic image P, and only the general morphological features of the subject are depicted in the radiographic image P. Therefore, the feature map for a large radiographic image P represents the microscopic features of the subject depicted in the radiographic image P, and the feature map for a small radiographic image P represents the macroscopic features of the subject depicted in the radiographic image P. The encoder 46 extracts a multi-channel feature map representing the macro and micro features of the radiation image P by performing such convolution processing and pooling processing on the radiation image P.

[0059] The first reject determiner LM1 is, for example, a classification model that derives one reject reason (including cases where there is no reject reason) that is most likely to be the reject reason indicated by the radiographic image P from among multiple reject reasons. Therefore, a classification unit 47 is provided as a configuration for deriving one reject reason based on the feature amount of the radiographic image P extracted by the encoder 46. The classification unit 47 includes, for example, multiple perceptrons, each having one output node for multiple input nodes. Furthermore, weights indicating the importance of the multiple input nodes are assigned to the perceptrons. In each perceptron, a sum of products, which is the sum of values ​​obtained by multiplying each of the input values ​​input to the multiple input nodes by the weight, is output from the output node as an output value. For example, such a perceptron is formulated using an activation function such as a sigmoid function.

[0060] In the classification unit 47, the outputs and inputs of multiple perceptrons are connected to form a multi-layered neural network with multiple hidden layers between the input layer and output layer. As a method for connecting multiple perceptrons between layers, for example, full connection is used, in which all output nodes of the previous layer are connected to one input node of the next layer.

[0061] All feature quantities contained in the feature map of the radiographic image P are input to the input layer of the classification unit 47. In the perceptrons constituting each layer of the classification unit 47, the feature quantities are input as input values ​​to the input nodes. Then, the sum of products, which is the sum of values ​​obtained by multiplying the feature quantities of each input node by a weight, is output as an output value from the output node, and the output value is passed to the input node of the perceptron in the next layer. In the output layer of the final layer of the classification unit 47, the probability of each of multiple reject reasons is output using a softmax function or the like based on the output values ​​of multiple perceptrons. Then, the most likely reject reason is derived based on these probabilities. Figure 4 shows an example in which the reject reason "insufficient dose" is derived based on the radiographic image P. The reject determination result output by the first reject determiner LM1 becomes the primary determination result.

[0062] This example is merely an example, and the first defect determiner LM1 may be of another form as long as it is a classification model that can derive the defect reason based on the radiographic image P. Furthermore, in this example, only one first defect determiner LM1 is used without distinguishing between imaging techniques, but a first defect determiner LM1 may be provided for each imaging technique. Since the defect reason may differ depending on the imaging technique, the determination accuracy is improved by using a first defect determiner LM1 for each imaging technique.

[0063] In FIG. 3, the captured image confirmation screen 19B displays the primary determination result RS1, which is the defect determination result RS output by the first defect determiner LM1.

[0064] The secondary judgment input button 42 is an operation button for inputting a secondary judgment result RS2, which is a reject judgment result RS by the operator OP. If the primary judgment result RS1 differs from the judgment of the operator OP, the operator OP can input the secondary judgment result RS2 of the operator OP. When the secondary judgment input button 42 is operated, it becomes possible to input the secondary judgment result RS2, and the operator OP operates the input device 31 to input the secondary judgment result RS2. In the example of FIG. 3, the primary judgment result RS1 is insufficient dose, while the secondary judgment result RS2 of the operator OP is input as "no reason for rejection."

[0065] The re-photographing button 43 is an operation button that is operated when re-photographing is necessary. When the operator OP determines that re-photographing is necessary, he or she operates the re-photographing button 43 to perform re-photographing. When the re-photographing button 43 is operated, for example, the radiographic image P of the first photographing and the radiographic image P of the re-photographing are associated with each other. The association is performed, for example, by using the order ID (Identification ID) of the photographing order. This allows the first radiographic image P and the re-photographed radiographic image P to be It is possible to handle P together.

[0066] The image output button 44 is a button for outputting an unapproved data set DS1 including a radiographic image P and a failure determination result RS to the image DB 37 and the failure image DB 12C. If a secondary determination result RS2 has been input, the failure determination result RS of the unapproved data set DS1 includes both the primary determination result RS1 and the secondary determination result RS2. If a secondary determination result RS2 has not been input, only the primary determination result RS1 is included.

[0067] As shown in Fig. 5, the unauthorized data set DS1 includes not only the reject determination result RS but also additional information associated with the radiological image P. The additional information is data such as the name, age, and sex of the subject H, the order ID, and the imaging technique. In the example of Fig. 5, the reject determination result RS includes both the primary determination result RS1 and the secondary determination result RS2.

[0068] The types of reject determination results RS included in the unauthorized data set DS1 can be roughly classified into four types shown in Fig. 5, depending on the combination of whether or not there is a reject reason in the primary determination result RS1 and whether or not there is a reject reason in the secondary determination result RS2. That is, there are four types: when there is a reason in both the primary determination result RS1 and the secondary determination result RS2, when there is a reject reason only in the primary determination result RS1, when there is a reason only in the secondary determination result RS2, and when there is no reason in either.

[0069] Of course, since the secondary judgment result RS2 is information that is input as needed at the discretion of the operator OP, more specifically, in addition to the four types shown in Fig. 5, there are also cases in which the secondary judgment result RS2 is not input. Also, even in cases in which both the primary judgment result RS1 and the secondary judgment result RS2 have a reason for rejection, the reasons for rejection may be the same or different for the primary judgment result RS1 and the secondary judgment result RS2. In order to distinguish between these cases, the unauthorized dataset DS1 records the presence or absence of a reason for rejection for the primary judgment result RS1 and the specific content of the reason for rejection ("insufficient dose" in the example of Fig. 5) in addition to the presence or absence of a reason for rejection for the secondary judgment result RS2, if there is one, and the specific content of the reason for rejection ("insufficient dose" in the example of Fig. 5).

[0070] Furthermore, as shown in Figure 6, when re-photographing is performed, the unapproved data set DS1 includes a re-photographed image PR. A primary judgment result RS1 and a secondary judgment result RS2 are also recorded for each re-photographed image PR. The re-photographed image PR is an image that has been judged by the operator OP to have no reason for rejection. Therefore, in the rejection judgment result RS of the re-photographed image PR, at least the secondary judgment result RS2 indicates that there is no reason for rejection. The re-photographed image PR in Figure 6 is an example in which both the primary judgment result RS1 and the secondary judgment result RS2 indicate that there is no reason for rejection.

[0071] When an unapproved data set DS1 is generated as a result of radiography and failure determination using the radiographic imaging device 11, the console 19 notifies the failed image management device 12 that the unapproved data set DS1 has been generated. The failed image management device 12 can recognize from this notification that the unapproved data set DS1 has been generated.

[0072] Next, the configuration and functions of the failed image management device 12 will be described in more detail with reference to Figures 7 to 16. As shown in Figure 7, the failed image management device 12, like the console 19, is configured by installing an operating program 56 on a computer such as a personal computer or a workstation. The operating program 56 is an example of an operating program for the failed image management device 12 according to the technology of the present disclosure. The failed image management device 12 includes a display 12A, an input device 51, a CPU 52, a storage device 54, a memory 53, and a communication unit 55. These are connected to each other via a data bus 58.

[0073] The display 12A is a display unit that displays various operation screens having GUI operation functions, and the radiation image P. The input device 51 is an input operation unit that includes a touch panel, a keyboard, or the like.

[0074] The storage device 54 is, for example, an HDD or SSD, and is either built into the failed image management device 12 or externally connected to the failed image management device 12. The storage device 54 stores control programs such as an operating system, various application programs, and various data associated with these programs.

[0075] The storage device 54 also stores an operating program 56 for causing the computer to function as the rejected image management device 12, and a second rejected image determiner LM2. The storage device 54 also temporarily stores an unapproved data set DS1 acquired from the console 19 or the image DB 37.

[0076] The memory 53 is a work memory for the CPU 52 to execute processing. The CPU 52 loads an operating program 56 stored in the storage device 54 into the memory 53 and executes processing in accordance with the operating program 56, thereby comprehensively controlling each part of the failed image management device 12. The communication unit 55 is a communication unit for communicating with a network such as a LAN (Local Area Network), Communication with the console 19 is performed via the network.

[0077] The failed image DB 12C is an external storage device of the failed image management device 12. As described above, the failed image DB 12C stores the unapproved data set DS1 and the approved data set DS2.

[0078] By executing the operating program 56, the CPU 52 functions as various processing units such as a dataset acquisition unit 61, a display control unit 62, a correction processing unit 63, an approval processing unit 64, a storage processing unit 66, a learning unit 67, a test processing unit 68, and an update notification unit 69.

[0079] As shown in FIG. 8 , the dataset acquisition unit 61 acquires the unapproved dataset DS1 from the failed image DB 12C and stores the acquired unapproved dataset DS1 in the storage device 54. In this manner, the dataset acquisition unit 61 executes a dataset acquisition process to acquire the unapproved dataset DS1 including the radiographic image P input to the first failure determiner LM1 mounted on the radiographic imaging device 11 and the failure determination result RS output from the first failure determiner LM1. In this example, the dataset acquisition unit 61 acquires the unapproved dataset DS1 from the failed image DB 12C, which is external storage of the failed image management device 12. However, the dataset acquisition unit 61 may also access the image DB 37 of the PACS and acquire the unapproved dataset DS1 from the image DB 37. Furthermore, if the unapproved dataset DS1 is stored in a shared folder or the like in the storage device 34 of the console 19, the dataset acquisition unit 61 may also access the storage device 34 of the console 19 and acquire the unapproved dataset DS1 from the shared folder.

[0080] The display control unit 62 loads screen data of the conference screen 12B including the unapproved data set DS1 into the memory 53, and performs control to display the conference screen 12B on the display 12A based on the screen data loaded into the memory 53. In this way, the display control unit 62 executes a display control process to control the display of the unapproved data set DS1 on the display 12A.

[0081] When a user inputs an instruction to correct the reject determination result RS of the unapproved data set DS1 displayed on the conference screen 12B, the correction processing unit 63 executes a correction process to correct the reject determination result RS. The correction processing unit 63 updates the reject determination result RS of the unapproved data set DS1 to be corrected.

[0082] The approval processing unit 64 executes approval processing when a user inputs an approval instruction for the reject determination result RS of the unapproved data set DS1 displayed on the conference screen 12B. The approval processing is a process of recording approval information indicating that the reject determination result RS has been approved, in association with the unapproved data set DS1 to be approved. The approval information is recorded, for example, as an approval history HSA. When the unapproved data set DS1 is approved, it changes to an approved data set DS2. The approval history HSA records, for example, the ID information and approval date and time of the approved data set DS2. The approval history HSA is stored, for example, in the rejected image DB 12C, in the same way as the approved data set DS2.

[0083] The approval history HSA has a meaning as information indicating that the unapproved data set DS1 has been displayed on the conference screen 12B. Furthermore, the fact that the unapproved data set DS1 has been displayed on the conference screen 12B is considered to be highly likely to be the subject of review at the conference. Therefore, the approval history HSA has a meaning as information indicating that the reject determination result RS of the unapproved data set DS1 has been reviewed by the medical staff ST, who is the user, at the conference. In this way, the approval history HSA is an example of information indicating that the unapproved data set DS1 has been displayed on the display screen, or information indicating that a review has been conducted.

[0084] When a save instruction is input by the user, the storage processing unit 66 executes a storage process for storing the approved data set DS2 and the approval history HSA in the failed image DB 12C.

[0085] FIG. 9 shows an example of the conference screen 12B. An unapproved data set DS1 is displayed on the conference screen 12B. As an example, the conference screen 12B displays each unapproved data set DS1 to be reviewed in the conference. The conference screen 12B displays the radiographic images P and the reject determination results RS included in the unapproved data set DS1. The unapproved data set DS1 shown in FIG. 9 is an example that includes both the initially captured radiographic image P and the re-captured image PR, and also includes the reject determination results RS for both the initially captured radiographic image P and the re-captured image PR. Furthermore, the reject determination result RS in the example of FIG. 9 includes a primary determination result RS1 and a secondary determination result RS2.

[0086] If the unauthorized data set DS1 includes both the primary judgment result RS1 output by the first defect determiner LM1 and the secondary judgment result RS2 by the operator OP, the display control unit 62 displays the primary judgment result RS1 and the secondary judgment result RS2 in a distinguishable manner during the display control process. This makes it possible to distinguish and verify the validity of each of the primary judgment result RS1 and the secondary judgment result RS2. As an example, the conference proceeds by multiple medical staff members ST exchanging opinions while viewing this conference screen 12B.

[0087] The conference screen 12B is provided with various operation buttons, including a display button 71, an end button 72, a correction button 73, an approval button 74, and a save button 75. The display button 71 is an operation button for displaying the unapproved data set DS1. When the display button 71 is operated, a display instruction is input to the display control unit 62, and the unapproved data set DS1 acquired via the data set acquisition unit 61 is displayed on the conference screen 12B. The end button 72 is an operation button for closing the conference screen 12B. When the conference is to be ended, the conference screen 12B is closed.

[0088] The correction button 73 is a button that is operated when correcting the reject determination result RS of the unapproved data set DS1. When the correction button 73 is operated, the reject determination result RS of the unapproved data set DS1 can be edited. Corrections are made using the keyboard of the input device 51 or the like. Correction instructions issued by the correction button 73 are input to the correction processing unit 63, which corrects the reject determination result RS in accordance with the correction instructions.

[0089] The approval button 74 is a button that is operated when approving the reject determination result RS of the unapproved data set DS1. When the approval button 74 is operated, an approval instruction is input to the approval processing unit 64, and the approval processing is executed in the approval processing unit 64.

[0090] The save button 75 is a button for saving the unapproved data set DS1 as an approved data set DS2 when the unapproved data set DS1 is approved. When the save button 75 is operated, a save instruction is input to the storage processing unit 66. Based on the save instruction, the storage processing unit 66 executes a storage process for storing the approved data set DS2 in the failed image DB 12C.

[0091] Furthermore, the conference screen 12B displays information indicating how many unapproved data sets DS1 exist in the failed image DB 12C. In this example, information indicating that there are six unapproved data sets DS1 is displayed.

[0092] FIG. 10 shows an example in which the reject determination result RS of the unapproved dataset DS1 shown in FIG. 9 has been corrected. In the example shown in FIG. 10, the reject determination result RS of the re-captured image PR has been corrected. As shown in FIG. 9, for the initially captured radiographic image P, both the primary determination result RS1 and the secondary determination result RS2 determined that there was a reject reason of "insufficient dose." However, for the re-captured image PR, both the primary determination result RS1 and the secondary determination result RS2 determined that there was no reject reason. By reviewing this unapproved dataset DS1 at the conference, the reject determination result RS of the re-captured image PR is corrected from "no reject reason" to "insufficient dose," as shown in FIG. 10, for example. On the other hand, the reject determination result RS of the initially captured radiographic image P is determined to be appropriate, and no correction is made.

[0093] When the approval process is performed for the unapproved data set DS1 shown in FIG. 10 by operating the approval button 74, the unapproved data set DS1 shown in FIG. 10 is changed to an approved data set DS2, as shown in FIG. 11. In the example of FIG. 11, a sign 70 indicating approval is displayed on the approved data set DS2 on the conference screen 12B. The sign 70 is, for example, the text "Approved." As shown in FIG. 11, in the approved data set DS2, a single reject determination result RS is recorded as a unified opinion of multiple medical staff ST, who are conference reviewers, for each of the initial radiographic image P and the re-photographed image PR. When the save button 75 is operated, this approved data set DS2 is stored in the rejected image DB 12C. When one new approved data set DS2 is stored, the display of the remaining number of unapproved data sets DS1 decreases. The number of data sets, which was six in FIG. 10, has changed to five in FIG. 11.

[0094] The examples of Figures 12 and 13 show an example in which the reject determination result RS of the unapproved dataset DS1 shown in Figure 12 is approved as is without any correction, and is changed to the approved dataset DS2 as shown in Figure 13. The reject determination result RS is the same for both the unapproved dataset DS1 shown in Figure 12 and the approved dataset DS2 shown in Figure 13. In this way, the reject determination result RS of the unapproved dataset DS1 may be approved as is, and may become the reject determination result RS of the approved dataset DS2.

[0095] As shown in FIG. 14 , the learning unit 67 executes a learning process to train a second failure determiner LM2 that can replace the first failure determiner LM1 and is not installed in the radiographic imaging device 11. The learning process executed by the learning unit 67 is a learning process to train the second failure determiner LM2 using training data selected from the approved data set DS2 displayed on the display 12A of the failed image management device 12, i.e., the approved data set DS2 stored in the failed image DB 12C. The learning process executed by the learning unit 67 is additional training for the second failure determiner LM2. The learning unit 67 selects the approved data set DS2 from the failed image DB 12C as training data based on the approval history HSA. It is not necessary to select the entire approved data set DS2 as training data; only a portion of the approved data set DS2 may be selected. Criteria for selecting training data from the approved data set DS2 can be set as appropriate. The selection of training data may be performed by the learning unit 67 or by the user.

[0096] The approved dataset DS2 includes a radiographic image P and a reject determination result RS. The reject determination result RS is used as correct answer data to be compared with the reject determination result RS output by the second reject determiner LM2. In addition to the initially captured radiographic image P, the approved dataset DS2 may also include a re-captured image PR. In this case, the set of the initially captured radiographic image P and the reject determination result RS, and the set of the re-captured image PR and the reject determination result RS are each treated as one piece of learning data.

[0097] The learning unit 67 can store the selected approved data set DS2 as learning data in the failed image DB12C. The learning unit 67 does not execute the learning process while the number of accumulated learning data is less than a preset number, and executes the learning process of the second failed image determiner LM2 when the number of accumulated learning data reaches the preset number. Setting information such as the set number is recorded in the storage device 54 and can be changed as needed.

[0098] The learning unit 67 has a main processing unit 67A, an evaluation unit 67B, and an update unit 67C. The main processing unit 67A reads the second defect determiner LM2 to be subjected to the learning process from the storage device 54, and inputs the radiographic image P or the re-photographed image PR included in the approved data set DS2 selected as learning data to the read second defect determiner LM2. Then, the second defect determiner LM2 executes a defect determination process to output a defect determination result RS including the presence or absence of a reason for the defect and the details thereof, based on the radiographic image P or the re-photographed image PR. The basic configuration of the second defect determiner LM2 is the same as that of the first defect determiner LM1 described in FIG. 4.

[0099] In this processing unit 67A, the second defect determiner LM2 outputs the defect determination result RS as output data OD to the evaluation unit 67B. The evaluation unit 67B compares the defect determination result RS, which is the output data OD, with the correct answer data AD included in the approved data set DS2, which is the learning data, and evaluates the difference between them as a loss using a loss function. As with the first defect determiner LM1 described above, the second defect determiner LM2 outputs multiple defect reasons as probabilities. Therefore, not only is there a loss when the defect reason in the output data OD is incorrect, but even if the output value output as the probability is correct, if the output value is lower than the target value, the difference between the target value and the output value is evaluated as a loss. The evaluation unit 67B outputs the evaluated loss as the evaluation result to the update unit 67C.

[0100] The update unit 67C updates values ​​such as the filter coefficients in the encoder 46 of the second defect determiner LM2 and the weights of the perceptron in the classifier 47 so as to reduce the loss included in the evaluation result.

[0101] The learning process is executed when a set number of learning data sheets or more have been accumulated. Therefore, the learning process is performed as a batch process using multiple learning data sheets. A series of processes from inputting the approved dataset DS2, which is the learning data, to updating it is repeated until the end timing is reached. Examples of the end timing include when learning of all the planned number of approved datasets DS2 has been completed, or when the loss falls below the target value.

[0102] When the end timing arrives, the learning unit 67 stores the additionally trained second defect determiner LM2A in the storage device 54. At this time, the additionally trained second defect determiner LM2A is tested by the test processing unit 68.

[0103] 15, the test processing unit 68 executes a test process on the second loss determiner LM2A that has undergone additional learning. The test process is initiated, for example, by a user instruction from the input device 51. In the test process, the test processing unit 68 inputs test data TD to the second loss determiner LM2A, causes the second loss determiner LM2A to execute a loss determination process, and outputs a test result TR. In this way, the test processing unit 68 executes a test process using the test data TD on the second loss determiner LM2A that has undergone learning processing, and can control the display 12A to display the test result TR.

[0104] Like the approved data set DS2, the test data TD is composed of a pair of a radiographic image P and a defect determination result RS as the correct answer data AD. In the test process, the second defect determiner LM2A that has undergone additional training outputs the defect determination result RS as the test result TR based on the radiographic image P of the test data TD. The test data TD is selected, for example, from the approved data set DS2, from among the data other than that selected as the training data. The test result TR is determined, for example, by the user. If the test result TR is not satisfactory, additional training is further performed by the training unit 67. If the test result TR is satisfactory, it is determined that the second defect determiner LM2A that has undergone additional training exhibits the target performance, and the additional training is terminated.

[0105] When the additional learning is completed, the second defect determiner LM2 before the additional learning is replaced by the second defect determiner LM2A that has undergone the additional learning, thereby updating the second defect determiner LM2.

[0106] When the additional learning is completed and the second failure determiner LM2 is updated, the update notification unit 69 transmits an update notification to the console 19 of the radiographic imaging device 11. That is, when the learning unit 67 executes learning processing on the second failure determiner LM2, the update notification unit 69 transmits an update notification to the console 19. The instruction to transmit the update notification is input, for example, via the input device 51 by a user who has confirmed the test result TR of the test processing in the failed image management device 12. The update notification is a notification that the first failure determiner LM1 of the console 19 can be updated.

[0107] When the CPU 32 receives the update notification, it displays a message such as "The first loss determiner LM1 can be updated" on the display 19A. When a user who has seen this message inputs an update instruction to the CPU 32 at any time, the CPU 32 executes an update process to update the first loss determiner LM1 by replacing the first loss determiner LM1 with the updated second loss determiner LM2. Of course, when the CPU 32 receives the update notification, the CPU 32 may execute an update process to update the first loss determiner LM1 at an appropriate time without waiting for the user to input an update instruction. In this way, the update notification is a trigger that causes the radiographic imaging device 11 to start the update process for the first loss determiner LM1, or a notification that can be displayed on the console 19.

[0108] The updated second failure determiner LM2 is transmitted, for example, from the failed image management device 12 to a shared folder in the storage device 34 of the console 19. The transmission process is performed by the storage processing unit 66, for example, when an instruction to transmit an update notification is input from the input device 51. In this way, the storage processing unit 66 stores the second failure determiner LM2, for which the learning process has been performed, in a shared folder in the storage device 34, which is a storage unit accessible by the console 19 of the radiographic image capture device 11. Note that the updated second failure determiner LM2 may be stored in the shared folder of the failed image management device 12, and the console 19 may access the shared folder to acquire the second failure determiner LM2. Furthermore, the update process of the first failure determiner LM1 using the second failure determiner LM2 may be performed via a computer other than the failed image management device 12 and the console 19, such as a maintenance device for the console 19.

[0109] The operation of the above configuration will be described below with reference to the flowcharts shown in Figures 17 to 20. When radiation imaging is performed using the radiation image capturing device 11, in step S1100, an unapproved data set DS1 is generated and output using the console 19.

[0110] As shown in FIG. 18, in step S1100, the console 19 first acquires a radiographic image P captured by the radiographic image detecting device 18 in step S1101, and displays the acquired radiographic image P on the captured image confirmation screen 19B in step S1102 (see FIG. 3). In step S1103, the console 19 waits for an input of an AI determination instruction via operation of the AI ​​determination button 41 (see FIG. 3). When the operator OP inputs the AI ​​determination instruction, the console 19 executes a defect determination by the first defect determiner LM1 using the radiographic image P as input data (step S1104). When the defect determination is executed by the first defect determiner LM1, a primary determination result RS1, which is the defect determination result RS by the first defect determiner LM1, is output. The output primary determination result RS1 is displayed on the captured image confirmation screen 19B (step S1105). The primary determination result RS1 and the radiographic image P are displayed as an unapproved data set DS1.

[0111] The operator OP checks the primary judgment result RS1 and the radiographic image P, and if necessary, for example, if the operator OP determines that the primary judgment result RS1 is inappropriate, the operator OP operates the secondary judgment input button 42 (see FIG. 3) to input a secondary judgment result RS2, which is the reject judgment result by the operator OP. Since the input of the secondary judgment result RS2 is performed at the discretion of the operator OP, if the primary judgment result RS1 and the judgment of the operator OP are the same, the same reject reason may be input, or the secondary judgment result RS2 may be omitted.

[0112] If the secondary determination result RS2 is input in step S1106, the secondary determination result RS2 by the operator OP is displayed on the captured image confirmation screen 19B in step S1107. If the secondary determination result RS2 is not input, the process proceeds to step S1108.

[0113] If re-imaging is necessary, the operator OP performs re-imaging by operating the re-imaging button 43. If re-imaging is performed, the console 19 repeats the processes from step S1101 to step S1107 for the re-imaging image PR, which is the re-imaging resultant radiation image P.

[0114] Then, when the operator OP operates the image output button 44, the unapproved data set DS1 including the radiation image P and the reject determination result RS is output to the image DB 37 and the rejected image DB 12C (step S1109).

[0115] In FIG. 17, the console 19 generates and outputs the unapproved data set DS1 in step S1100, and then transmits a generation notification of the unapproved data set DS1 to the failed image management device 12 in step S1200.

[0116] The failed image management device 12 waits for reception of a notification of the generation of an unapproved data set DS1 (step S2100). If an unapproved data set DS1 exists (YES in step S2100), the failed image management device 12 notifies the user, the medical staff member ST, by displaying that fact on a display screen, for example. Then, based on an instruction from the medical staff member ST, the failed image management device 12 generates and outputs an approved data set by conference (step S2200).

[0117] As shown in FIG. 19, in step S2200, the failed image management device 12 waits for a display instruction to be input via the display button 71 (FIG. 9) on the conference screen 12B. When the failed image management device 12 receives the display instruction to be input via the display button 71, the dataset acquisition unit 61 acquires an unapproved dataset DS1 from the failed image DB 12C (step S2202). Then, in step S2203, the display control unit 62 displays the unapproved dataset DS1 including the radiographic image P and the failure determination result RS on the conference screen 12B (see FIG. 9). If the unapproved dataset DS1 includes a re-photographed image PR, the re-photographed image PR is also displayed on the conference screen 12B. A plurality of medical staff members ST review the validity of the failure determination result RS while viewing the unapproved dataset DS1 displayed on the conference screen 12B of FIG. 9.

[0118] In step S2204, the failed image management device 12 waits for input of a correction instruction from the medical staff ST regarding the failed image determination result RS. Then, when a correction instruction for the failed image determination result RS is input by operating the correction button 73, the correction processing unit 63 corrects the failed image determination result RS based on the correction instruction (step S2205). Then, in step S2206, the failed image determination result RS corrected by the medical staff ST is displayed. If a correction instruction is not input in step S2204, the failed image management device 12 proceeds to step S2207.

[0119] In step S2207, the failed image management device 12 waits for input of an approval instruction from the medical staff ST regarding the failed image determination result RS. Then, when an approval instruction for the failed image determination result RS is input by operating the approval button 74, the approval processing unit 64 records the approval information (step S2208). When the approval information is recorded, the unapproved data set DS1 is changed to an approved data set DS2. The approval information is recorded as an approval history HSA in association with the approved data set DS2 including the failed image determination result RS. In step 2209, the failed image management device 12 waits for a save instruction by operating the save button 75, and when a save instruction is input, outputs an approved data set DS2 including the radiographic image P, the re-photographed image PR, and the approved failed image determination result RS. The approved data set DS2 is stored in the failed image DB 12C by the storage processing unit 66.

[0120] 17, the failed image management device 12 determines whether the accumulated number of learning data selected from the approved data set DS2 is equal to or greater than a set number. The learning unit 67 does not execute the learning process while the accumulated number of learning data is less than the set number, but if the accumulated number of learning data is equal to or greater than the set number (YES in step S2300), the process proceeds to step S2400, where the second failed image determiner LM2 executes additional learning.

[0121] In step S2400 shown in FIG. 20, first, the learning unit 67 executes a learning process using learning data selected from the approved data set DS2 (step S2401). In the learning process, additional learning of the second defect determiner LM2 is performed, as shown in FIG. 14. When the learning process is completed, a test process using test data TD is executed on the second defect determiner LM2 that has completed the additional learning, as shown in FIG. 15 (step S2402). When the test process is executed, the test result TR is displayed (step S2403). The test result TR is confirmed by the user. If the user determines that the test result TR is good, an instruction to end the test process is input, and the additional learning is terminated (YES in step S2404). If the test result TR is not good, an instruction to end the test process is not input (step S2404), and the learning process is repeated.

[0122] 17, when the additional learning is completed, the second loss determiner LM2A that has undergone additional learning is stored in the storage device 54. The second loss determiner LM2 in the storage device 54 is updated by the second loss determiner LM2A that has undergone additional learning. Then, in step S2600, the update notification unit 69 transmits an update notification to the console 19 of the radiographic imaging device 11, indicating that the first loss determiner LM1 can be updated by the second loss determiner LM2.

[0123] In step S1300, the console 19 waits to receive an update notification. Then, when the console 19 receives the update notification (YES in step S1300), it displays the update notification on the display 19A, as shown in Fig. 16, and executes the process of step S1400 based on a user instruction. In step S1400, the CPU 32 of the console 19 updates the first defect determiner LM1 by replacing it with the second defect determiner LM2 updated by additional learning.

[0124] As described above, the failed image management device 12 according to the technique of the present disclosure manages at least failed images among the radiographic images (for example, radiographic images P and re-photographed images PR) of a subject captured by the radiographic imaging device 11. The processor (for example, the CPU 52) of the failed image management device 12 is a first failure determiner (for example, the first failure determiner LM1) configured by a machine learning model that performs a failure determination for the radiographic image P, and executes a dataset acquisition process that acquires a dataset (for example, an unapproved dataset DS1) including a radiographic image input to the first failure determiner mounted on the console 19 of the radiographic imaging device 11 and a determination result (for example, a failure determination result RS) output from the first failure determiner LM1, and a display control process that controls displaying the dataset on a display. Furthermore, when a correction instruction is input for the judgment result of the displayed dataset, the processor executes a correction process to make corrections to the image defect judgment result, and a learning process to train a second image defect judger (for example, second image defect judger LM2) that can replace the first image defect judger and is not installed in the radiological image capturing device, in which the processor additionally trains the second image defect judger using a dataset selected from the displayed datasets as learning data.

[0125] This prevents a decrease in the accuracy of the determination by the first failure determiner installed in the radiographic imaging device. In other words, the failed image management device 12 according to the technology of the present disclosure has a function for displaying datasets such as the unauthorized dataset DS1 on a display, and can therefore be used, for example, in a conference where multiple medical staff members ST, such as doctors and technicians, review the failure determination results RS. When such a conference is held, the multiple medical staff members ST verify the validity of the failure determination results RS included in the displayed dataset. Furthermore, the failed image management device 12 can also correct the failure determination results RS as needed if the failure determination results RS are incorrect. Therefore, the failure determination results RS for the datasets displayed in the failed image management device 12 are more reliable than the sole judgment of the operator OP.

[0126] The rejected image management device 12 then executes a learning process to train a second rejected image determiner that can replace the first rejected image determiner, using a data set selected from the displayed data sets as learning data. Therefore, the second rejected image determiner is additionally trained using more reliable learning data than conventional methods, and the first rejected image determiner is replaced by the additionally trained second rejected image determiner, thereby updating the first rejected image determiner. Therefore, the technology disclosed herein can suppress a decrease in the determination accuracy of the first rejected image determiner installed in the radiographic imaging device, compared to conventional methods in which the first rejected image determiner is trained using learning data whose determination results are corrected solely by the operator's judgment.

[0127] 9, when the unapproved data set DS1, which is an example of a data set, includes both the primary judgment result RS1 output by the first defect determiner and the secondary judgment result RS2 by the operator OP, the display control unit 62 of the CPU 52 displays the primary judgment result RS1 and the secondary judgment result RS2 in a distinguishable manner in the display control process. This makes it possible to distinguish and verify the validity of each of the primary judgment result RS1 and the secondary judgment result RS2.

[0128] In the CPU 52, when a user inputs an approval instruction for the reject determination result RS of the unapproved dataset DS1, which is an example of a displayed dataset, the approval processing unit 64 records approval information (for example, approval history HSA) indicating approval in association with the dataset to be approved. By recording the approval information in association with the dataset, it becomes easier to search for approved datasets. Furthermore, since the recording of the approval information makes it clear that the dataset was approved at a conference, etc., it becomes easier to search for highly reliable datasets compared to when the approval information is not present.

[0129] Furthermore, the approval information refers to information indicating that a user has reviewed the reject determination result RS, which is an example of a determination result. The approved dataset is the dataset to be reviewed, and the approval information is an example of information indicating that a review has been conducted. That is, when the reject determination result RS has been reviewed by the user, the approval processing unit 64 records information indicating that the review has been conducted (for example, the approval information) in association with the unapproved dataset DS1 to be reviewed. Since the determination result of a reviewed dataset is considered to be highly reliable, recording information indicating that a review has been conducted makes it easier to search for a highly reliable dataset.

[0130] In this example, approval information is shown as an example of information indicating that a review has been performed. However, approval information and information indicating that a review has been performed may be distinguished. Both types of information may be recorded, or only one of them may be recorded. While approval information is preferable as information indicating that a dataset is highly reliable, information indicating that a review has been performed alone is also effective as information indicating that a dataset is highly reliable. When recording information indicating that a review has been performed, for example, an operation button for inputting a review record may be provided on the conference screen 12B of the failed image management device 12 in addition to or instead of the approval button 74. When this operation button is operated, the CPU 52 associates information indicating that a review has been performed with the dataset and records it.

[0131] Furthermore, instead of the approval information and information indicating that a review has been performed, a display history indicating that the dataset has been displayed on the display screen of the failed image management device 12 may be recorded. When a dataset is displayed on the failed image management device 12, it is highly likely that a review has been performed on the determination results included in the dataset, such as at a conference. In the failed image management device 12, when a dataset such as an unapproved dataset DS1 is displayed on a display screen such as the conference screen 12B, the CPU 52 records information indicating that the dataset has been displayed in association with the dataset. The CPU 52 records the information indicating the display, for example, as a display history. In the display history, for example, the ID information of the dataset and the display date and time are recorded, similar to the approval history HSA.

[0132] That is, in the above example, the learning data is selected from the approved dataset DS2, but the source of the learning data does not have to be the approved dataset DS2. Specifically, the source of the learning data may be a dataset in which information indicating that a review has been performed is recorded, and further, it may be a dataset that has at least a record of being displayed in the failed image management device 12.

[0133] Furthermore, the CPU 52 can store the approved data set DS2, which is an example of learning data, in the failed image DB 12C, which is an example of a storage unit. As shown in this example, by storing the learning data, the learning process of the second failure determiner LM2 can be performed by batch processing using multiple pieces of learning data. Performing the learning process by batch processing has the following advantages.

[0134] First, compared to sequential processing, which executes the learning process every time a piece of learning data is generated, batch processing can reduce the number of times the learning process is executed, thereby reducing the processing load.

[0135] Second, batch processing can reduce adverse effects on the judgment accuracy of the second image reject detector LM2 compared to sequential processing. This is because sequential processing processes each piece of learning data as it occurs. For example, if there is a series of learning data for the same image reject reason, the parameters are optimized for that image reject reason, which may result in a decrease in the judgment accuracy for other image reject reasons. For example, if there is a series of learning data for the image reject reason of internal rotation of the knee joint, the judgment accuracy for the image reject reason of insufficient dose may decrease. Thus, if there is bias in the learning data, sequential processing is likely to have an adverse effect on the judgment accuracy. Since batch processing involves a large number of learning data, bias in the learning data is likely to be reduced, which reduces adverse effects on the judgment accuracy compared to sequential processing.

[0136] Furthermore, in sequential processing, for example, if a piece of training data is unique and significantly deviates from the average characteristics of the training data that has already been trained, the training process will be performed using only the unique training data, which may have a negative impact on the accuracy of the judgment. In the case of batch processing, the characteristics of the unique training data are averaged by other training data, which is likely to suppress the negative impact on the accuracy of the judgment. For these reasons, batch processing is preferable to sequential processing. Note that although batch processing has been described in this example, sequential processing may also be performed.

[0137] 16 and 17, when the CPU 52 has executed the learning process for the second loss determiner LM2, it transmits an update notification to the console 19 of the radiographic imaging device 11, indicating that the first loss determiner LM1 can be updated by the second loss determiner LM2. By receiving the update notification, the console 19 can appropriately determine the timing to update the first loss determiner LM1 on the radiographic imaging device 11 side.

[0138] The update notification is a trigger for the radiographic imaging device 11 to start the update process of the first defect determiner LM1, or a notification that can be displayed on the console 19 of the radiographic imaging device 11. The console 19 of the radiographic imaging device 11 can update the first defect determiner LM1 at an appropriate update timing by starting the update process using the update notification as a trigger. Furthermore, if the update notification is a notification that can be displayed on the console 19, the user can update the first defect determiner LM1 at an appropriate timing.

[0139] The CPU 52 stores the second defect determiner LM2, for which the additional learning process has been executed, in a storage unit accessible to the radiographic imaging device 11. As shown in Fig. 16, the storage unit is, for example, a shared folder in the storage device 34 of the console 19 shown in Fig. 16. If the second defect determiner LM2 is stored in a storage unit accessible to the radiographic imaging device 11, it can be updated at an appropriate time on the radiographic imaging device 11 side.

[0140] The manner in which the second failure determiner LM2 updates the first failure determiner LM1 is not limited to this, and the CPU 52 of the failure image management device 12 may access the first failure determiner LM1 of the radiographic imaging device 11 and forcibly update it. Also, the first failure determiner LM1 may be updated using a maintenance device other than the console 19 and the failure image management device 12.

[0141] The CPU 52 can perform control to execute a test process using test data on the second defect determiner LM2 for which the additional learning process has been executed, and display the test result TR on the display 12A. The defective image management device 12 has a function to test the second defect determiner LM2 with test data, and can therefore provide a second defect determiner with higher determination accuracy as an update target compared to when the test process is not executed.

[0142] The second defect determiner LM2 immediately before the additional learning process is executed is the same as the first defect determiner LM1, and the CPU 52 executes additional learning on the second defect determiner LM2. This makes it easy for the defective image management device 12 to grasp the state of the first defect determiner LM1 used in the radiographic image capture device 11 and its performance, such as the degree to which the determination accuracy has improved as a result of the additional learning.

[0143] The first and second defect determiners LM1 and LM2 are not identical, and the second defect determiner LM2 may be an improved version of the first defect determiner LM1.

[0144] In the above example, the test process is executed after additional learning of the second defect determiner LM2, but the test process does not have to be executed. Of course, executing the test process is preferable in terms of reliably ensuring the quality of the second defect determiner LM2.

[0145] In the above example, the failed image management device 12 acquires all of the unapproved data sets DS1 in the failed image DB 12C as the subject of review in the conference, but it may also acquire a portion of the unapproved data sets DS1 and use the acquired portion as the subject of review. For example, in the dataset acquisition process, the CPU 52 may acquire a dataset that includes a failed image determination result indicating that the radiological image P may be a failed image in at least one of the primary determination result RS1 and the secondary determination result RS2, from among the datasets shown as an example of the unapproved data set DS1.

[0146] That is, as shown in FIG. 21 , the CPU 52 excludes an unapproved dataset DS1 for which there is no reject reason in both the primary judgment result RS1 and the secondary judgment result RS2 from the review target in the conference. This would impose a heavy review burden on the medical staff ST if all datasets including radiographic images P captured by the radiographic imaging device 11 were subject to review. Narrowing down the datasets to be acquired reduces the review burden. Furthermore, a dataset for which there is no reject reason in both the primary judgment result RS1 and the secondary judgment result RS2 is highly unlikely to be a rejected image. Even if such datasets are excluded, the number of training data with reject reasons can be secured.

[0147] Furthermore, in the above embodiment, the reject determination result RS is shown in text, but it may also be displayed as an image. For example, an image may be output in which the area in the radiographic image P that is the basis for the reject reason is highlighted. As an example of this image, an image with annotations indicating the area that is the basis may be used. As another example, a heat map in which the color or density is changed according to the degree of contribution to the determination of the reject reason may be used. Note that the reject determination result RS may also be displayed in text in addition to the heat map. Furthermore, the reject determination result RS may include an enlarged image of the area in the radiographic image P that is the basis for the reject reason. For example, if the reject reason is internal rotation of the knee joint, the reject determination result RS may include an enlarged image of the knee joint.

[0148] Furthermore, in the above embodiment, an example has been described in which the failed image management device 12 only has the additional learning function of the second failure determiner LM2, and the radiographic image capturing device 11 does not have the additional learning function, but the radiographic image capturing device 11 may also have the additional learning function of the first failure determiner LM1. Although the additional learning function of the radiographic image capturing device 11 has issues similar to those of the conventional technology, by combining it with the failed image management device 12 of this example, it is expected to have the effect of suppressing a decrease in determination accuracy.

[0149] Furthermore, although the radiation image capturing device 11 has been described as being installed in an imaging room, the radiation image capturing device 11 may also be a mobile imaging device that can be moved by a dolly.

[0150] Furthermore, the technology of the present disclosure is not limited to X-rays, but can also be applied to systems that use other radiation such as gamma rays to image a subject.

[0151] In each of the above embodiments, the hardware structure of the processing units that perform various processes, such as the dataset acquisition unit 61, display control unit 62, correction processing unit 63, approval processing unit 64, storage processing unit 66, learning unit 67, test processing unit 68, and update notification unit 69, is various processors as shown below.

[0152] Various types of processors include CPUs, programmable logic devices (PLDs), and dedicated electrical circuits. As is well known, a CPU is a general-purpose processor that executes software (programs) and functions as various processing units. A PLD is a device such as an FPGA (Field Programmable Gate Array) that can change its circuit configuration after manufacturing. A dedicated electrical circuit is a processor that has a circuit configuration designed specifically to perform a specific process, such as an ASIC (Application Specific Integrated Circuit). The memory may be connected to the processor, such as memory connected to the CPU via a data bus, or may be built into the processor itself, such as in an FPGA.

[0153] A single processing unit may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, multiple FPGAs, or a combination of a CPU and an FPGA). Also, multiple processing units may be configured with a single processor. As an example of configuring multiple processing units in one processor, first, there is a form in which one processor is configured by combining one or more CPUs and software, and this processor functions as multiple processing units. Second, there is a system on chip (System On Chip). As typified by SoCs, there is a form in which a processor is used to realize the functions of an entire system including multiple processing units on a single IC chip. In this way, various processing units are configured as a hardware structure using one or more of the above-mentioned various processors.

[0154] Furthermore, the hardware structure of these various processors is, more specifically, an electric circuit made up of a combination of circuit elements such as semiconductor elements.

[0155] The present invention is not limited to the above-described embodiments, and various configurations can be adopted without departing from the spirit of the present invention. Furthermore, the present invention extends to a computer-readable storage medium that non-temporarily stores a program, in addition to the program itself.

Claims

1. A failed image management device that manages at least failed images among radiographic images of a subject captured by a radiographic image capturing device, A processor and a memory embedded in or connected to the processor, The processor: a first defect determiner configured by a machine learning model and performing defect determination on the radiographic image, the first defect determiner being mounted on the radiographic image capturing device; and a data set acquisition process for acquiring a data set including the radiographic image input to the first defect determiner and a determination result output from the first defect determiner; a display control process for controlling the display of the data set on a display; a correction process for correcting the determination result when an instruction to correct the determination result of the displayed data set is input; a learning process for training a second defect determiner that can replace the first defect determiner and is not installed in the radiographic image capturing apparatus, the second defect determiner being trained using a data set selected from the displayed data sets as learning data; When the data set includes both a primary determination result, which is the determination result output by the first defect determiner, and a secondary determination result, which is the determination result by an operator of the radiographic image capturing device, the primary determination result and the secondary determination result are displayed in a distinguishable manner in the display control process. A defective image management device.

2. In the data set acquisition process, the processor acquires, from the data sets, a data set that includes a failure determination result indicating that the radiographic image may be a failure image in at least one of the first determination result and the second determination result. The failed image management device according to claim 1 .

3. 3. The failed image management device according to claim 1, wherein when a user inputs an approval instruction for the judgment result of the displayed data set, the processor records approval information indicating that approval has been granted in association with the data set to be approved.

4. A failed image management device that manages at least failed images among radiographic images of a subject captured by a radiographic image capturing device, A processor and a memory embedded in or connected to the processor, The processor: a first defect determiner configured by a machine learning model and performing defect determination on the radiographic image, the first defect determiner being mounted on the radiographic image capturing device; and a data set acquisition process for acquiring a data set including the radiographic image input to the first defect determiner and a determination result output from the first defect determiner; a display control process for controlling the display of the data set on a display; a correction process for correcting the determination result when an instruction to correct the determination result of the displayed data set is input; a learning process for training a second defect determiner that can replace the first defect determiner and is not installed in the radiographic image capturing apparatus, the second defect determiner being trained using a data set selected from the displayed data sets as learning data; When a user reviews the determination result, information indicating that the review has been performed is recorded in association with the dataset to be reviewed. A defective image management device.

5. The failed image management device according to claim 1 , wherein the processor is capable of storing the learning data in a storage unit.

6. The processor does not execute the learning process while the number of accumulated learning data is less than a preset number, and executes the learning process when the number of accumulated learning data reaches the preset number.

7. The processor, when executing the learning process on the second image defect determiner, sends an update notification to the radiographic imaging device indicating that the first image defect determiner can be updated by the second image defect determiner.

8. The failed image management device according to claim 7 , wherein the update notification is a trigger for the radiographic image capturing device to start an update process for the first failed image determiner, or a notification that can be displayed on a console of the radiographic image capturing device.

9. The failed image management device according to claim 1 , wherein the processor stores the second failed image determiner on which the learning process has been performed in a storage unit accessible by the radiographic imaging device.

10. the processor executes a test process using test data on the second defect determiner on which the learning process has been executed, The failed image management device according to claim 1 , wherein the test results can be controlled to be displayed on a display.

11. the second defect determiner immediately before the learning process is executed is the same as the first defect determiner, The failed image management device according to claim 1 , wherein the processor executes the learning process on the second failed image determiner.

12. A method for operating a failed image management device that manages at least failed images among radiation images of a subject captured by a radiation image capturing device, comprising: a first defect determiner configured by a machine learning model and performing defect determination on the radiographic image, the first defect determiner being mounted on the radiographic image capturing device; and a data set acquisition process for acquiring a data set including the radiographic image input to the first defect determiner and a determination result output from the first defect determiner; a display control process for controlling the display of the data set on a display; a correction process for correcting the determination result when an instruction to correct the determination result of the displayed data set is input; a learning process for training a second defect determiner that can replace the first defect determiner and is not installed in the radiographic image capturing apparatus, the learning process training the second defect determiner using a data set selected from the displayed data sets as learning data; When the data set includes both a primary determination result, which is a determination result output by the first defect determiner, and a secondary determination result, which is a determination result by an operator of the radiographic image capturing device, the display control process displays the primary determination result and the secondary determination result in a distinguishable manner; A method for operating a failed image management device, comprising:

13. a first defect determiner configured by a machine learning model and performing defect determination on a radiographic image, the first defect determiner being mounted on a radiographic image capturing device; and a data set acquisition process for acquiring a data set including the radiographic image input to the first defect determiner and a determination result output from the first defect determiner; a display control process for controlling the display of the data set on a display; a correction process for correcting the determination result when an instruction to correct the determination result of the displayed data set is input; a learning process for training a second defect determiner that can replace the first defect determiner and is not installed in the radiographic image capturing apparatus, the learning process training the second defect determiner using a data set selected from the displayed data sets as learning data; When the data set includes both a primary determination result, which is a determination result output by the first defect determiner, and a secondary determination result, which is a determination result by an operator of the radiographic image capturing device, the display control process displays the primary determination result and the secondary determination result in a distinguishable manner; An operating program that causes a computer to execute the above.

14. The operating program according to claim 13, which causes the computer to function as a failed image management device that manages at least failed images among the radiation images of the subject captured by the radiation image capturing device.

15. A radiographic image capturing system comprising: the failed image management device according to any one of claims 1 to 11; and a radiographic image capturing device.

Images