X-ray diagnostic apparatus and method

Abstract

To improve the efficiency of a series of processes including X-ray image acquisition and image reconstruction processing.SOLUTION: An X-ray diagnostic apparatus according to an embodiment includes: a setting unit configured to obtain a second exposure parameter set by inputting an X-ray image acquired using a first exposure parameter set to a first model; an acquisition unit configured to obtain a reconstructed image by inputting an X-ray image acquired using the second exposure parameter set to a second model trained together with the first model; and a control unit configured to output the reconstructed image.SELECTED DRAWING: Figure 5

Description

[Technical Field]

[0001] SUMMARY OF THE INVENTION The embodiments disclosed herein and in the drawings relate to X-ray diagnostic devices and methods. [Background technology]

[0002] The background discussion provided herein is intended to generally set forth the context for the present disclosure. The work of the presently named inventors, to the extent that it is described in this background section, as well as aspects of this specification that may not be admitted as prior art at the time of filing, are not admitted expressly or impliedly as prior art to the present disclosure.

[0003] The image quality of a fluoroscopy sequence can depend on both the exposure conditions and the image restoration process. Some exposure control processes can determine the acquisition or exposure parameters (e.g., mA, ms, kV, filters, etc.) of a single frame based on the content (e.g., histogram) of the current frame to maximize image quality (e.g., contrast-to-noise ratio). However, the radiation dose during fluoroscopy can generally be set low, which can pose challenges for downstream image denoising and image restoration processes. Therefore, a method for providing various customized acquisition parameters during fluoroscopy prior to image denoising and image restoration processes is desirable. [Prior art documents] [Non-patent literature]

[0004] [Non-Patent Document 1] Yim D. C ET AL: "A deep convolutional neural network for simultaneous denoising and deblurring in computed tomography", Journal of Instrumentation, 1 December 2020 (2020-12-01), pages 1-12, XP055982807, Retrieved from the Internet:URL:https: / / iopscience.iop.org / article / 10.1088 / 1748-0221 / 15 / 12 / P12001 / pdf [retrieved on 2022-11-17] Summary of the Invention [Problem to be solved by the invention]

[0005] One of the problems to be solved by the embodiments disclosed in this specification and the drawings is to improve the efficiency of a series of processes including X-ray image collection and image restoration processing. However, the problems to be solved by the embodiments disclosed in this specification and the drawings are not limited to the above problem. Problems corresponding to the effects of each configuration shown in the embodiments described below can also be positioned as other problems. [Means for solving the problem]

[0006] An X-ray diagnostic apparatus according to an embodiment includes a setting unit that inputs an X-ray image acquired using a first exposure parameter set into a first model to acquire a second exposure parameter set, an acquisition unit that inputs the X-ray image acquired using the second exposure parameter set into a second model trained together with the first model to acquire a restored image, and a control unit that outputs the restored image. [Brief explanation of the drawings]

[0007] [Figure 1] FIG. 1 is a schematic diagram illustrating a collection parameter adjustment process according to one embodiment of the present disclosure. [Figure 2] FIG. 2 is a schematic diagram illustrating a process of adjusting acquisition parameters via frame rate adjustment according to one embodiment of the present disclosure. [Figure 3] FIG. 3 is a schematic diagram of an image restoration process according to one embodiment of the present disclosure. [Figure 4] FIG. 4 is a schematic diagram illustrating a neural network architecture for image restoration according to one embodiment of the present disclosure. [Figure 5] FIG. 5 is a schematic diagram illustrating training of a neural network according to one embodiment of the present disclosure. [Figure 6A] FIG. 6A is a schematic diagram illustrating the generation of a training dataset according to one embodiment of the present disclosure. [Figure 6B] FIG. 6B is a schematic diagram illustrating the generation of a training dataset according to one embodiment of the present disclosure. [Figure 7A] FIG. 7A illustrates a non-limiting example flowchart of a method for adjusting collection parameters, according to an embodiment of the present disclosure. [Figure 7B] FIG. 7B illustrates a non-limiting example flowchart of a method for adjusting collection parameters, according to an embodiment of the present disclosure. [Figure 7C] FIG. 7C illustrates a non-limiting example flowchart of a method for adjusting collection parameters according to one embodiment of the present disclosure. [Figure 7D] FIG. 7D illustrates a non-limiting example of a method including acquisition parameter adjustment and image restoration using a trained model according to one embodiment of the present disclosure. [Figure 8A] FIG. 8A illustrates an example of a general artificial neural network (ANN) with N inputs, K hidden layers, and 3 outputs, according to one embodiment of the present disclosure. [Figure 8B] FIG. 8B illustrates a non-limiting example of a Convolutional Neural Network (CNN) according to one embodiment of the present disclosure. [Figure 9]FIG. 9 is a block diagram of an X-ray device according to an embodiment of the present disclosure. [Figure 10] FIG. 10 is a schematic diagram of a hardware system for performing a method according to an embodiment of the present disclosure. [Figure 11] FIG. 11 is a schematic diagram of a hardware configuration of an apparatus for performing a method according to an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0008] The following disclosure defines many different embodiments or examples for realizing different features of the provided subject matter. To simplify the disclosure, specific examples of components and arrangements are described below. Of course, these are merely examples and are not intended to be limiting. For example, in the following description, the formation of a first feature that is not in contact with or in contact with a second feature may include an embodiment in which the first feature and the second feature are formed in direct contact with each other, and may also include an embodiment in which an additional feature may be formed between the first feature and the second feature such that the first feature and the second feature are not in direct contact with each other. Furthermore, the present disclosure may repeat reference numerals and / or letters in various examples. This repetition is for simplicity and clarity and does not in itself dictate a relationship between the various embodiments and / or configurations described. Furthermore, spatially relative terms such as “top,” “bottom,” “below,” “belower,” “lower of,” “upper,” and the like may be used herein for ease of description and to describe the relationship of one element or feature to another element(s) or feature(s), as shown in the drawings. Spatially relative terms are intended to encompass various orientations of the device in use or operation in addition to the orientation depicted in the drawings. The device may be in other orientations (rotated 90 degrees or at other orientations) and the spatially relative descriptors used herein interpreted accordingly.

[0009] The order of description of the various steps described herein is provided for clarity. In general, these steps may be performed in any suitable order. In addition, although different features, techniques, configurations, etc. herein may be discussed in different locations in this disclosure, it is contemplated that the concepts may be implemented independently of each other or in combination with each other. Thus, the embodiments may be embodied and tested in various ways.

[0010] X-ray imaging systems and methods are medical imaging tools widely used in diagnosis and clinical intervention. Imaging systems are essential tools for imaging-guided therapeutic procedures lasting from minutes to hours. Imaging systems typically create two-dimensional projection images through a subject's body. A radiation source, such as an x-ray tube, delivers radiation from one side of the body. A collimator is typically adjacent to the x-ray source and limits the angular range of the x-ray beam. This substantially confines the radiation delivered to the subject's body to a cone-beam / fan-beam region (i.e., x-ray projection volume) that defines the body's image volume. At least one detector on the opposite side of the body receives radiation that has passed through the body substantially within the projection volume. The attenuation of the radiation passing through the body is measured by processing electrical signals received from the detector.

[0011] In both fluoroscopy and acquisition modes, a sequence of x-ray exposures can be acquired at a selected frame rate, dose, tube voltage, and scan time. In clinical practice, the object of interest in the image to the radiologist or operator may change frequently during an interventional procedure (e.g., from a guidewire to a catheter, iodinated vessel, or stent), and exposure parameters can be adjusted to better image the object. Particularly during periods of low object movement or change, exposure parameters can be adjusted to reduce the patient's dose or radiation exposure level.

[0012] Some fluoroscopy systems can perform image restoration processing after images are acquired by the system. In some fluoroscopy systems, noise reduction and contrast enhancement may be desirable goals for automatic exposure determination and adjustment. Some automatic exposure control processes can control the fluoroscopy system to maximize output image quality from a physical perspective.

[0013] The image restoration process may rely on prior information. For example, an image sequence may provide prior information that can be used for image restoration. In particular, exposure control may rely on the histograms of the images (in the image sequence) and may be processed separately from the subsequent image restoration process. Such exposure control processing methods do not necessarily use prior information.

[0014] Thus, the present embodiment describes an acquisition parameter adjustment system and method that can control acquisition parameters through the image restoration process.

[0015] In one embodiment, a neural network can be used to identify informative frames (or informative scenarios, or informative images). As described herein, an informative frame can be a frame or image that contains a change or difference compared to a previous frame (or frames). For example, a frame containing a deployed object, such as a stent, can be an informative frame, where the previous frame did not contain the stent or the position of the stent did not change by more than a threshold parameter. For example, a frame containing a blood vessel injected with contrast agent can be an informative frame, where the previous frame did not contain the injected contrast agent. For example, an informative frame can be a frame containing an object that rapidly or frequently changes characteristics or parameters, such as a region containing blood flow, heartbeat, etc. Generally, an informative frame is a frame or image in which a significant event or change occurs. Therefore, more accurate or less noise acquisition of the informative frame and frames following the informative frame may be desirable.

[0016] In one embodiment, once a frame is determined to be an informative frame, exposure parameters can be adjusted based on the informative frame, including but not limited to dose, acquisition frame rate, acquisition time, voltage, and filter, to obtain an informative frame with more optimized exposure parameters.

[0017] In one embodiment, a neural network can be used to analyze informative frames and corresponding exposure parameters as training data, particularly simulated and clinical images.

[0018] In this regard, FIG. 1 is a schematic diagram illustrating an acquisition parameter adjustment process according to an embodiment of the present disclosure. In one embodiment, a first image 105 can be obtained via a fluoroscopy system having a first exposure setting or exposure parameters. The fluoroscopy system can include, for example, an X-ray source configured to emit X-rays at a predetermined intensity and a processing circuit, such as a computer, that analyzes the image collected by the fluoroscopy system. In particular, the first image 105 can be a low-dose image as shown. The low dose of the first image 105 can be understood to mean that the first image 105 was collected using exposure parameters including a low-dose setting or low radiation level of the X-ray source (or other radiation source in other imaging systems). The first image 105 can be analyzed by the processing circuit (fluoroscopy system) to identify any object, such as a stent, located within the first image 105. In the first image 105, the processing circuit can determine a first parameter of the object, such as a first position of the object.

[0019] In one embodiment, a second image 110 can be acquired via a fluoroscopy system at a first exposure setting. The second image 110 can be a low-dose image, as shown. The second image 110 can be analyzed by processing circuitry to identify an object located in the second image 110 and a second parameter of the object, such as a second position of the object. For example, the object can move during acquisition, and the second position of the object is different from the first position. The processing circuitry can determine the difference between the first parameter and the second parameter, such as the first position and the second position. The difference between the first position and the second position can be expressed as a change in distance. The difference between the first position and the second position can be used to determine whether the first image 105 and the second image 110 are informative frames.

[0020] In one embodiment, a first exposure control process 130 (first exposure control step) executed by the processing circuitry (fluoroscopy system) can analyze the first image 105 and the second image 110 to determine whether the first image 105 and the second image 110 are informative frames. Again, the object (or objects) in the first image 105 and the second image 110 can be determined, as can corresponding first and second parameters (e.g., corresponding first and second positions). If it is determined that the difference between the first position of the object in the first image 105 and the second position of the object in the second image 110 is greater than a predetermined threshold (or a specific threshold, or a set threshold, or a target threshold), the exposure control process can determine that the first image 105 and the second image 110 are informative frames. That is, the first image 105 and the second image 110 contain object changes that can be imaged at a higher dose or exposure to acquire images with less noise compared to the same images obtained at a lower dose.

[0021] In one embodiment, the purpose of increasing the exposure does not have to be noise reduction. For example, informative frames can provide information other than noise reduction. For example, general tracking of objects can be the goal. For example, high-dose images can identify objects that can be subsequently tracked in low-dose images. Collecting high-dose images can establish object parameters, such as shape, size, etc., of the object to be tracked through the collection of low-dose images, while still effectively or more accurately tracking the object compared to a process using only low-dose images, without increasing the dose of future low-dose image collections.

[0022] In one embodiment, the first parameter (and second parameter, and subsequently collected parameters of the object) may be, for example, the shape of the object, the size of the object, the density of the object, or other measurable characteristic of the object as it appears in the image or is represented in the imaging (projection) data. In one example, the object is an artery with blood flow, and the varying density (or corresponding varying brightness / contrast) of the blood flow can indicate that a frame or image capturing the blood flow is an informative frame.

[0023] In one embodiment, once the first image 105 and the second image 110 are determined to be useful frames, the fluoroscopy system can adjust first exposure parameters 135 via a first exposure control 140. The first exposure parameters 135 can be adjusted based on the useful frames, such as the exposure parameters of the useful frames, the objects in the useful frames, and the determined image parameters (brightness, contrast, etc.) of the useful frames. The first exposure control 140 can be configured to adjust, for example, the dose, voltage, frame rate, filters, etc., in the first exposure parameters 135. As shown in FIG. 1 , the first exposure control 140 can increase the dose, followed by acquisition of a third image 115. Thus, the third image 115 can be labeled as a high-dose image. Similarly, the fourth image 120 can also be acquired at a high dose.

[0024] In one embodiment, if the first image 105 and the second image 110 are determined to be not informative frames, the fluoroscopy system may maintain the current first exposure parameters 135 used to acquire subsequent images, which may continue to be considered low-dose images. For example, the fluoroscopy system may determine that the difference between the first position of the object in the first image 105 and the second position of the object in the second image 110 is not greater than a predetermined threshold, and maintain the low dose used to acquire the images.

[0025] In one embodiment, the processing circuitry can analyze the second image 110 and the third image 115 to determine whether the second image 110 and the third image 115 are informative frames. Again, an object of interest (or objects of interest) can be determined in the second image 110 and the third image 115, and a corresponding second position of the object in the second image 110 and a corresponding third position of the object in the third image 115 can be determined. If it is determined that the difference between the second position of the object in the second image 110 and the third position of the object in the third image 115 is still greater than a predetermined threshold, then the high dose can be maintained.

[0026] In one embodiment, a second exposure control process 145 (second exposure control step) executed by the processing circuitry can analyze the third image 115 and the fourth image 120 to determine whether the third image 115 and the fourth image 120 are useful frames. Again, an object (or objects) can be determined in the third image 115 and the fourth image 120, and a corresponding third position of the object in the third image 115 and a corresponding fourth position of the object in the fourth image 120 can be determined. If it is determined that the difference between the third position of the object in the third image 115 and the fourth position of the object in the fourth image 120 is not greater than a predetermined threshold, the exposure control process can determine that the third image 115 and the fourth image 120 are not useful frames. That is, it can determine that there is negligible or no change in the object between the third image 115 and the fourth image 120. This may occur, for example, in a clinical environment where a patient is being imaged during a medical procedure but medical personnel are not currently adjusting the object. For example, a surgeon may guide a stent to the imaging site and place the stent temporarily, thus allowing high-dose imaging to be used while the stent is being guided into position, but once the stent is in place (either temporarily or permanently), imaging of the subject with high resolution or low noise quality is not required, and the patient does not need to be exposed to high doses.

[0027] In one embodiment, if the third image 115 and the fourth image 120 are determined not to be useful frames, the fluoroscopy system can adjust the second exposure parameters 150 via the second exposure control 155. The second exposure parameters 150 can be adjusted back to a low dose, i.e., a dose lower than the dose used to acquire the third image 115 and the fourth image 120, which were identified as high-dose images. The second exposure control 155 can be configured to adjust, for example, the dose, voltage, imaging time, frame rate, filters, etc., of the second exposure parameters 150. As shown in FIG. 1 , for example, the dose can be reduced via the second exposure control 155, followed by acquisition of the fifth image 125. Thus, the fifth image 125 can be labeled as a low-dose image.

[0028] In one embodiment, the first exposure control 140 and the second exposure control 155 may be applications or modules included in a fluoroscopy system configured to receive data regarding useful frames and / or data regarding exposure parameters used to obtain the useful frames and adjust the fluoroscopy system. For example, the first exposure control 140 and the second exposure control 155 may be configured to adjust the exposure parameters described above, such as dose, voltage, frame rate, and filters. The first exposure control 140 and the second exposure control 155 may be configured to adjust the exposure parameters via a processor, such as an ASIC or FPGA. In one embodiment, the first exposure control 140 and the second exposure control 155 may be configured to adjust the exposure parameters via manual input from a user, such as a technician. For example, the technician may continuously monitor the generated images and adjust the exposure parameters using an input device. For example, the technician may use a software program configured to adjust the exposure parameters, adjust physical features (e.g., knobs, sliders, buttons, switches, etc.) on the equipment or computing device, etc. Updated images may be generated based on the adjusted exposure parameters for further analysis and adjustment. For example, in the case of manual input, a technician can visually inspect and analyze the updated image to further adjust the exposure parameters, or in the case of a processor or computing device, a trained model can receive the updated image as input and use the trained model to further adjust the exposure parameters.

[0029] In one embodiment, the first exposure control process 130 (and the second exposure control process 145, as well as any additional exposure control processes) may be a neural network configured to receive image data as input. The output of the neural network may be a determination of whether an image corresponding to the image data is an informative frame. That is, for example, the first exposure control process 130 and the second exposure control process 145 may receive two of the captured images and determine whether the image contains an object and whether there is a significant change in the object between the two images. For example, the first exposure control process 130 and the second exposure control process 145 may determine whether a difference between a first position of an object in the first image 105 and a second position of the object in the second image 110 is greater than a predetermined threshold.

[0030] In one embodiment, the image data input for the first exposure control process 130 and the second exposure control process 145 may include two or more images (or frames). The images or frames may also be used as input for an image restoration process, which uses a neural network to restore the image. The restored images obtained and the quality of the restored images may be used as feedback to the acquisition parameter adjustment process. In particular, the quality of the restored images may be used as feedback to the exposure control process (described below).

[0031] FIG. 2 is a schematic diagram illustrating an acquisition parameter adjustment process via frame rate adjustment, according to one embodiment of the present disclosure. In one embodiment, a first image 105 may be acquired via a fluoroscopy system at a first exposure setting. In particular, the first image 105 may be a low-dose image, as shown. The low dose of the first image 105 may be understood to mean that the first image 105 was acquired using exposure parameters including a low exposure or low-dose setting of the radiation source. The first image 105 may be analyzed by processing circuitry to identify any objects, such as a stent, located within the first image 105. In the first image 105, the processing circuitry may determine a first position of the object.

[0032] In one embodiment, a second image 110 can be acquired via a fluoroscopy system at a first exposure setting. The second image 110 can be a low-dose image as shown. The second image 110 can be analyzed by processing circuitry to identify an object located within the second image 110 and a second position of the object. For example, the object can move during acquisition, and the second position of the object is different from the first position. The difference between the first position and the second position can be used to determine whether the first image 105 and the second image 110 are informative frames.

[0033] In one embodiment, a first exposure control process 130 executed by the processing circuitry (fluoroscopy system) can analyze the first image 105 and the second image 110 to determine whether the first image 105 and the second image 110 are informative frames. Again, an object (or objects) can be determined in the first image 105 and the second image 110, as well as a corresponding first position and a corresponding second position. If it is determined that the difference between the first position of the object in the first image 105 and the second position of the object in the second image 110 is greater than a predetermined threshold, the exposure control process can determine that the first image 105 and the second image 110 are informative frames. That is, the first image 105 and the second image 110 contain object changes that can be imaged at a higher dose or exposure to obtain images with less noise compared to the same images obtained at a lower dose. As previously mentioned, the purpose of increasing the exposure need not be noise reduction; other purposes may be desired.

[0034] In one embodiment, once the first image 105 and the second image 110 are determined to be useful frames, the fluoroscopy system can adjust first exposure parameters 135 via a first exposure control 140. The first exposure parameters 135 can be adjusted based on the useful frames, such as the exposure parameters of the useful frames, the objects in the useful frames, and the determined image parameters (brightness, contrast, etc.) of the useful frames. The first exposure control 140 can be configured to adjust, for example, the dose, voltage, frame rate, filters, etc., in the first exposure parameters 135. As shown in FIG. 2 , the first exposure control 140 can increase the frame rate, followed by acquisition of a third image 115. Thus, the third image 115 can be labeled as still a low-dose image but acquired at a high frame rate. Similarly, the fourth image 120 can also be acquired at a high frame rate.

[0035] In one embodiment, if the first image 105 and the second image 110 are determined to be not informative frames, the fluoroscopy system may maintain the current first exposure parameters 135 used to acquire subsequent images, which may continue to be considered low-frame-rate, low-dose images. For example, if the fluoroscopy system determines that the difference between the first position of the object in the first image 105 and the second position of the object in the second image 110 is not greater than a predetermined threshold, the fluoroscopy system may maintain the low frame rate (dose) used to acquire the images.

[0036] In one embodiment, the third image 115 and the fourth image 120 may be analyzed by a second exposure control process 145 executed by the processing circuit to determine whether the third image 115 and the fourth image 120 are useful frames. Again, an object (or objects) may be determined in the third image 115 and the fourth image 120, and a corresponding third position of the object in the third image 115 and a corresponding fourth position of the object in the fourth image 120 may be determined. If it is determined that the difference between the third position of the object in the third image 115 and the fourth position of the object in the fourth image 120 is not greater than a predetermined threshold, the exposure control process may determine that the third image 115 and the fourth image 120 are not useful frames. That is, it may be determined that there is negligible or no change in the object between the third image 115 and the fourth image 120.

[0037] In one embodiment, if the third image 115 and the fourth image 120 are determined to be not useful frames, the fluoroscopy system can adjust the second exposure parameters 150 via the second exposure control 155. The second exposure parameters 150 can be adjusted back to a lower frame rate or a less frequent frame rate than the frame rate used to acquire the third image 115 and the fourth image 120, which were acquired using the higher frame rate. As shown in FIG. 2, the frame rate can be reduced by the second exposure control 155, and the fifth image 125 can then be acquired.

[0038] Although described in one embodiment as parameters that are adjusted separately, the aforementioned parameters can be adjusted together, i.e., the dose level and frame rate can be adjusted together to obtain subsequent images with less noise. Furthermore, the dose can be adjusted in combination with different parameters, such as acquisition time.

[0039] 1 and 2 illustrate an example in which the determination of whether a frame is useful is performed based on multiple images, but the determination of whether a frame is useful may also be performed based on a single image. For example, the determination of whether a frame is useful may be performed based on the first image 105 shown in FIG. 1 or 2, and adjusted exposure parameters may be acquired based on the determination result. As an example, if the first image 105 contains an object such as a stent or a contrast agent, the first image 105 may be determined to be a useful frame. If the first image 105 is a useful frame, the exposure parameters may be adjusted to increase the dose and frame rate, and the second image 110 may be acquired.

[0040] FIG. 3 is a schematic diagram of an image restoration process 300 according to one embodiment of the present disclosure. As previously described, a sequence of images or frames can be used as input to the image restoration process 300. The image restoration process 300 can include a neural network trained to restore the image. In one embodiment, the neural network for restoring the image can be used to improve image quality. The image restoration process 300 can include noise removal, deblurring, artifact removal, etc. The image data input to the image restoration process 300 can include two or more images (or frames). As shown, a current frame (t), a previous frame (t-1), and up to any number of frames (tn) can be used as input to the image restoration process 300, which generates a restored image. Again, the obtained restored image and the quality of the restored image can be used as feedback to the acquisition parameter adjustment process. In particular, the quality of the restored image can be used as feedback to the exposure control process.

[0041] FIG. 4 is a schematic diagram illustrating a neural network architecture for image restoration according to one embodiment of the present disclosure. In one embodiment, the input of the neural network can include image data, which can include an image sequence such as that obtained by a fluoroscopy system. The neural network architecture can include a first encoder and a second encoder configured to extract features from images of different exposure parameters. The encoders can generally be configured to convert high-dimensional input data into a lower-dimensional representation that retains the information necessary for further processing in various machine learning tasks. For example, the first encoder can be configured to extract features from high-dose images, and the second encoder can be configured to extract features from low-dose images. The encoders can be used in a variety of other neural network architectures, such as autoencoders, convolutional neural networks (CNNs), and recurrent neural networks (RNNs).

[0042] In one embodiment, the neural network architecture can also include a non-local block, specifically a fuse non-local block (FNB). The non-local block can be an image block module used in a neural network. The neural network wraps non-local operations and captures long-range dependencies in data, such as image data. The FNB is able to function by allowing each element in the input to interact with every other element regardless of the spatial or temporal distance between them, and can fuse features from images with different exposure settings. For example, by calculating a similarity value between every pair of image locations, the FNB can help the neural network determine any connections between images. The similarity value can be used to adjust the importance or score of each pixel in the image, so that more strongly related pixels are weighted more.

[0043] In one embodiment, the neural network can include a decoder configured to accept as input the fused features from the image to generate and output a reconstructed image. For example, the neural network architecture can use an autoencoder, which includes an encoder and a decoder, where the encoder maps the input data to a low-dimensional latent space representation (latent vector), and the decoder attempts to reconstruct the original data from the low-dimensional latent space representation. By attempting to match the output to the input, the autoencoder can learn to grasp relevant information and ignore unimportant data. Autoencoders can be used for tasks such as data compression, noise removal, and anomaly detection.

[0044] In particular, the neural network of the first exposure control process 130 (or the second exposure control process 145) and the neural network of the image restoration process 300 can be trained alternately.

[0045] FIG. 5 is a schematic diagram illustrating training of a neural network according to an embodiment of the present disclosure. In one embodiment, the neural network of the image restoration process 300 can be fixed, while the neural network of the first exposure control process 130 can be trained. Referring to FIG. 5 , a first input frame n1 and a second input frame n2 can be input to the exposure control process 505, which has a neural network for training. An image quality simulator 515 can be configured to simulate the simulated degraded frame n2. The image quality simulator 515 can simulate an image based on inputs including exposure parameters 510 (similar to the exposure parameters described above). The loss function of the exposure control process 505 can be defined to (i) maximize the image quality of the entire sequence of images and (ii) minimize the dose level used to acquire the entire sequence of images. The loss function of the exposure control process 505 can be expressed, for example, as Equation (1) below:

[0046]

number

[0047] "n" is the total number of frames in the sequence. In this way, multiple image sequences can be used to train the neural network.

[0048] In one embodiment, the neural network of the exposure control process 505 can be fixed, while the neural network of the image restoration process 520 can be trained. As shown in FIG. 5, the pseudo-degraded frame n1 and the pseudo-degraded frame n2 can be input to the image restoration process 520, which has a training neural network. The output of the image restoration process 520 can be a restored image n2. This can be compared with the target image n2, and further training and modification of the neural network can be performed to improve the accuracy of the output of the image restoration process 520. The loss function of the image restoration process 520 can be defined to maximize the image quality of each image or frame. The loss function in the processing of the image restoration process 520 can be written, for example, as in Equation (2) below.

[0049]

number

[0050] The training dataset can be generated according to a variety of processes. In this regard, Figures 6A and 6B are schematic diagrams illustrating training dataset generation according to one embodiment of the present disclosure.

[0051] As one embodiment, FIG. 6A illustrates the use of a phantom (digital or physical). A phantom is an object with known parameters (e.g., size, shape, density) that is imaged using a fluoroscopy system to evaluate, analyze, and optimize the system's performance. In particular, this phantom can be used in conjunction with a 3D device model to simulate an interventional procedure, adding noise to high-dose and low-dose frames. In this manner, data from the phantom, along with the phantom's known characteristics (e.g., the phantom's shape), can be used as input to a 2D projection simulator to generate projection data (images) without degradation. The known acquisition and simulation parameters, along with the undegraded projection data, can be used as input to an image quality simulator to generate degraded simulated images for the training dataset.

[0052] In one embodiment, Figure 6B illustrates the use of high-dose frames with known noise addition: high-dose images, e.g., acquired in a clinical setting, are acquired with acquisition and simulation parameters as inputs to the image quality simulator to generate simulated degraded images for the training data set.

[0053] In one embodiment, the neural network of the exposure control process 505 can be trained together with the neural network of the image restoration process 520 using a training sequence of images in a training process. During the training process, inputs to the neural network of the image restoration process 520 can be based on the output of the neural network of the exposure control process 505, and the neural network of the image restoration process 520 outputs a sequence of restored images based on the training sequence of images. As described above, the neural network of the exposure control process 505 can be trained using a first loss function having a first term representing a value related to the total dose used in acquiring the training sequence of images and a second term representing a value related to the total error in the sequence of restored images. The neural network of the image restoration process 520 can be trained using a second loss function having a term representing a value related to the total error in the sequence of restored images, as described above. In particular, the neural network of the exposure control process 505 can be trained alternately with the neural network of the image restoration process 520.

[0054] 5, during the training process, the neural network of the exposure control process 505 receives input frames n1 and n2 and outputs exposure parameters 510. The neural network of the exposure control process 505 is an example of a first model. The input frames n1 and n2 are examples of first training images. The exposure parameters 510 are an example of a set of exposure parameters for training.

[0055] Also, in the training process, the neural network of the image restoration process 520 receives pseudo-degraded frame n1 and pseudo-degraded frame n2 as input and outputs restored image n2. The neural network of the image restoration process 520 is an example of a second model. The pseudo-degraded frame n1 and pseudo-degraded frame n2 are examples of second training images collected based on a training exposure parameter set. The restored image n2 is also an example of a training restored image.

[0056] In the training process, the first model is updated based on a combined evaluation of the X-ray dose corresponding to the training exposure parameter set and the image quality of the training reconstructed image. For example, the first model is trained to minimize the loss function shown in Equation (1).

[0057] Specifically, if the X-ray dose corresponding to the training exposure parameter set is large, the value of the second term on the right side of Equation (1) increases. On the other hand, if the X-ray dose corresponding to the training exposure parameter set is small, the image quality of the second training image acquired based on the exposure parameter set decreases. A decrease in the image quality of the input image generally reduces the accuracy of the image restoration process. That is, the image quality of the training restored image also decreases. As a result, for example, the error between the target image n2 and the restored image n2 shown in Figure 5 increases, and the value of the first term on the right side of Equation (1) increases. In this way, by minimizing the loss function shown in Equation (1), the first model can be trained to simultaneously achieve the two objectives of reducing radiation exposure and improving image quality.

[0058] During the training process, alternating training of the first and second models allows the two goals of reducing radiation exposure and improving image quality to be achieved at a higher level. That is, by training the first model, both radiation exposure reduction and image quality improvement are temporarily achieved. Next, by training the second model, the image restoration performance is improved, making it possible to restore images acquired under lower radiation dose conditions to a sufficient level. Then, by further training the first model, the second model with improved performance further reduces radiation exposure while maintaining image quality.

[0059] The first model may be further trained to improve the image quality of informative frames, i.e., the first model may be updated based on a combined evaluation of the informativeness of the first training images, the X-ray dose corresponding to the training exposure parameter set, and the image quality of the training reconstructed images.

[0060] For example, in FIG. 5, the position of the object in each of input frames n1 and n2 is identified, and the amount of movement of the object is calculated, thereby calculating an index indicating the usefulness of input frames n1 and n2. Then, for example, Equation (1) is modified by multiplying the first term on the right side by the calculated index, and the loss function is minimized to train a first model. In this way, when input frames n1 and n2 are useful frames, the first model can be trained so that improving image quality is given higher priority. On the other hand, when input frames n1 and n2 are not useful frames, the first model can be trained so that reducing the X-ray dose is given higher priority.

[0061] 5 illustrates an example in which input frame n1 and input frame n2 are input to the first model. That is, an example in which multiple training images are input to the first model has been described. However, the embodiment is not limited to this, and only one training image may be input to the first model. For example, in FIG. 5, the neural network of exposure control process 505 may only accept input of input frame n2 and output exposure parameters 510.

[0062] 5 has been described as an example in which pseudo-degraded frame n1 and pseudo-degraded frame n2 are input to the second model. That is, an example in which a plurality of training images are input to the second model has been described. However, the embodiment is not limited to this, and only one training image may be input to the second model. For example, in FIG. 5, the neural network of the image restoration process 520 may only accept input of pseudo-degraded frame n2 and output restored image n2.

[0063] 5 has been described as an example in which a pseudo-degraded frame is input to the second model. That is, as shown in, for example, FIGS. 6A and 6B, an example in which a low-quality image is simulated based on a high-quality image, and the simulated pseudo-degraded frame is input to the second model has been described. However, the embodiment is not limited to this.

[0064] That is, the input to the second model may be an image acquired without simulation. For example, instead of the pseudo-degraded frame shown in FIG. 5, a clinical image or a phantom image acquired under low-dose conditions may be input to the neural network of the image restoration process 520. In this case, an image acquired under high-dose conditions from the same subject or phantom may be used as the target image n2.

[0065] 7A and 7B illustrate a non-limiting example flowchart of an acquisition parameter adjustment method 700 according to an embodiment of the present disclosure.

[0066] In one embodiment, in step 705, the fluoroscopy system may acquire a first image including projection data representing the intensities of x-rays emitted from the x-ray source and detected by the plurality of detectors at a first x-ray exposure setting, for example, the first x-ray exposure setting is a low dose or low radiation level, and the first image is a low-dose image.

[0067] In one embodiment, in step 710, the fluoroscopy system may determine whether an object is located in the first image, and if the first image is determined to include an object, may determine a first parameter of the first object in the first image. For example, the first parameter of the first object may include a first position of the first object.

[0068] In one embodiment, in step 715, the fluoroscopy system may acquire a second image including projection data representing the intensities of x-rays emitted from the x-ray source and detected by the plurality of detectors at the first x-ray exposure setting. For example, the second image may be a low-dose image.

[0069] In one embodiment, the fluoroscopy system may determine a second parameter of the first object in the second image in step 720. For example, the second parameter of the first object may include a second position of the first object.

[0070] In one embodiment, the fluoroscopy system may determine a difference between a first parameter and a second parameter of the first object in step 725. For example, the difference between the first parameter and the second parameter may be a difference (distance) between a first position and a second position of the first object.

[0071] In one embodiment, in step 730, the fluoroscopy system can determine whether a difference between the first parameter and the second parameter is greater than a predetermined threshold. That is, the fluoroscopy system can determine whether the first image and the second image are informative frames or images in which a significant event occurred. For example, the predetermined threshold can be determined based on a distance between a first position and a second position of a first object. For example, the predetermined threshold can be determined based on a size difference between a first size of the first object in the first image and a second size of the first object in the second image. For example, the predetermined threshold can be determined based on an intensity difference between a first density of the first object in the first image and a second density of the first object in the second image.

[0072] In one embodiment, if it is determined in step 730 that the difference is not greater than the predetermined threshold, the fluoroscopy system may continue to acquire images at the first exposure setting by proceeding to step 705. That is, since the first and second images are not informative frames, imaging of the first object may continue using low-dose imaging.

[0073] In one embodiment, if step 735 determines that the difference is greater than a predetermined threshold, the fluoroscopy system can determine a second x-ray exposure setting to use for acquiring a third image, i.e., the first and second images are informative frames in which a significant event occurred within the image, such as a significant change in position of the object.

[0074] In one embodiment, the fluoroscopy system may acquire a third image including projection data representing the intensities of x-rays emitted from the x-ray source and detected by the plurality of detectors at a second x-ray exposure setting in step 740. For example, to obtain a third image with low noise or high image quality, the second x-ray exposure setting may be a high dose or high radiation level, and the third image may be a high-dose image.

[0075] 7B, in one embodiment, the fluoroscopy system may determine a third parameter of the first object in the third image in step 745. For example, the third parameter of the first object may include a third position of the first object.

[0076] In one embodiment, the fluoroscopy system may acquire a fourth image including projection data representing the intensities of x-rays emitted from the x-ray source and detected by the plurality of detectors at a second x-ray exposure setting in step 750. For example, the fourth image may be a high-dose image.

[0077] In one embodiment, the fluoroscopy system may determine a fourth parameter of the first object in the fourth image in step 755. For example, the fourth parameter of the first object may include a fourth position of the first object.

[0078] In one embodiment, the fluoroscopy system may determine a difference between a third parameter and a fourth parameter of the first object in step 760. For example, the difference between the third parameter and the fourth parameter is the difference (distance) between a third position and a fourth position of the first object.

[0079] In one embodiment, in step 765, the fluoroscopy system can determine whether the difference between the third parameter and the fourth parameter is greater than a predetermined threshold, i.e., the fluoroscopy system can determine whether the third and fourth images are informative frames or images in which a significant event occurred.

[0080] In one embodiment, if the difference is determined to be greater than the predetermined threshold in step 765, the fluoroscopy system may continue to acquire images at the second exposure setting by proceeding to step 740. That is, the third and fourth images are still informative frames, so imaging of the first object may continue using high-dose imaging.

[0081] In one embodiment, if step 765 determines that the difference is not greater than the predetermined threshold, the fluoroscopy system may proceed to step 770 to again acquire an image, such as the fifth image, at the first exposure setting. That is, the third and fourth images are not useful frames, and the first object may be again imaged using low-dose imaging. It will be appreciated that in step 770, instead of reverting to the first x-ray exposure setting, a third x-ray exposure setting may be determined to be used to acquire the fifth image.

[0082] FIG. 7C illustrates a non-limiting example flowchart of a collection parameter adjustment method 702 according to an embodiment of the present disclosure.

[0083] In one embodiment, a sequence of training images is acquired in step 775. The sequence of training images is collected sequentially using a corresponding set of exposure parameters of the image scanning device.

[0084] In one embodiment, in step 780, a set of images from a training image sequence are input to a first model to obtain an output set of exposure parameters output from the first model.

[0085] In one embodiment, a simulated image is generated based on the output exposure parameter set in step 785. The generated simulated image is then input to a second model to obtain a restored image output from the second model.

[0086] In one embodiment, in step 790, the reconstructed image is compared to a corresponding target image in the sequence of acquired target images to generate an image error term.

[0087] In one embodiment, in step 795, the input, generate, input, and compare steps are repeated for successive sets of collected training image sequences.

[0088] In one embodiment, in step 797, one of the first model and the second model is updated using a second loss function having a first term representing the total dose used in acquiring the training sequence of images and a second term representing the total number of image error terms in the reconstructed sequence of images, and the second model is updated using a second loss function having a term representing the total number of image error terms in the reconstructed sequence of images.

[0089] FIG. 7D illustrates a non-limiting example flowchart of a method including acquisition parameter adjustment and image restoration using a trained model according to one embodiment of the present disclosure.

[0090] In one embodiment, a sequence of images including a first image and a second image is acquired in step 805. The first and second images are collected sequentially using a first set of exposure parameters of the image scanning device.

[0091] In one embodiment, the first and second collected images are input into a trained first model to obtain a second set of exposure parameters output from the trained first model in step 810. That is, in step 810, a plurality of x-ray images collected using the first set of exposure parameters are input into the first model to obtain a second set of exposure parameters.

[0092] The multiple X-ray images input to the first model in step 810 may be multiple adjacent frames of X-ray images. For example, the second image input to the first model may be the image of the frame next to the first image. In this case, the amount of movement of the object between adjacent frames can be calculated to evaluate usefulness, and the evaluation result can be input to the first model. Conversely, a single X-ray image may be input to the first model.

[0093] In one embodiment, step 815 acquires a third image collected by an image scanning device using the acquired second set of exposure parameters.

[0094] In one embodiment, in step 820, at least the second image and the third image are input to a trained second model to obtain a reconstructed third image output from the trained second model. Note that a single X-ray image may be input to the second model. For example, in step 820 of FIG. 7D, a third image may be input to the trained second model to obtain a reconstructed third image output from the trained second model.

[0095] In an embodiment, step 825 outputs the restored third image.

[0096] 8A and 8B show various examples of deep learning (DL) networks.

[0097] Figure 8A shows an example of a general artificial neural network (ANN) with N inputs, K hidden layers, and three outputs. Each layer consists of nodes (also called neurons), which perform a weighted sum of the inputs and compare the result of the weighted sum with a threshold to generate an output. ANNs constitute a class of functions whose members are obtained by varying the threshold, connection weights, or architectural specifications such as the number and connectivity of nodes. Nodes in an ANN are called neurons (or neuron nodes), and neurons can have interconnections between different layers of the ANN system. The simplest ANN has three layers and is called an autoencoder.

[0098]

number

[0099] N is the number of pixels in the reconstructed image (sinogram). Synapses (i.e., connections between neurons) store values ​​called "weights" (interchangeably called "coefficients" or "weighting coefficients") that manipulate the data in the calculation. The output of an ANN depends on three types of parameters: (i) the interconnection pattern between different layers of neurons, (ii) a learning process for updating the interconnection weights, and (iii) an activation function that converts the weighted input of a neuron into an output activation.

[0100] Mathematically, a network function m(x) of neurons is a function of other functions n im(x) is defined as a composition of m(x). Other functions can be defined as compositions of other functions. This can be conveniently represented as a network structure, with arrows representing the dependencies between variables, as shown in the figure. For example, an ANN can use a nonlinear weighted sum. Here, m(x) is expressed as equation (3) below. In equation (3), K (commonly called the activation function) is a predefined function, such as the hyperbolic tangent.

[0101]

number

[0102] In Figure 8A (and similarly in Figure 8B), neurons (i.e., nodes) are depicted as circles around the threshold function. In certain embodiments, the DL network is a feedforward network (e.g., can be represented as a directed acyclic graph) as shown in Figures 8A and 8B.

[0103]

number

[0104] The cost function C is a measure of how far a particular solution is from the optimal solution for the problem being solved (e.g., error). A learning algorithm iteratively searches the solution space to find the function with the minimum cost. In some implementations, the cost is minimized over a sample of data (i.e., the training data).

[0105] Figure 8B shows a non-limiting example in which the DL network is a convolutional neural network (CNN). CNNs are a type of ANN with properties that are beneficial for image processing, making them particularly relevant for image denoising and sinogram restoration applications. CNNs use feedforward ANNs, where the pattern of connections between neurons can represent convolutions in image processing. For example, CNNs can be used to optimize image processing by using multiple layers of small collections of neurons, called receptive fields, that process portions of the input image. The outputs of these collections can be tiled, overlapping each other, to obtain a better representation of the original image. This processing pattern can be repeated across multiple layers with alternating convolutional and pooling layers.

[0106] As generally applied above, after the convolutional layers, the CNN may include local and / or global pooling layers that combine the outputs of the neuron clusters in the convolutional layers. Furthermore, in particular implementations, the CNN may include various combinations of convolutional and fully connected layers, with pointwise nonlinearities applied at or after each layer.

[0107] 9 is a diagram showing the configuration of an X-ray diagnostic apparatus 100 according to one embodiment. As shown in the figure, the X-ray diagnostic apparatus 100 includes a high-voltage generator 11, an X-ray tube 12, a collimator 13, a tabletop 14 (top board), a C-arm 15, an X-ray detector 16, a C-arm rotation / movement mechanism 17, a tabletop movement mechanism 18, a C-arm / tabletop mechanism control circuit 19, a collimator control circuit 20, a processing circuit 21, an input circuit 22, a display 23, an image data generation circuit 24, a storage 25, and an image processing circuit 26.

[0108] In X-ray diagnostic apparatus 100, each processing function is stored in storage 25 in the form of a computer program executable by a computer. C-arm / tabletop mechanism control circuit 19, collimator control circuit 20, processing circuit 21, image data generation circuit 24, and image processing circuit 26 are processors that read the computer programs from storage 25 and execute the computer programs to realize the functions corresponding to the computer programs. In other words, each circuit in a state in which a computer program has been read has the function corresponding to the read computer program.

[0109] The term "processor" used in the above description refers to a circuit such as a central processing unit (CPU), a graphics processing unit (GPU), an application specific integrated circuit (ASIC), or a programmable logic device (e.g., a simple programmable logic device (SPLD), a complex programmable logic device (CPLD), or a field programmable gate array (FPGA)). The processor reads and executes a computer program stored in a memory circuit to implement its functions. The computer program may not be stored in a memory circuit, but may be directly embedded in the circuitry within the processor. In this case, the processor implements its functions by reading and executing the computer program embedded in the circuitry. Each processor in this embodiment may not be configured as a single circuit, but may implement its functions by combining multiple independent circuits into a single processor.

[0110] The high voltage generator 11 generates a high voltage under the control of the processing circuit 21, and supplies the generated high voltage to the X-ray tube 12. The X-ray tube 12 uses the high voltage supplied from the high voltage generator 11 to generate X-rays.

[0111] The collimator 13, under the control of the collimator control circuit 20, focuses the X-rays generated by the X-ray tube 12 so that the target region of the subject P is selectively irradiated with X-rays. For example, the collimator 13 has four slidable collimator blades. Under the control of the collimator control circuit 20, the collimator 13 focuses the X-rays generated by the X-ray tube 12 by sliding these collimator blades, and irradiates the subject P with the focused X-rays. The collimator 13 also has an additional filter for adjusting the quality of irradiation. The additional filter is set, for example, by testing. The tabletop 14 is a bed on which the subject P lies, and is placed on a table (couch) not shown. The subject P is not included in the X-ray diagnostic apparatus 100.

[0112] The X-ray detector 16 detects X-rays that have passed through the subject P. For example, the X-ray detector 16 includes detection elements arranged in a matrix. Each detection element converts the X-rays that have passed through the subject P into an electrical signal, accumulates the electrical signal, and transmits the accumulated electrical signal to the image data generation circuit 24. Note that, of the X-ray diagnostic apparatus 100, the imaging mechanisms for capturing X-ray images, such as the X-ray tube 12, collimator 13, and X-ray detector 16, are also referred to as an image scanning device or an X-ray device.

[0113] The C-arm 15 holds the X-ray tube 12, the collimator 13, and the X-ray detector 16. The C-arm 15 rotates at high speed like a propeller around the subject P lying on the tabletop 14 by a motor provided on a support (not shown). The C-arm 15 is supported rotatably about three mutually orthogonal axes, i.e., the X-, Y-, and Z-axes, and is rotated independently about each axis by a drive unit (not shown). The X-ray tube 12 and the collimator 13 are disposed opposite the X-ray detector 16 by the C-arm 15, with the subject P interposed therebetween. The X-ray diagnostic apparatus 100 is, for example, a single-plane type, but the embodiment is not limited thereto and a bi-plane type may also be adopted. Although FIG. 9 illustrates the X-ray diagnostic apparatus 100 equipped with the C-arm 15, the embodiment is not limited thereto and can be applied to any type of X-ray diagnostic apparatus, such as an X-ray TX apparatus.

[0114] The C-arm rotation / movement mechanism 17 is a mechanism that rotates and moves the C-arm 15. The C-arm rotation / movement mechanism 17 can also change the source image receptor distance (SID), which is the distance between the X-ray tube 12 and the X-ray detector 16. The C-arm rotation / movement mechanism 17 can also rotate the X-ray detector 16 held by the C-arm 15. The tabletop movement mechanism 18 is a mechanism that moves the tabletop 14.

[0115] Under the control of the processing circuitry 21, the C-arm / tabletop mechanism control circuit 19 controls the C-arm rotation / movement mechanism 17 and the tabletop movement mechanism 18 to adjust the rotational movement of the C-arm 15 and the movement of the tabletop 14. For example, under the control of the processing circuitry 21, the C-arm / tabletop mechanism control circuit 19 controls rotational imaging so as to collect projection data at a predetermined frame rate while rotating the C-arm 15. Under the control of the processing circuitry 21, the collimator control circuit 20 adjusts the opening of the collimator blades of the collimator 13 to control the irradiation range of the X-rays irradiated on the subject P.

[0116] The image data generation circuit 24 generates projection data using the electrical signals obtained by the conversion of X-rays by the X-ray detector 16, and stores the generated projection data in the storage 25. For example, the image data generation circuit 24 performs current-voltage conversion, analog-digital (A / D) conversion, and parallel-serial conversion on the electrical signals received from the X-ray detector 16 to generate projection data. The image data generation circuit 24 then stores the generated projection data in the storage 25.

[0117] The storage 25 receives and stores the projection data generated by the image data generation circuitry 24. The storage 25 stores computer programs corresponding to the various functions that are read and executed by the illustrated circuits. As an example, the storage 25 stores a computer program corresponding to an acquisition function 211, a computer program corresponding to a setting function 212, and a computer program corresponding to a control function 213, which are read and executed by the processing circuitry 21.

[0118] The image processing circuit 26, under the control of the processing circuit 21 described later, performs various image processing on the projection data stored in the storage 25 to generate an X-ray image. Alternatively, under the control of the processing circuit 21 described later, the image processing circuit 26 directly acquires projection data from the image data generation circuit 24 and performs various image processing on the acquired projection data to generate an X-ray image. The image processing circuit 26 may store the processed X-ray image in the storage 25. For example, the image processing circuit 26 can perform various processes using image processing filters such as a moving average (smoothing) filter, a Gaussian filter, a median filter, a recursive filter, and a band-pass filter.

[0119] The image processing circuit 26 further forms reconstruction data (volume data) from the projection data collected by rotational imaging. The image processing circuit 26 then stores the reconstructed volume data in the storage 25. The image processing circuit 26 generates a three-dimensional image from the volume data. For example, the image processing circuit 26 generates a volume rendering image or a multi-planar reconstruction (MPR) image from the volume data. The image processing circuit 26 then stores the generated three-dimensional image in the storage 25.

[0120] The input circuitry 22 is realized by, for example, a trackball, a switch button, a mouse, a keyboard, a foot switch for irradiating X-rays, or the like, for setting a predetermined region (for example, a target region such as the cross section). The input circuitry 22 is connected to the processing circuitry 21, and converts input operations received from an operator into electrical signals to be output to the processing circuitry 21. The display 23 displays a graphical user interface (GUI) for receiving instructions from the operator and various images generated by the image processing circuitry 26.

[0121] The processing circuitry 21 controls the overall operation of the X-ray diagnostic apparatus 100. Specifically, the processing circuitry 21 performs various processes by reading from the storage 25 and executing a computer program corresponding to a control function 213 for controlling the entire apparatus. For example, the control function 213 controls the high-voltage generator 11 in accordance with operator instructions transferred from the input circuitry 22, and adjusts the voltage supplied to the X-ray tube 12 to control the amount of X-rays irradiated onto the subject P and controls its on / off. For example, the control function 213 controls the C-arm / tabletop mechanism control circuit 19 in accordance with operator instructions, and adjusts the rotation and movement of the C-arm 15 and the movement of the tabletop 14. For example, the control function 213 controls the collimator control circuit 20 in accordance with operator instructions, and adjusts the opening of the collimator blades of the collimator 13 to control the irradiation range of the X-rays irradiated onto the subject P.

[0122] Furthermore, the control function 213 controls, for example, the image data generation process by the image data generation circuit 24 and the image processing and analysis process by the image processing circuit 26 in accordance with instructions from the operator. The control function 213 also controls a GUI for accepting instructions from the operator and controls images stored in the storage 25 to be displayed on the display 23.

[0123] In one embodiment, the processing circuitry 21 executes the above-described control function 213, as well as the acquisition function 211 and the setting function 212. Note that the processing circuitry 21 is an example of a processing circuit in the claims. For example, the acquisition function 211 is an example of an acquisition unit, and inputs an X-ray image acquired using a first exposure parameter set into a first model to acquire a second exposure parameter set. The setting function 212 is an example of a setting unit, and inputs an X-ray image acquired using a second exposure parameter set into a second model trained together with the first model to acquire a reconstructed image. The control function 213 is an example of a setting unit, and outputs the reconstructed image. The control function 213 may also execute the various training processes described above. Alternatively, the various training processes described above may be executed in an external device different from the X-ray diagnostic apparatus 100, and the control function 213 may be configured to acquire a model trained by the external device.

[0124] Embodiments of the radiation detection device data and functional operations described herein can be implemented in digital electronic circuitry, embodied computer software or firmware, computer hardware including the structures disclosed herein and their structural equivalents, or one or more combinations thereof. Embodiments of the subject matter described herein can also be implemented as one or more computer programs, i.e., one or more modules of computer program instructions encoded on a tangible, non-transitory program carrier for execution by or control the operation of a data processing device, such as a networked device or server, user device, etc. A computer storage medium can be a machine-readable storage device, a machine-readable storage substrate, a random-access memory device, or a serial-access memory device, or one or more combinations thereof.

[0125] The term "data processing apparatus" refers to data processing hardware and may encompass any type of apparatus, device, or machine for processing data, including, by way of example, a programmable processor, a computer, or multiple processors or computers. The apparatus may also be or further include special-purpose logic circuitry, such as an FPGA (field-programmable gate array) or an ASIC (application-specific integrated circuit). In addition to hardware, the apparatus may optionally include code that establishes an execution environment for a computer program, such as code that constitutes processor firmware, a protocol stack, a database management system, an operating system, or one or more combinations thereof.

[0126] A computer program may also be referred to or written as a program, software, software application, module, software module, script, or code, and may be written in any type of programming language, including compiled or interpreted, or declarative or procedural, and may be deployed in any form, such as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment. A computer program may, but need not, correspond to a file in a file system. A program may be stored as part of a file that holds other programs or data, such as one or more scripts stored in a markup language document, in a single file dedicated to the program, or in multiple cooperating files, such as files that store one or more modules, subprograms, or portions of code. A computer program may be deployed to run on a single computer or on multiple computers located at a single site or distributed across multiple sites and interconnected by a communications network.

[0127] According to one embodiment, the processes and logic flows described herein may be performed by one or more programmable computers executing one or more computer programs to perform functions by operating on input data and generating output. The processes and logic flows may also be implemented in, and apparatus may be implemented as, special purpose logic circuitry, such as FPGAs or ASICs.

[0128] Computers suitable for executing computer programs include, by way of example, general-purpose or special-purpose microprocessors, or both, or other types of central processing units. Typically, a CPU receives instructions and data from read-only memory or random-access memory, or both. The elements of a computer are the CPU for running or executing instructions and one or more memory devices for storing instructions and data. Typically, a computer includes one or more mass storage devices for storing data, such as magnetic disks, magneto-optical disks, or optical disks, or is coupled to receive and / or transfer data. However, a computer need not have such devices. Furthermore, a computer can be incorporated into other devices, such as a mobile phone, a personal digital assistant (PDA), a mobile audio or video player, a game console, a Global Positioning System (GPS) receiver, or a portable storage device, such as a Universal Serial Bus (USB) flash drive. Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media, and memory devices, including, by way of example, semiconductor memory devices such as EPROM, EEPROM, and flash memory devices, magnetic disks, such as internal hard disks or removable disks, magneto-optical disks, and CD-ROM and DVD-ROM disks. The processor and memory can be supplemented by, or incorporated in, special purpose logic circuitry.

[0129] To provide for user interaction, embodiments of the subject matter described herein can be implemented on a computer having a display device, such as a cathode ray tube (CRT) or liquid crystal display (LCD) monitor, for displaying information to the user, and a keyboard and pointing device, such as a mouse or trackball, through which the user can provide input to the computer. Other types of devices can be used to provide for user interaction. For example, feedback provided to the user can be any form of sensory feedback, such as visual feedback, auditory feedback, or tactile feedback, and input from the user can be received in any form, such as acoustic, speech, or tactile input. Additionally, a computer can interact with a user by sending documents to or receiving documents from a device used by the user. For example, a computer can send a web page to a web browser on a user's device in response to a request received from the web browser.

[0130] In another embodiment, the subject matter described herein can be implemented in a computing system that includes a back-end component, e.g., a data server, or a middleware component, e.g., an application server, or a front-end component, e.g., a client computer having a graphical user interface or web browser through which a user can interact with an embodiment of the subject matter described herein, or any combination of one or more such back-end, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication, e.g., a communication network. Examples of communication networks include a local area network (LAN) and a wide area network (WAN), such as the Internet.

[0131] An example of one type of computer is shown in FIG. 10. According to one embodiment, the computer 9900 can be used to perform the operations described in connection with any of the computer-implemented methods described above. For example, the computer 9900 can be an example of the X-ray diagnostic apparatus 100. As described above, the computer 9900 includes processing circuitry. The computer 9900 may include other components not explicitly shown in FIG. 10 , such as a CPU, a GPU, a frame buffer, etc. The processing circuitry includes one or more of the elements described below with reference to FIG. 10. In FIG. 10, the computer 9900 includes a processor 9910, a memory 9920, a storage device 9930, and an input / output device 9940. Each of the components 9910, 9920, 9930, and 9940 are interconnected using a system bus 9950. The processor 9910 can process instructions for execution within the system 9900. In one embodiment, the processor 9910 is a single-threaded processor. In another embodiment, the processor 9910 is a multi-threaded processor. The processor 9910 can process instructions stored in the memory 9920 or storage device 9930 to display graphical information for a user interface on the input / output device 9940 .

[0132] The memory 9920 stores information within the computer 9900. In one embodiment, the memory 9920 is a computer-readable medium. In one embodiment, the memory 9920 is a volatile memory unit. In another embodiment, the memory 9920 is a non-volatile memory unit.

[0133] The storage device 9930 is capable of providing mass storage for the computer 9900. In one embodiment, the storage device 9930 is a computer-readable medium. In various different implementations, the storage device 9930 may be a floppy disk drive, a hard disk drive, an optical disk drive, or a tape drive.

[0134] The input / output devices 9940 provide input and output operations to the computer 9900. In one embodiment, the input / output devices 9940 include a keyboard and / or a pointing device. In another embodiment, the input / output devices 9940 include a display device for displaying a graphical user interface.

[0135] Next, referring to FIG. 11 , a hardware description of an apparatus according to this embodiment is shown. In FIG. 11 , device 1201 includes a processing circuit as described above. Referring to FIG. 11 , the processing circuit includes one or more elements described below. The device may include other components not explicitly shown in FIG. 11 , such as a CPU, a GPU, and a frame buffer. In FIG. 11 , the device includes a CPU 1200 that executes the processes described above / below. Process data and instructions can be stored in memory 1202. These processes and instructions may be stored on a storage medium disk 1204, such as a hard drive (HDD) or a portable storage medium, or may be stored remotely. Furthermore, the claimed advancement is not limited by the form of computer-readable medium on which the instructions for the processes of the embodiment are stored. For example, the instructions may be stored on a CD, DVD, flash memory, RAM, ROM, PROM, EPROM, EEPROM, hard disk, or other information processing device, such as a server or computer, with which device 1201 communicates.

[0136] Furthermore, the claimed advancements may be provided as a utility application, background daemon, or operating system component, or combinations thereof, running in conjunction with CPU 1200 and an operating system such as Microsoft Windows®, UNIX®, Solaris, LINUX®, Apple MAC-OS, and other systems known to those skilled in the art.

[0137] The hardware elements for implementing device 1201 may be implemented using various circuit elements known to those skilled in the art. For example, CPU 1200 may be an Intel Xenon or Core processor, an AMD Opteron processor, or other processor types recognized by those skilled in the art. Alternatively, CPU 1200 may be implemented using FPGA, ASIC, PLD, or discrete logic circuitry, as recognized by those skilled in the art. Furthermore, CPU 1200 may be implemented as multiple processors cooperating in parallel to execute instructions of the processes described above.

[0138] The device of Figure 11 also includes a network controller 1206, such as an Intel Ethernet PRO network interface card from Intel Corporation of America, for interfacing with network 120 and communicating with other devices. Clearly, network 120 can be a public network, such as the Internet, a private network, such as a LAN or WAN network, or any combination thereof, and can also include a PSTN or ISDN subnetwork. Network 120 can be wired, such as an Ethernet network, or wireless, such as a cellular network, including EDGE, 3G, 4G, and 5G wireless cellular systems. The wireless network can be Wi-Fi, Bluetooth, or other known forms of wireless communication.

[0139] The device further includes a display controller 1208, such as an NVIDIA GeForce GTX or Quadro graphics adapter manufactured by NVIDIA Corporation of the United States, for coupling to a display 1210, such as an LCD monitor. A general-purpose I / O interface 1212 couples to a keyboard and / or mouse 1214 and a touchscreen panel 1216 on or separate from the display 1210. The general-purpose I / O interface also couples to various peripherals 1218, including printers and scanners.

[0140] A sound controller 1220 is also provided in the device 1201 and cooperates with a speaker / microphone 1222 to provide sound and / or music.

[0141] A generic storage controller 1224 connects the storage media disks 1204 to a communication bus 1226, such as ISA, EISA, VESA, PCI, etc., which interconnects all of the components of the device. A description of the general features and functionality of the display 1210, keyboard and / or mouse 1214, as well as the display controller 1208, storage controller 1224, network controller 1206, sound controller 1220, and generic I / O interface 1212 will not be provided herein for the sake of brevity, as these features are well known.

[0142] While many specific implementation details are set forth in this specification, these should not be construed as limitations on the scope of what may be claimed, but rather as descriptions of features that may be unique to particular embodiments.

[0143] Certain features that are described in this specification in the context of separate embodiments can also be implemented in combination in a single embodiment. Conversely, various features that are described in the context of a single embodiment can also be implemented in multiple embodiments separately or in any suitable subcombination. Further, while features may be described as working in a particular combination and may initially be claimed as such, one or more features from a claimed combination may, in some cases, be excluded from the combination, and the claimed combination may be directed to any subcombination or variation of the subcombination.

[0144] In the preceding description, specific details have been set forth, such as the particular configuration of the processing system and descriptions of the various components and processes used therein. However, it should be understood that the technology herein may be practiced in other embodiments that depart from these specific details, and that such details are for purposes of explanation and not limitation. The embodiments disclosed herein have been described with reference to the accompanying drawings. Similarly, for purposes of explanation, specific numbers, materials, and configurations are set forth to provide a thorough understanding. However, embodiments may be practiced without such specific details. Components having substantially the same functional structure are designated by similar reference numerals, and redundant description may be omitted.

[0145] With respect to the above embodiment, the following supplementary notes are disclosed as one aspect and optional features of the invention. (Appendix 1) a setting unit that inputs the X-ray images acquired using the first set of exposure parameters into the first model to obtain a second set of exposure parameters; an acquisition unit that inputs the X-ray images acquired using the second set of exposure parameters into a second model trained together with the first model to acquire a restored image; a control unit that outputs the restored image; An X-ray diagnostic device comprising: (Appendix 2) The setting unit may input a plurality of X-ray images acquired using the first exposure parameter set into the first model to obtain the second exposure parameter set. (Appendix 3) The setting unit may input adjacent frames of X-ray images acquired using the first exposure parameter set into the first model to obtain the second exposure parameter set. (Appendix 4) In the training process, the first model may be trained to receive input of a first training image and output a training exposure parameter set, and the second model may be trained to receive input of a second training image collected based on the training exposure parameter set and output a training reconstructed image, and the first model may be updated based on a comprehensive evaluation of the X-ray dose corresponding to the training exposure parameter set and the image quality of the training reconstructed image. (Appendix 5) the setting unit acquires a sequence of a plurality of X-ray images including a first X-ray image and a second X-ray image acquired sequentially using the first exposure parameter set of an image scanning device, inputs the acquired first X-ray image and the acquired second X-ray image into the first trained model, and acquires the second exposure parameter set output from the first model; the acquisition unit acquires a third X-ray image acquired by the image scanning device using the second exposure parameter set, inputs at least the second X-ray image and the third X-ray image into the trained second model, and acquires a reconstructed third X-ray image output from the second model as the reconstructed image; the control unit outputs the restored third X-ray image as the restored image; the first model and the second model are jointly trained using a training sequence of X-ray images in a training process; During the training process, the input to the second model is the output of the first model, and the output of the second model may be a sequence of reconstructed X-ray images. (Appendix 6) The training process may involve training the first model using a first loss function having a first term representing a value related to the total dose used in acquiring the training sequence of X-ray images and a second term representing a value related to the total error in the sequence of reconstructed X-ray images, and training the second model using a second loss function having a term representing the total error in the sequence of reconstructed X-ray images. (Appendix 7) The first exposure parameter set of exposure parameters may include a first dose level and the second exposure parameter set of exposure parameters may include a second dose level, the first dose level being lower than the second dose level. (Appendix 8) The first model and the second model may be neural network models. (Appendix 9) The first model and the second model may be trained alternately during the training process. (Appendix 10) 1. An X-ray diagnostic device including a controller configured to perform a training process, 10. An X-ray diagnostic apparatus, comprising: an X-ray imaging system; an image scanning device; an image scanning apparatus; an image scanning method; an image scanning method for performing the training process; (Appendix 11) The controller may further perform the training process by updating the first model with a first loss function having a first term representing a value related to a total dose used in acquiring the training sequence of X-ray images and a second term representing a total number of image error terms in the sequence of reconstructed X-ray images, and updating the second model with a second loss function having a term representing a total number of image error terms in the sequence of reconstructed X-ray images. (Appendix 12) In the updating, the control unit may be further configured to update the first model using the first loss function while keeping the second model fixed, and to update the second model using the second loss function while keeping the first model fixed. (Appendix 13) In the updating, the controller may further repeat the training process for another sequence of training x-ray images that are sequentially acquired using another corresponding set of exposure parameters and another sequence of target x-ray images. (Appendix 14) The step of inputting the generated simulation image into the second model may further include inputting another simulation image into the second model along with the simulation image, where the other simulation image is generated based on a different set of exposure parameters. (Appendix 15) The first model and the second model may be neural network models. (Appendix 16) The method may include acquiring the training X-ray image sequence using clinical X-ray images or X-ray images acquired from imaging a phantom. (Appendix 17) In the generating, the control unit may be further configured to generate uncorrupted projections based on a digital phantom, and generate the simulation image from the generated projections based on the output exposure parameter set. (Appendix 18) inputting the x-ray images acquired using the first set of exposure parameters into the first model to obtain a second set of exposure parameters; inputting the x-ray images collected using the second set of exposure parameters into a second model trained together with the first model to obtain a restored image; Output the restored image A method comprising: (Appendix 19) obtaining a sequence of training X-ray images acquired sequentially using corresponding sets of exposure parameters of an image scanning device; inputting a set of X-ray images of the training X-ray image sequence into a first model to obtain a set of exposure parameters output from the first model; generating a simulation image based on the output exposure parameter set; inputting the simulated image into a second model to obtain a reconstructed X-ray image output from the second model, and comparing the reconstructed X-ray image with a corresponding target X-ray image in a sequence of acquired target X-ray images to generate an image error term; performing a training process by repeating the input, generation, input, and comparison steps described above for successive sets of the acquired training X-ray image sequences and updating one of the first model and the second model; A method comprising:

[0146] According to at least one of the embodiments described above, it is possible to improve the efficiency of a series of processes including acquisition of X-ray images and image restoration processing.

[0147] Although several embodiments of the present invention have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These embodiments can be implemented in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their modifications are included within the scope and spirit of the invention, as well as within the scope of the invention and its equivalents as defined in the claims. [Explanation of symbols]

[0148] 100: X-ray diagnostic equipment 21: Processing circuit 211: Acquisition function 212: Setting function 213: Control function

Claims

1. a setting unit that inputs the X-ray images acquired using the first exposure parameter set into the first model to obtain a second exposure parameter set; an acquisition unit that inputs the X-ray images acquired using the second exposure parameter set into a second model trained together with the first model to acquire a restored image; a control unit that outputs the restored image; An X-ray diagnostic apparatus comprising:

2. The X-ray diagnostic apparatus according to claim 1 , wherein the setting unit inputs a plurality of X-ray images acquired using the first exposure parameter set into the first model to acquire the second exposure parameter set.

3. 3. The X-ray diagnostic apparatus according to claim 2, wherein the setting unit inputs adjacent frames of X-ray images acquired using the first exposure parameter set into the first model to acquire the second exposure parameter set.

4. 2. The X-ray diagnostic apparatus according to claim 1, wherein in a training process, the first model is trained to receive input of first training images and output a training exposure parameter set, the second model is trained to receive input of second training images acquired based on the training exposure parameter set and output a training reconstructed image, and the first model is updated based on a comprehensive evaluation of the X-ray dose corresponding to the training exposure parameter set and the image quality of the training reconstructed image.

5. the setting unit acquires a sequence of a plurality of X-ray images including a first X-ray image and a second X-ray image acquired sequentially using the first exposure parameter set of an image scanning device, inputs the acquired first X-ray image and the acquired second X-ray image into the trained first model, and acquires the second exposure parameter set output from the first model; the acquisition unit acquires a third X-ray image acquired by the image scanning device using the second exposure parameter set, inputs at least the second X-ray image and the third X-ray image into the trained second model, and acquires a reconstructed third X-ray image output from the second model as the reconstructed image; the control unit outputs the restored third X-ray image as the restored image; the first model and the second model are jointly trained using a training sequence of X-ray images in a training process; 2. The X-ray diagnostic apparatus according to claim 1, wherein in the training process, the input to the second model is the output of the first model, and the output of the second model is a sequence of reconstructed X-ray images.

6. 6. The X-ray diagnostic apparatus of claim 5, wherein in the training process, the first model is trained using a first loss function having a first term representing a value related to a total dose used in acquiring the training sequence of X-ray images and a second term representing a value related to a total error in a sequence of reconstructed X-ray images, and the second model is trained using a second loss function having a term representing a total error in the sequence of reconstructed X-ray images.

7. 6. The X-ray diagnostic device of claim 5, wherein the first exposure parameter set of exposure parameters includes a first dose level and the second exposure parameter set of exposure parameters includes a second dose level, the first dose level being lower than the second dose level.

8. The X-ray diagnostic apparatus according to claim 5 , wherein the first model and the second model are neural network models.

9. The X-ray diagnostic apparatus of claim 5 , wherein the first model and the second model are trained alternately during the training process.

10. 1. An X-ray diagnostic device including a controller configured to perform a training process, 1. An X-ray diagnostic device comprising: an X-ray imaging system; an X-ray imaging system including: an X-ray imaging system; an X-ray imaging system including a first model and a second model; an X-ray imaging system including a second ...

11. 11. The X-ray diagnostic apparatus of claim 10, wherein the controller further performs the training process by updating the first model using a first loss function having a first term representing a value related to a total dose used in acquiring a training sequence of X-ray images and a second term representing a total number of image error terms in the sequence of reconstructed X-ray images, and updating the second model using a second loss function having a term representing a total number of image error terms in the sequence of reconstructed X-ray images.

12. 12. The X-ray diagnostic apparatus according to claim 11, wherein, in the updating, the control unit is further configured to update the first model using the first loss function while keeping the second model fixed, and to update the second model using the second loss function while keeping the first model fixed.

13. 11. The X-ray diagnostic apparatus according to claim 10, wherein, in the updating, the control unit further repeats the training process for another sequence of training X-ray images that are sequentially acquired using another corresponding set of exposure parameters and another sequence of target X-ray images.

14. 11. The X-ray diagnostic apparatus of claim 10, wherein the step of inputting the generated simulation image into the second model further includes inputting another simulation image into the second model together with the simulation image, the other simulation image being generated based on a different set of exposure parameters.

15. 11. The X-ray diagnostic apparatus of claim 10, further comprising acquiring the training X-ray image sequence using clinical X-ray images or X-ray images acquired from imaging a phantom.

16. 11. The X-ray diagnostic apparatus according to claim 10, wherein in the generating, the control unit is further configured to generate uncorrupted projections based on a digital phantom, and to generate the simulation image from the generated projections based on the output exposure parameter set.

17. inputting the x-ray images acquired using the first set of exposure parameters into the first model to obtain a second set of exposure parameters; inputting the x-ray images acquired using the second set of exposure parameters into a second model trained together with the first model to obtain a reconstructed image; Output the restored image A method comprising:

18. obtaining a sequence of training X-ray images acquired sequentially using corresponding sets of exposure parameters of an image scanning device; inputting a set of x-ray images of the training x-ray image sequence into a first model to obtain a set of exposure parameters output from the first model; generating a simulation image based on the output exposure parameter set; inputting the simulated image into a second model to obtain a reconstructed X-ray image output from the second model, and comparing the reconstructed X-ray image with a corresponding target X-ray image in a sequence of acquired target X-ray images to generate an image error term; performing a training process by repeating the input, generating, input, and comparing steps described above for successive sets of the acquired training X-ray image sequences and updating one of the first model and the second model; A method comprising:

Images