Prospective quality assessment of pre-acquisition imaging examinations

Abstract

The present invention relates to medical imaging. In order to reduce imaging repetitions, it is proposed to enable automatic prediction of quality metrics before imaging by utilizing data from sensors. This can significantly improve the quality of medical image data acquired in real imaging examinations, leading to fewer retakes, shorter treatment delays for patients, shorter workflows, and increased patient rates. In X-ray and CT examinations, fewer retakes can reduce the radiation dose for patients.

Description

[Technical field]

[0001] The present invention relates to medical imaging, and in particular to a computer-implemented method, apparatus, system and computer program for prospective quality assessment of pre-acquisition imaging exams. [Background technology]

[0002] US Patent Application Publication No. 2019 / 183439 discloses a method for positioning a body region of a patient for radiographic acquisition by an X-ray imaging system.

[0003] Proper patient positioning is important for diagnostic quality. Relevant quality standards and recommendations (or recommendations) are formulated in official standards. In clinical routine, there are many factors that cause deviations from these standards, such as high workload, lack of training, lack of education, lack of feedback, and lack of reward. Images violating one or more quality standards can often result in extra workload for medical staff. They may lead to misdiagnosis and therefore pose a health risk to the patient. Summary of the Invention [Problem to be solved by the invention]

[0004] To reduce misdiagnosis, computer-aided methods have been developed to analyze medical image data and evaluate its quality after the image is taken. If the quality of the acquired medical image does not meet the quality criteria and recommendations in official standards, the examination may need to be repeated. However, the difficulties that arise from repeating imaging include delayed treatment for the patient, reduced patient rates, and increased radiation dose to the patient in the case of X-ray and CT examinations.

[0005] It may be necessary to reduce imaging repetitions. [Means for solving the problem]

[0006] The invention is defined by the independent claims. The dependent claims define advantageous embodiments.

[0007] It should be noted that the aspects described below also apply to computer-implemented methods, apparatus, systems, and computer programs.

[0008] According to a first aspect of the present invention there is provided a computer-implemented method for prospective quality assessment of a pre-acquisition imaging examination comprising:

[0009] a) receiving sensor data of a body part of a patient imaged by a medical imaging device using a medical imaging modality;

[0010] b) generating a quality metric from the received sensor data using a data-driven model, the data-driven model being trained based on a training dataset having a plurality of training examples, each training example having sensor data of a body part acquired during an imaging session and an associated quality metric derived from image data acquired using a medical imaging modality during the imaging session;

[0011] and c) providing the generated quality metric for a prospective quality assessment of a pre-acquisition imaging study.

[0012] In other words, a computer-implemented method is proposed that allows automatic prediction of quality metrics before imaging by leveraging data from (multiple) sensors. The proposed method includes acquiring sensor data of the patient's body part to be imaged during positioning / preparation for the imaging examination. The sensor data can be acquired by any suitable sensor, such as optical sensors, thermal sensors, depth sensors, an array of radio frequency sensors, fiber optic distance sensors, or any combination thereof. If the patient is lying on the patient support, the sensor data can also be acquired by sensors embedded in the patient support, such as pressure sensors embedded in the patient support, ultrasound sensors embedded in the patient support, etc.

[0013] The quality metric can be derived from the received sensor data using a trained data-driven model that is trained to predict the quality metric of the acquired medical image data directly from the sensor data input prior to the actual imaging exam. In some examples, the quality metric can be derived directly from the received sensor data. In some examples, an anatomical structure model of the target anatomical structure can be fitted to the received sensor data, and the quality metric can be derived from the fitted anatomical structure model.

[0014] The quality metric may also be referred to as a predicted quality metric. The data-driven model is trained on a training dataset having multiple training examples. Each training example has sensor data of a body part acquired during an imaging session and an associated quality metric derived from image data acquired using a medical imaging modality during the same imaging session. The quality metric can be manually annotated by a skilled clinician and / or derived from the image data in an automated manner. In general, the data-driven model is trained to learn the association between the sensor data acquired during positioning / preparation for an imaging exam and the quality metric derived from image data acquired by a medical scanner after the actual imaging exam. An exemplary training process is described with respect to the embodiment shown in FIG. 4.

[0015] In some examples, the quality metric may be a single value. In some examples, the quality metric may be a vector of numbers, each number representing a deviation of a position and / or rotation of an anatomical feature in image data acquired using a medical imaging modality from a desired position and / or rotation of the anatomical feature.

[0016] The data-driven model may also be referred to as a quality predictive model. Exemplary data-driven models may include, but are not limited to, artificial neural networks, trained random forests, and model-based segmentation approaches.

[0017] The generated quality metric is then provided for prospective quality assessment of the imaging exam prior to acquisition. For example, the trained data-driven model can be applied by prospectively displaying a predicted quality measure during preparation / positioning for the exam. For example, the generated quality metric can be compared to a reference quality metric and an acoustic signal can be generated to indicate whether the patient is correctly positioned. In another example, the generated quality metric can be compared to a reference quality metric and the medical scanner can be triggered to perform an image acquisition if the difference between the generated quality metric and the reference quality metric is less than a threshold value.

[0018] The proposed method can be provided for prospective quality assessment of various imaging examinations prior to acquisition. Exemplary imaging modalities may include, but are not limited to, X-ray imaging, MR imaging, CT imaging, Positron Emission Tomography (PET) imaging, and auto-steering ultrasound imaging. Other examples of imaging modalities may include combined therapeutic / diagnostic devices such as MR-Linac devices, MR proton therapy devices, and / or cone-beam CT devices.

[0019] A computer-implemented method is described below with particular reference to the embodiment shown in FIG.

[0020] The proposed method can allow the operator to optimize the predicted quality metric before performing the actual examination. For example, an instantaneous prediction of the current positioning quality can be displayed while moving the patient under observation from the sensor. This can significantly improve the quality of the medical image data acquired in the actual imaging examination, leading to fewer retakes, less delays in treatment for the patient, shorter workflow times, and increased patient rates. In X-ray and CT examinations, fewer retakes can reduce the radiation dose for the patient.

[0021] According to one embodiment of the present invention, generating a quality metric from the received sensor data includes: generating a quality metric directly from the received sensor data; or fitting an anatomical structure model of the target anatomical structure to the sensor data and generating a quality metric from the fitted anatomical structure model of the target anatomical structure.

[0022] The anatomical model of the target anatomical structure can be a two-dimensional model, a three-dimensional model, or a higher dimensional model that models joint-dependent articulation movements (eg, flexion) or anatomical variations.

[0023] According to one embodiment of the present invention, a computer-implemented method includes determining whether the generated quality metric meets a predetermined criterion, and generating a signal indicative of whether the generated quality metric meets the predetermined criterion.

[0024] In one example, it may be determined whether the deviation between the generated quality metric and a reference quality metric is less than a threshold.

[0025] In one example, it can be determined whether the generated quality metric is within an acceptable range.

[0026] According to one embodiment of the present invention, the signal includes a signal for controlling a device to inform an operator whether the patient is ready for image acquisition.

[0027] In one example, the device is a speaker configured to generate an acoustic signal that indicates whether the patient is correctly positioned.

[0028] In another example, the device is a lighting device configured to generate a light signal that indicates whether the patient is correctly positioned.

[0029] In another example, the device is a tactile device configured to apply a force, vibration, or motion to the patient and / or operator that indicates whether the patient is correctly positioned.

[0030] According to one embodiment of the present invention, the signal includes a signal for triggering a medical imaging device to begin image acquisition.

[0031] In other words, the medical imaging device may automatically initiate image acquisition in response to a signal indicating that the patient is correctly positioned.

[0032] According to one embodiment of the invention, a computer-implemented method comprises receiving image data of a patient's body part after image acquisition, determining a further quality metric based on the received imaging data, determining a difference between a quality metric generated from sensor data before image acquisition and the further quality metric derived from the image data after image acquisition, and using the difference to further train a data-driven model.

[0033] In other words, it is proposed to be able to rate the predictions by comparing them with a reference quality metric derived from the final image after the actual image capture. This may enable a closed loop of continuous supervised learning, thereby allowing the strategy to continuously improve the data-driven model. Furthermore, a closed training loop may enable the generation of models trained on very large amounts of data without compromising data privacy. This is explained below, in particular with respect to the embodiment shown in Figures 7 and 8.

[0034] According to one embodiment of the invention, a computer-implemented method includes receiving a user input indicating a user-defined quality metric for received image data, determining a difference between a quality metric generated from sensor data prior to image capture and the user-defined quality metric, and using the difference to further train the data-driven model.

[0035] In other words, a user (e.g., a radiologist) can review / check the quality metric vector for an exam and provide feedback, e.g., confirm / modify certain metrics. By feeding this expert feedback into a continuous learning loop, a more reliable strongly supervised learning strategy is realized. This is described below, particularly with respect to the embodiment shown in Figures 7 and 8.

[0036] According to one embodiment of the present invention, the quality metric is a vector of numbers, each number representing the deviation of a position and / or rotation of an anatomical feature in image data acquired using a medical imaging modality from a desired position and / or rotation of the anatomical feature.

[0037] Different numerical values ​​can reflect different quality aspects. For example, as shown in Fig. 8, X-ray image quality can be determined in terms of a quality vector having different quality aspects such as field of view (FOV) compliance, rotation, and curvature. This is explained below with particular reference to the embodiment shown in Fig. 8.

[0038] According to one embodiment of the invention, the sensor data is acquired by one or more of an array of optical sensors, depth sensors, thermal sensors, pressure sensors, ultrasonic sensors, and radio frequency sensors.

[0039] According to an embodiment of the invention, the medical imaging modality includes one or more of magnetic resonance imaging, ultrasound imaging, x-ray imaging, computed tomography imaging, and positron emission tomography imaging.

[0040] According to an embodiment of the present invention, the medical imaging modality includes a hybrid modality including one or more of MR-Linac, MR Proton Therapy, and Cone Beam Computed Tomography.

[0041] According to a second aspect of the present invention, there is provided an apparatus for prospective quality assessment of a pre-acquisition imaging examination, the apparatus comprising one or more processing units for generating a quality metric, the processing units comprising instructions for performing the method steps according to the first aspect and any associated examples when executed on the one or more processing units.

[0042] This is explained below with particular reference to the embodiment shown in FIG.

[0043] According to a third aspect of the present invention there is provided a system comprising a medical imaging device configured to acquire image data of a body part of a patient; a sensor configured to acquire sensor data of the body part of the patient; and an apparatus according to the second aspect and any related examples configured to receive said sensor data and provide quality metrics for prospective quality assessment of a pre-acquisition imaging examination.

[0044] This is explained below, particularly with reference to the embodiment shown in FIGS.

[0045] According to an embodiment of the invention, the medical imaging apparatus is configured to start the image acquisition according to the provided quality metric. Alternatively or additionally, the system further comprises a device configured to signal whether the patient is ready for image acquisition based on the quality metric.

[0046] In one example, the device is a speaker configured to generate an acoustic signal that indicates whether the patient is correctly positioned.

[0047] In another example, the device is a lighting device configured to generate a light signal that indicates whether the patient is correctly positioned.

[0048] In another example, the device is a haptic device configured to provide force, vibration, or motion to the patient and / or operator to indicate whether the patient is correctly positioned.

[0049] According to another aspect of the invention there is provided a program product comprising instructions which, when executed by at least one processing unit, cause the at least one processing unit to perform the steps of the method according to the first aspect and any associated examples.

[0050] It should be understood that all combinations of the foregoing concepts, and additional concepts described in more detail below, provided such concepts are not mutually inconsistent, are contemplated as part of the inventive subject matter disclosed herein. In particular, all combinations of subject matter recited in the claims at the end of this disclosure are considered to be part of the inventive subject matter disclosed herein.

[0051] These and other aspects of the invention will be apparent from and elucidated with reference to the embodiments described hereinafter.

[0052] In the drawings, like reference numbers generally refer to the same parts throughout the different views, and the drawings are not necessarily to scale, emphasis instead being placed upon illustrating the principles of the invention. [Brief description of the drawings]

[0053] [Figure 1] 1 is a block diagram of an exemplary apparatus. [Diagram 2] 1 is a flow chart of an exemplary computer-implemented method. [Diagram 3] FIG. 1 is a diagram illustrating an example of a system. [Figure 4] 1 is a flowchart of a method for generating training data for training a data-driven model. [Diagram 5] FIG. 13 is a diagram showing another example of the system. [Figure 6] FIG. 2 illustrates an exemplary lateral image of the ankle. [Figure 7] FIG. 1 illustrates an exemplary method for achieving weakly supervised continuous learning. [Figure 8] FIG. 1 illustrates an example of a continuous learning framework for quality metric prediction using example data. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0054] 1 shows a block diagram of an exemplary apparatus 10 for prospective quality assessment of a pre-acquisition imaging examination. The apparatus 10 may comprise an input unit 12, one or more processing units 14, and an output unit 16.

[0055] In general, the apparatus 10 may have various physical and / or logical components for communicating and manipulating information, which may be implemented as hardware components (e.g., computing devices, processors, logic devices), executable computer program instructions (e.g., firmware, software) executed by the various hardware components, or any combination thereof, as desired for a given set of design parameters or performance constraints. Although FIG. 1 may show a limited number of components as examples, it may be understood that for a given implementation, a greater or lesser number of components may be used.

[0056] For example, the apparatus 10 may be embodied as or in a device or apparatus such as a server, a workstation, an imaging device, or a mobile device. The apparatus 10 may have one or more microprocessors or computer processors that execute appropriate software. The processing unit 14 of the apparatus 10 may be embodied by one or more of these processors. The software may be downloaded and / or stored in a corresponding memory, e.g., a volatile memory such as RAM, or a non-volatile memory such as Flash. The software may have instructions that configure the one or more processors to perform the functions described herein.

[0057] It should be noted that the apparatus 10 may be implemented with or without a processor, or as a combination of dedicated hardware for performing some functions and a processor (e.g., one or more programmable microprocessors and associated circuits) for performing other functions. For example, the functional units of the apparatus 10, such as the input unit 12, the processing unit 14, and the output unit 16, may be implemented in a device or apparatus in the form of programmable logic, for example as a field programmable gate array (FPGA). The input section 12 and the output section 16 may be implemented by the respective interfaces of the apparatus. In general, each functional unit of the apparatus may be implemented in the form of a circuit.

[0058] The device 10 may also be realized in a distributed manner: for example, some or all units of the device 10 may be configured as separate modules in a distributed architecture and connected within a suitable communication network, such as a 3rd Generation Partnership Project (3GPP) network, a Long Term Evolution (LTE) network, the Internet, a LAN (Local Area Network), a wireless LAN (Local Area Network), a WAN (Wide Area Network), etc.

[0059] The input unit 12 may include hardware and / or software for enabling the device 10 to receive sensor data from a sensor via a wired connection or via a wireless connection.

[0060] The processing unit 14 is capable of executing instructions for performing the methods described herein, which are explained in detail with respect to the embodiment shown in FIG.

[0061] The output unit 16 may comprise hardware and / or software that enables the apparatus to communicate with other devices (e.g., displays, storage devices, etc.) and / or networks (LTE, LAN, wireless LAN, etc.) to provide the generated quality metrics.

[0062] 2 is a flow chart of a computer-implemented method 200 for prospective quality assessment of an imaging exam prior to acquisition. Method 200 is described below with reference to FIG 3. However, method 200 is not limited to the example of FIG 3.

[0063] The computer-implemented method 200 can be implemented as a device, module, or associated component in configurable logic, such as, for example, a programmable logic array (PLA), a field programmable gate array (FPGA), a complex programmable logic device (CPLD), fixed function hardware logic, such as, for example, an application specific integrated circuit (ASIC), using circuit technologies such as complementary metal oxide semiconductor (CMOS) or transistor transistor logic (TTL) technology, a set of logical instructions stored in a non-transitory machine-readable or computer-readable storage medium, such as a random access memory (RAM), a read only memory (ROM), a programmable ROM (PROM), firmware, a flash memory, or any combination thereof. For example, computer program code for performing the operations shown in the method 200 can be written in any combination of one or more programming languages, including object-oriented programming languages, such as JAVA, SMALLTALK, C++, Python, and conventional procedural programming languages, such as the "C" programming language or a similar programming language. For example, the computer-implemented method 200 can be implemented as the apparatus 10 shown in FIG. 1.

[0064] 3 shows a schematic diagram of a system 100 according to an embodiment of the present disclosure. The system 100 comprises an apparatus 10, a medical imaging device 20, a patient support 22 movable along a central system axis of the medical imaging device, a sensor 30, and a console 40. The medical imaging device 20 acquires image data of a body part of a patient. The sensor 30 is configured to acquire sensor data of the body part of the patient. The apparatus 10 is configured to receive the sensor data and provide quality metrics for a prospective quality assessment of the imaging exam prior to acquisition.

[0065] The overall operation of the medical imaging device 20 can be controlled by an operator from a console 40. The console 40 can be coupled to a screen or monitor 42 on which acquired images or imaging device settings can be viewed or reviewed. An operator, such as a medical lab technician, can control image acquisition via the console 40. In the example of FIG. 3, the device 10 is a separate device configured to communicate with the sensor 30 via a wireless and / or wire-based interface. However, in an alternative example (not shown), the device 10 can reside on the console 40 operating as a software routine.

[0066] Returning to FIG. 2, starting from block 210, i.e. step a), an apparatus such as the apparatus 10 shown in FIG. 1 receives sensor data of a patient's body part from a sensor monitoring the body part during positioning / preparation for an imaging examination. The patient's body part is to be imaged by a medical imaging device using a medical imaging modality. The apparatus may receive the sensor data from the sensor via a wired connection or via a wireless connection. The apparatus may have a sensor data interface for accessing the sensor data, which may take various forms such as a network interface to a local area network, a video interface like HDMI, etc.

[0067] In the example of Fig. 3, the sensor 30 is a depth camera. The sensor 30 may also be, for example, a thermal sensor, an ultrasonic sensor, for example embedded in the patient support 22, an array of radio frequency sensors, a pressure sensor embedded in the patient support 22, a fiber optic distance sensor, or any combination thereof. Although Fig. 3 shows only a single sensor, it will be understood that a sensor arrangement having two or more sensors may be provided. In some examples, the sensor 30 may be located at a distance from the medical imaging device 20, for example on the ceiling of a room, so that the sensor has a field of view that includes at least a portion of the area where the patient is positioned for imaging. In some examples, a sensor, such as a pressure sensor, may be embedded in the patient support 22.

[0068] In the example of Fig. 3, the patient's body part is imaged by an MR imaging device. Other examples of the medical imaging device 20 include, but are not limited to, an X-ray imaging device, a CT imaging device, a PET imaging device, an auto-steering ultrasound imaging device, etc. The medical imaging device 20 may also be a combined treatment / diagnosis device, such as an MR-Linac device, an MR proton therapy device, a cone-beam CT device, etc.

[0069] Returning to FIG. 2, in block 220, i.e., step b), the device generates quality metrics from the received sensor data using a data-driven model. The data-driven model is trained to predict quality metrics of image data to be acquired by the medical imaging device from (non-ionized) sensor data acquired by a sensor (e.g., a depth camera) before the actual imaging examination. For example, the data-driven model can be an artificial neural network, a trained random forest, a model-based segmentation approach, etc.

[0070] The device may generate the quality metric directly from the received sensor data. Alternatively, the device may fit an anatomical model of the target anatomical structure to the received sensor data and generate the quality metric from the fitted anatomical model of the target anatomical structure. The anatomical model of the target anatomical structure may be, for example, a two-dimensional model, a three-dimensional model, or a four-dimensional model.

[0071] The quality metric is a vector of numerical values. Each numerical value represents a deviation of a position and / or rotation of an anatomical feature in image data acquired using a medical imaging modality from a desired position and / or rotation of the anatomical feature. In some examples, the numerical values ​​can have ordinal values ​​such as 0 representing a very small deviation, 0.5 representing a small deviation, 1 representing a large deviation, etc. The ordinal value can be determined by comparing the deviation to one or more predefined thresholds. In some examples, the numerical values ​​can have continuous values, e.g., continuous values ​​indicating deviation in a certain target coordinate system, such as shift expressed in [mm], rotation expressed in degrees, FOV opening expressed in percent, etc. Different numerical values ​​can reflect different quality aspects. For example, as shown in FIG. 8, X-ray image quality can be determined with respect to a quality vector having different quality features, such as field of view (FOV) compliance, rotation, and curvature. The quality metric can be a single value, such as a weighted sum of the numerical values ​​mentioned above.

[0072] The data-driven model is trained based on a training dataset having a number of training examples, each training example having sensor data of a body part acquired in an imaging session and an associated quality metric derived from image data acquired in said imaging session using a medical imaging modality. An exemplary training process is described below, particularly with respect to the embodiment shown in FIG.

[0073] Returning to FIG. 2, in block 230, step c), the apparatus provides the generated quality metrics for prospective quality assessment of the imaging exam prior to acquisition.

[0074] The trained data-driven model can be inferred during positioning / preparation of an imaging exam to provide predicted quality metrics. This allows an operator (e.g., radiologist) to optimize the predicted quality metrics before performing the actual exam. For example, a momentary prediction of the current positioning quality can be displayed while moving the patient under observation from the sensor.

[0075] In the example shown in Figure 3, the device 10 provides the generated quality metrics to a console 40. On a screen 42, the generated quality metrics can be viewed or reviewed. An operator can control image acquisition via the console 40 by activating a joystick or pedal or other suitable input means coupled to the console 40 if the generated quality metrics meet a criteria.

[0076] Optionally, the apparatus 10 may determine whether the generated quality metric meets a predetermined criterion and generate a signal indicative of whether the generated quality metric meets the predetermined criterion.

[0077] In some examples, the generated signal can include a signal for controlling a device that indicates whether the patient is ready for image acquisition. In one example, the device can be a speaker configured to generate an audio signal that indicates to the patient that the patient is correctly positioned and that the body part to be imaged should be kept motionless throughout the examination. In another example, the device can be a lighting device that generates a light signal (e.g., a green light) that indicates to the patient and / or operator that the patient is correctly positioned.

[0078] In some examples, the generated signal may include a signal to trigger the medical imaging device to begin image acquisition. In these examples, the operator does not need to manually control image acquisition. Rather, the medical imaging device 20 may be triggered to perform automatic image acquisition by a signal indicating that the patient is correctly positioned for image acquisition. Additionally, a warning signal may also be generated to inform the patient to keep the body part to be imaged still during the imaging examination.

[0079] Thus, the computer-implemented methods, apparatus, and systems described herein can improve the quality of medical image data acquired during real-world imaging exams, leading to fewer retakes, fewer delays in patient care, shorter workflow, and increased patient rates. In X-ray and CT exams, fewer retakes can reduce radiation dose to the patient.

[0080] 4 is a flowchart of a method 300 for generating training data for training a data-driven model. The method 300 is described below with reference to Figures 5 and 6. However, the method 300 is not limited to the examples of Figures 5 and 6.

[0081] Figure 5 shows another example of a system 100. In the example of Figure 5, the system 100 includes a medical imaging device 20 that is an X-ray imager having an X-ray source 20a and an X-ray detector 20b. The X-ray detector 20b is spaced from the X-ray source 20a to accommodate the body part of the patient being imaged, such as the ankle shown in Figure 5.

[0082] Generally, during image acquisition, a collimated X-ray beam (indicated by an arrow) emanates from an X-ray source 20a, passes through a patient in a region of interest (ROI), experiences attenuation due to interactions with materials therein, and then the attenuated beam strikes the surface of an X-ray detector 20b. The density of the organic material that makes up the ROI determines the level of attenuation. High density materials (e.g. bone) cause higher attenuation than less dense materials (e.g. tissue). The registered digital values ​​for the X-rays are then integrated into an array of digital values ​​to form an X-ray projection image for a given acquisition time and projection direction.

[0083] The overall operation of the X-ray imaging apparatus 20 can be controlled by an operator from a console 40. The console 40 can be coupled to a screen or monitor 42 on which acquired X-ray images or imager settings can be viewed or reviewed. An operator, such as a medical lab technician, can control image acquisitions performed via the console 40, for example by activating a joystick or pedal or other suitable input means coupled to the console 40, releasing individual X-ray exposures.

[0084] Depending on the request, various images of the ankle can be generated. Standard image series may include anterior-posterior (AP), mortis, and lateral images. If a calcaneal lesion is suspected, an additional image can be generated in the axial direction. In the example of FIG. 5, an AP image of the ankle is acquired by the radiography device 20.

[0085] The apparatus 10 may be an apparatus as described with respect to Figure 1. In the example of Figure 5, the apparatus 10 is a separate device configured to communicate with the sensor 30 via a wireless and / or wire-based interface. However, in an alternative example (not shown), the apparatus 10 may reside on the console 40 and operate as a software routine.

[0086] The sensor 30 may be a depth camera placed near or attached to the X-ray tube head 20a. The depth camera can provide optical depth information about the observed scene. If the depth camera is placed near the X-ray tube head 20a, pointed towards the X-ray detector 20b and calibrated, its geometry is known. Such a sensor can be used prior to acquisition to assist the radiologist in assessing the patient's position.

[0087] Returning to Figure 4, beginning at block 310, image data of a patient's body part is received. For example, the image data of the body part may be retrieved from a database. The image data in the database may be obtained from routine and / or study exams in one or more departments or sites. The database may include image data of one or more body parts from one or more patients.

[0088] The image data can be acquired, for example, by an X-ray imaging device, an MR imaging device, a CT imaging device, or a PET imaging device. The image data can also be acquired by a combined treatment / diagnosis device, such as an MR-Linac device, an MR proton therapy device, or a cone-beam CT device. For example, the image data can include various images of the ankle, which can be generated by the X-ray imaging device 20 shown in FIG. 5. An exemplary lateral image of the ankle is shown in FIG. 6.

[0089] At block 320, sensor data of a patient's body part is received. The received sensor data and the received image data can be previously acquired in the same imaging exam. The sensor data can be acquired by any suitable sensor, such as an array of optical sensors, depth sensors, thermal sensors, pressure sensors, ultrasonic sensors, radio frequency sensors, or any combination thereof. For example, as shown in FIG. 5, the sensor 30 can be a depth camera for capturing a depth image of the ankle.

[0090] In block 330, one or more features are extracted from the image data. The one or more features extracted from the image data may include, but are not limited to, landmarks, organ boundaries, regions of interest, image labels in the image data acquired by the medical imaging device, or any combination thereof. A fully automated procedure can be used to calculate one or more features, such as the bone contours C1-C4 of the tibia, fibula, talus, and calcaneus shown in FIG. 6. One option is to use image processing algorithms, such as image detection algorithms, that find the approximate center and bounding box of one or more features in the medical image. The position and / or orientation of each extracted feature is compared to the desired position and / or orientation of the respective feature to determine the deviation of the feature. For example, skeletal X-ray image quality can be evaluated based on field of view (FOV) compliance, medial-lateral rotation, cranio-caudal rotation, joint flexion, etc.

[0091] In block 340, a quality metric is generated. The quality metric may be a vector of values. Each value represents a deviation of a position and / or rotation of an anatomical feature in image data acquired using a medical imaging modality from a desired position and / or rotation of the anatomical feature. For example, an X-ray image quality can be determined with respect to a quality vector having different quality aspects such as FOV, rotation, and curvature.

[0092] In block 350, the quality metric derived from the image data acquired by the medical imaging device and the sensor data acquired by the sensor are provided as training data. Alternatively, an anatomical structure model of the target anatomical structure can be fitted to the sensor data. The quality metric derived from the image data acquired by the medical imaging device and the fitted anatomical structure model are provided as training data.

[0093] In block 360, the data-driven model is trained on the training data.

[0094] The annotated training data can be generated from routine examinations at one or more clinical sites. This continuous, unsupervised process of obtaining annotated training data provides the basis for the generation of a large-scale ground truth database, thereby overcoming the shortcomings of a limited number of training sets obtained by manual or semi-automated processes. When data with new, unforeseen characteristics arise, or when the desired results of the trained system need to be adapted, the continuously expanding training set database can make it possible to maintain the machine learning model for changes that occur over time.

[0095] Optionally, the data-driven model can be continuously trained in the inference phase, i.e., in the process of using the trained machine learning algorithm to make predictions. Figure 7 shows an example method 400 for performing weakly supervised continuous learning.

[0096] In this example, a quality metric is generated from sensor data acquired prior to image acquisition according to a procedure indicated by the dashed arrow. The quality metric may also be referred to as a predicted quality metric. Specifically, in block 420, acquired sensor data is received in a manner similar to the embodiment described with respect to block 210 of FIG. 2. In block 470, a quality metric is generated from the received sensor data using a trained data-driven model in a manner similar to the embodiment described with respect to block 220 of FIG. 2. In block 480, the generated quality metric (i.e., the predicted quality metric) is provided to train a model.

[0097] A further quality metric is generated from image data acquired by the medical imaging device after the actual imaging examination according to the procedure indicated by the solid arrows. The further quality metric may also be referred to as an actual quality metric. In particular, in block 410, acquired image data is received in a manner similar to the embodiment described with respect to block 310 of FIG. 4. In block 430, one or more features are extracted from the image data. The position and / or orientation of each extracted feature is compared to the desired position and / or orientation of the respective feature to determine a deviation of the feature, similar to the embodiment described with respect to block 330 of FIG. 4. In block 440, a quality metric is generated, which may be a vector of values. Each value represents a deviation of the position and / or rotation of an anatomical feature in the image data acquired with the medical imaging modality from the desired position and / or rotation of the anatomical feature. This may be done in a manner similar to the embodiment described with respect to block 340 of FIG. 4.

[0098] Optionally, in block 460, an operator (e.g., a radiologist) can review and check the quality metric vector for the exam and provide feedback, e.g., confirm / modify certain metrics, and feed this expert feedback into a continuous learning loop that implements a more reliable, strongly supervised learning strategy. For example, the apparatus 10 can receive user input indicating a user-defined quality metric of the received image data, determine a difference between the quality metric generated from the sensor data prior to image acquisition and the user-defined quality metric, and use the difference to further train the data-driven model.

[0099] The difference between the predicted quality metric derived from the sensor data prior to image acquisition and the actual quality metric derived from image data acquired after the actual image acquisition can be used as a loss function to perform successive weakly supervised learning.

[0100] For example, by setting thresholds, the continuous learning loop can be adapted to domain-specific quality requirements (e.g., field of view width) without the need for manual annotation.

[0101] FIG. 8 illustrates an example of a continuous learning framework for quality metric prediction using exemplary data. In this example, the actual quality metric is derived from image data acquired by a medical imaging device (e.g., the X-ray imager shown in FIG. 5) according to the procedure described with respect to blocks 410, 430, 440, and 460 of FIG. 7. The predicted quality metric is derived from sensor data acquired prior to image acquisition according to the procedure described with respect to blocks 420, 470, and 480 of FIG. 7. In this example, the X-ray image quality is predicted with respect to a quality vector with different quality aspects (field of view, rotation, curvature, etc.). An RGB / depth camera can provide sensor data for predicting the quality metric using a continuously updated data-driven model.

[0102] Also, unless expressly indicated to the contrary, it should be understood that in methods including multiple steps or acts recited in the claims, the order of the method steps or acts is not necessarily limited to the order in which the method steps or acts are recited.

[0103] In another exemplary embodiment of the invention, a computer program or a computer program element is provided, characterized in that it is adapted to carry out, on a suitable system, the method steps of the method according to one of the previous embodiments.

[0104] Thus, the computer program element can be stored in a computing unit which can be part of an embodiment of the present invention. This computing unit can be configured to execute or to direct the execution of the steps of the above-mentioned method. Furthermore, the computing unit can be adapted to operate each component of the above-mentioned device. The computing unit can be configured to operate automatically and / or to execute a user's order. The computer program can be loaded into the working memory of a data processor. The data processor can thus be equipped to execute the method of the present invention.

[0105] This exemplary embodiment of the invention encompasses both computer programs that use the invention from the outset and computer programs that use modern means to transform existing programs into programs that use the invention.

[0106] Moreover, the computer program may provide all the steps necessary to carry out the procedures of the exemplary embodiments of the procedures described above.

[0107] According to a further exemplary embodiment of the present invention, a computer readable medium, such as a CD-ROM, is presented, having stored thereon a computer program element, which computer program element is described by the previous section.

[0108] The computer program may be stored and / or distributed on a suitable medium, such as an optical storage medium or a solid-state medium provided together with or as part of other hardware, but may also be distributed in other forms, such as via the Internet or other wired or wireless communication systems.

[0109] However, the computer program can also be presented over a network such as the World Wide Web and downloaded from such a network into the working memory of a data processor. According to a further exemplary embodiment of the invention, a medium is provided for making available for downloading a computer program element, which computer program element is configured to perform a method according to one of the aforementioned embodiments of the invention.

[0110] It should be noted that the embodiments of the present invention are described with reference to different subject matters. In particular, some embodiments are described with reference to method type claims, and other embodiments are described with reference to mechanism type claims. However, those skilled in the art will understand from the above and the following description that, unless otherwise indicated, any combination of features belonging to one type of subject matter, as well as any combination between features relating to different subject matters, is disclosed in the present application. However, all features can be combined to provide a synergistic effect that is higher than the simple sum of the features.

[0111] While the invention has been illustrated and described in detail in the drawings and the foregoing description, such illustration and description are to be considered exemplary or explanatory and not restrictive. The invention is not limited to the disclosed embodiments. Other variations to the disclosed embodiments can be understood and effected by those skilled in the art in practicing the claimed invention, from a study of the drawings, the disclosure and the dependent claims.

[0112] In the claims, the word "comprising" does not exclude other elements or steps, and the indefinite article "a" or "an" does not exclude a plurality. A single processor or other unit may fulfill the functions of several items recited in the claims. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage. Any reference signs in the claims should not be interpreted as limiting their scope.

Claims

1. 1. A computer-implemented method for prospective quality assessment of a pre-acquisition imaging study, comprising: receiving sensor data of a body part of a patient imaged by a medical imaging device using a medical imaging modality; generating a quality metric from the received sensor data using a data-driven model, the data-driven model being trained based on a training dataset having a plurality of training examples, each training example comprising sensor data of the body part acquired in an imaging session and an associated quality metric derived from image data acquired using the medical imaging modality in the imaging session; providing the generated quality metric for a prospective quality assessment of a pre-acquisition imaging study, the quality metric being a vector of numbers, each number representing a deviation of a position and / or rotation of an anatomical feature in the image data acquired using the medical imaging modality from a desired position and / or rotation of the anatomical feature; A method having the following.

2. generating a quality metric from the received sensor data; generating the quality metric directly from the received sensor data; or fitting an anatomical model of a target anatomical structure to the received sensor data and generating the quality metric from the fitted anatomical model of the target anatomical structure; The computer-implemented method of claim 1 , comprising:

3. determining whether the generated quality metric meets a predetermined criterion; generating a signal indicating whether the generated quality metric meets a predetermined criterion; The computer-implemented method of claim 1 or 2, further comprising:

4. The computer-implemented method of claim 3 , wherein the signal comprises a signal that controls a device to indicate whether the patient is ready for image acquisition.

5. The computer-implemented method of claim 3 , wherein the signal comprises a signal that triggers the medical imaging device to begin image acquisition.

6. receiving image data of the patient's body part after image acquisition; determining other quality metrics based on the received image data; determining a difference between the quality metric generated from the sensor data before image capture and the other quality metric derived from the image data after image capture; using the difference to further train the data-driven model; The computer-implemented method of claim 1 or 2, further comprising:

7. receiving a user input indicating a user-defined quality metric for the received image data; determining a difference between the quality metric generated from the sensor data before image acquisition and the user-defined quality metric; using the difference to further train the data-driven model; The computer-implemented method of claim 6 further comprising:

8. The computer-implemented method of claim 1 or 2, wherein the sensor data is acquired by one or more of an array of optical sensors, depth sensors, thermal sensors, pressure sensors, ultrasonic sensors, and radio frequency sensors.

9. The computer-implemented method of claim 1 or 2, wherein the medical imaging modality is one or more of magnetic resonance imaging, ultrasound imaging, X-ray imaging, computed tomography imaging, and positron emission tomography imaging.

10. The computer-implemented method of claim 1 or 2, wherein the medical imaging modality comprises a hybrid modality having one or more of MR-Linac, MR Proton Therapy, and Cone Beam Computed Tomography.

11. 10. An apparatus for prospective quality assessment of pre-acquisition imaging examinations, the apparatus having one or more processing units for generating quality metrics, the processing units having instructions for performing the method of claim 1 or 2 when executed on the one or more processing units.

12. a medical imaging device configured to acquire image data of a body part of a patient; a sensor for acquiring sensor data of the body part of the patient; 12. The apparatus of claim 11, configured to receive the sensor data and provide quality metrics for prospective quality assessment of a pre-acquisition imaging examination; A system having:

13. 13. The system of claim 12, wherein the medical imaging device is configured to initiate image acquisition according to the provided quality metric, and / or the system further comprises a device configured to indicate whether the patient is ready for image acquisition based on the quality metric.

14. 3. A computer program comprising instructions which, when said computer program is executed by at least one processing unit, cause the at least one processing unit to carry out the steps of the method according to claim 1 or 2.

Images