Systems and methods for synchronizing an imaging system and an edge computing system

Abstract

Methods and systems for synchronizing an imaging system with an edge computing system (ECS) are provided. In one embodiment, the system includes an imaging system including at least a scanner and a processor configured to reconstruct an image from data acquired during scanning of a subject via the scanner; and a computing device communicatively coupled to the imaging system and positioned external to the imaging system, the computing device configured to generate a decision support computation based on the data, wherein the imaging system transmits the data to the computing device during scanning. In this way, the processing capabilities of the imaging system can be substantially extended. Furthermore, post-processing techniques can be performed in parallel with scanning, thereby reducing the time for clinical diagnosis.

Description

[0001] Cross-reference to related applications

[0002] This application claims priority to U.S. Provisional Application No. 62 / 657,633, entitled “Systems and Methods for Synchronizing Imaging Systems and an Edge Computing System,” filed April 13, 2018. The entire contents of the foregoing application are incorporated herein by reference for all purposes. Technical Field

[0003] The embodiments of the subject matter disclosed herein relate to non-invasive diagnostic imaging, and more specifically to expanding the processing capabilities of imaging systems. Background Technology

[0004] Non-invasive imaging techniques allow for the acquisition of images of the internal structures of a patient or object without the need for invasive procedures on that patient or object. Specifically, techniques such as computed tomography (CT) use various physical principles, such as differential transmission of X-rays through a target volume, to acquire image data and construct tomographic images (e.g., a three-dimensional representation of the interior of the human body or other imaging structures).

[0005] New post-processing techniques can significantly improve the functionality of imaging systems and the accuracy of clinical diagnosis. For example, modern deep learning techniques can allow for the accurate detection of lesions in tomographic images with lower image quality, thereby enabling the reduction of radiation dose (and thus potentially image quality) without sacrificing the diagnostic effectiveness of the imaging system.

[0006] However, imaging systems have limited processing power, and any overhead in processing power may be insufficient to support new post-processing techniques. Upgrading the imaging system to accommodate additional processing power or replacing the entire imaging system is prohibitively expensive. Furthermore, even if such post-processing techniques can be performed by the imaging system, they may be performed after the imaging procedure, which will increase the time required for clinical diagnosis. Summary of the Invention

[0007] In one embodiment, the system includes an imaging system comprising at least a scanner and a processor configured to reconstruct images from data acquired during scanning of a subject via the scanner; and a computing device communicatively coupled to the imaging system and located externally to the imaging system, configured to generate decision support computations based on the data, wherein the imaging system transmits data to the computing device during scanning. In this way, the processing power of the imaging system can be substantially extended. Furthermore, post-processing techniques can be performed in parallel with scanning, thereby reducing the time required for clinical diagnosis.

[0008] It should be understood that the above brief description is provided to introduce selected concepts further described in the detailed embodiments in a simplified form. This is not intended to identify key or essential features of the claimed subject matter, the scope of which is uniquely defined by the claims following the detailed embodiments. Furthermore, the claimed subject matter is not limited to embodiments that address any shortcomings mentioned above or in any part of this disclosure. Attached Figure Description

[0009] The invention will be better understood by referring to the following description of non-limiting embodiments, in which:

[0010] Figure 1 A simplified exemplary system for expanding the capabilities of an imaging system according to one embodiment is shown in block diagram;

[0011] Figure 2 A high-level flowchart illustrating an exemplary method for synchronizing an imaging system with an edge computing system according to one embodiment is shown;

[0012] Figure 3 A high-level flowchart illustrating an exemplary method for generating deep learning training data using an imaging system, according to one embodiment, is shown.

[0013] Figure 4 A high-level flowchart illustrating an exemplary method for a shortcut mode of an imaging system according to one embodiment is shown;

[0014] Figure 5 A high-level flowchart illustrating an exemplary method for managing applications on an edge computing system according to one embodiment is shown;

[0015] Figure 6 A drawing view of an imaging system according to one embodiment is shown;

[0016] Figure 7 A block diagram of an exemplary imaging system according to one embodiment is shown;

[0017] Figure 8 The following is a comparison of the capabilities of an extended imaging system according to one embodiment. Figure 1 A more detailed block diagram of the exemplary system; and

[0018] Figure 9 A flowchart illustrating an exemplary method for generating decision support outputs of an imaging system using an edge computing system, according to one embodiment, is shown. Detailed Implementation

[0019] The following describes various implementation schemes related to diagnostic imaging. Specifically, systems and methods for synchronizing at least one imaging system with an edge computing system (ECS) are provided. The processing power of the imaging system can be extended by coupling the imaging system to the ECS, such as... Figure 1 As shown in the diagram. Imaging data acquired by the imaging system can be transmitted or streamed to the ECS during scanning. Methods for synchronizing imaging data transmission to the ECS include... Figure 2 The method described herein allows imaging data to be processed by a deep learning (DL) application during scanning, thereby reducing the amount of time required to obtain decision support. The DL application can be trained using additional information related to the acquired imaging data, as well as information characterizing the scanned subject, such as… Figure 3 As described in [the text]. When coupled to an ECS, the imaging system's extended processing capabilities allow it to operate in a "shortcut mode," where scanning is performed with minimal intervention by the imaging system operator or operator input, such as [example needed]. Figure 4 As described in [the document]. New and updated DL applications can be retrieved from a remote repository, such as [examples of DL applications]. Figure 5 As described herein, this allows ECS to offer state-of-the-art DL capabilities compatible with imaging systems. Figure 6 and Figure 7 Examples of CT imaging systems that can be used to acquire images processed according to the technology of the present invention are provided.

[0020] The ECS connects to the imaging system / scanner. The ECS includes a CPU / GPU that runs one or more virtual machines (VMs) configured for different types of tasks. Data is streamed from the scanner to the ECS in real time, where the ECS processes the data (in the image and / or projection space) and returns the results. In this way, the imaging system appears to have additional processing power because the post-processing performed by the ECS is output along with the reconstructed image through the imaging system's user interface.

[0021] Data is streamed to the ECS in sync with the scanner's status. Data is only transmitted from the scanner to the ECS when the scanner is not in a critical state. In interventional mode (e.g., when a physician performs an intervention such as contrast agent injection at the scanner), the scanner does not transmit data at all to avoid data corruption.

[0022] ECS provides task-based decision support. A specific task input to the imaging system triggers secondary tasks input to and executed by the ECS. For example, the task might specify a particular scanning protocol and / or image reconstruction type performed by the scanner, while the secondary task might specify the application of relevant post-processing techniques to the acquired data. These post-processing techniques could include deep learning analysis of the acquired data. Figure 9 As described in the method, the ECS can select an appropriate DL application based on secondary tasks, inspection types, and / or other information, and generate decision support using the selected DL application. Each instance of decision support generated at the ECS can be saved along with associated data (e.g., the DL application used, the inspection type, and the final / edited inspection report). After a threshold number of decision supports have been generated and saved on the ECS, one or more of the DL applications can be retrained using new data both locally (e.g., in the DL applications stored on the ECS) and globally (e.g., by sending updated model weights along with data from other locations to a central server to create an updated global model).

[0023] Figure 1 A block diagram of an exemplary system 100 for extending the capabilities of an imaging system 101 using an edge computing system (ECS) 110, according to one embodiment, is shown. The imaging system 101 may include any suitable non-invasive imaging system, including but not limited to computed tomography (CT) imaging systems, positron emission tomography (PET) imaging systems, single-photon emission computed tomography (SPECT) imaging systems, magnetic resonance (MR) imaging systems, X-ray imaging systems, ultrasound systems, and combinations thereof (e.g., multimodal imaging systems such as PET / CT, PET / MR, or SPECT / CT imaging systems). This document relates to... Figure 6 and Figure 7 An exemplary imaging system is further described.

[0024] Imaging system 101 includes processor 103 and nontransitory memory 104. One or more methods described herein can be implemented as executable instructions in nontransitory memory 104, which, when executed by processor 103, cause processor 103 to perform various actions. This document relates to... Figures 2 to 4 Further describe this type of method.

[0025] Imaging system 101 also includes scanner 105 for scanning a subject (such as a patient) to acquire imaging data. Depending on the type of imaging system 101, scanner 105 may include several components necessary for scanning a subject. For example, if imaging system 101 includes a CT imaging system, scanner 105 may include a CT tube and detector array, as well as various components for controlling the CT tube and detector array, as described herein. Figure 6 and Figure 7 Further discussion follows. For example, if imaging system 101 includes an ultrasound imaging system, then scanner 105 may include an ultrasound transducer. Therefore, as used herein, the term "scanner" refers to a component of an imaging system that is used and controlled to perform a scan of a subject.

[0026] The type of imaging data acquired by scanner 105 also depends on the type of imaging system 101. For example, if imaging system 101 includes a CT imaging system, the imaging data acquired by scanner 105 may include projection data. Similarly, if imaging system 101 includes an ultrasound imaging system, the imaging data acquired by scanner 105 may include analog and / or digital echoes of ultrasound waves emitted by an ultrasound transducer into the subject's body.

[0027] In some examples, the imaging system 101 includes a protocol engine 106 for automatically selecting and adjusting a scanning protocol for scanning a subject. The scanning protocol selected by the protocol engine 106 specifies various settings for controlling the scanner 105 during the scanning of the subject. As further discussed herein, the protocol engine 106 may select or determine the scanning protocol based on an indicated primary task. Although the protocol engine 106 is depicted as a separate component from the non-transitory memory 104, it should be understood that in some examples, the protocol engine 106 may include software modules stored as executable instructions in the non-transitory memory 104 that, when executed by the processor 103, cause the processor 103 to select and adjust the scanning protocol.

[0028] The imaging system 101 also includes a user interface 107 configured to receive input from an operator of the imaging system 101 and to display information to the operator. For this purpose, the user interface 107 may include one or more of input devices (including but not limited to a keyboard, mouse, touchscreen device, microphone, etc.) and output devices (including but not limited to a display device, printer, etc.).

[0029] In some examples, the imaging system 101 also includes a camera 108 for assisting in the automatic positioning of the subject within the imaging system. For example, the camera 108 may capture real-time images of the subject within the imaging system, and the processor 103 may determine the subject's position within the imaging system based on these real-time images. The processor 103 may then control a stage motor controller, for example, to adjust the position of the stage against which the subject rests, in order to position at least one region of interest (ROI) of the subject within the imaging system. Furthermore, in some examples, the scan range may be determined at least approximately based on the real-time images captured by the camera 108.

[0030] System 100 also includes an ECS 110, which is communicatively coupled to imaging system 101 via a wired or wireless connection. ECS 110 includes multiple processors 113 running one or more virtual machines (VMs) 114 configured for different types of tasks. The multiple processors 113 include one or more graphics processing units (GPUs) and / or one or more central processing units (CPUs). ECS 110 also includes non-transitory memory 115 storing executable instructions that can be executed by one or more of the multiple processors 113. Non-transitory memory 115 may also include a deep learning (DL) model 116, which can be executed by the VMs 114 of the multiple processors 113. Although in Figure 1 Only one DL model is described in this document, but it should be understood that the ECS 110 may include more than one DL model stored in non-transitory memory.

[0031] In some examples, system 100 also includes one or more external databases 130 to which imaging system 101 and ECS 110 are communicatively coupled via network 120. As an exemplary and non-limiting example, the one or more external databases 130 may include one or more of a hospital information system (HIS), a radiology information system (RIS), a picture archiving and communication system (PACS), and an electronic medical record (EMR) system. Imaging system 101 and / or ECS 110 may retrieve information such as subject metadata, which may include metadata describing a specific subject to be scanned or related to a specific subject to be scanned (e.g., the patient's age, sex, height, and weight), which may be retrieved from an EMR for subjects. As further described herein, imaging system 101 and / or ECS 110 may use the subject metadata retrieved from one or more external databases 130 to automatically determine scanning protocols, train deep learning models, etc.

[0032] In some examples, system 100 also includes a storage library 140 communicatively coupled to one or more of imaging system 101 and ECS 110 via network 120. Storage library 140 stores a plurality of applications 142 that can be used by one or more of imaging system 101 and ECS 110. For this purpose, imaging system 101 and / or ECS 110 can retrieve one or more of the plurality of applications 142 from storage library 140 via network 120. Alternatively, storage library 140 can push one of the plurality of applications 142 to one or more of imaging system 101 and ECS 110. This document relates to... Figure 5 The method for retrieving the application from the repository 140 is further described.

[0033] Figure 2 A high-level flowchart illustrating an exemplary method 200 for synchronizing an imaging system with an edge computing system (ECS) according to one embodiment is shown. Specifically, method 200 involves streaming imaging data acquired during scanning from an imaging system (such as imaging system 101) to an ECS (such as ECS 110) for processing the imaging data concurrently with the scanning. Regarding... Figure 1 The system and components described herein and hereinafter are used to describe method 200, but it should be understood that the method can be implemented with other systems and components without departing from the scope of this disclosure. Method 200 can be implemented as executable instructions in non-transitory memory 104 of imaging system 101, which can be executed by processor 103 of imaging system 101.

[0034] Method 200 begins at 205. At 205, method 200 receives instructions for an initial task, also referred to herein as the primary task. The initial task includes clinical tasks to be performed by the imaging system and thus typically specifies the type of scan that should be performed by the imaging system. The initial task is generally a clinical action that must be completed during pre-scanning and scanning of the imaging prescription. Some examples of tasks related to pre-scanning are patient setup, receiving and viewing data from EMR, CDSS, and HIS / RIS, and selecting a protocol for scanning. Some examples of tasks related to scanning are patient positioning, image acquisition, and image reconstruction. In one example, method 200 may receive instructions for the initial task, for example, via user interface 107 of imaging system 101. That is, the operator of imaging system 101 may select or otherwise indicate the initial task via user interface 107. In another example, method 200 may automatically identify the initial task, for example, by evaluating the EMR of the subject to be scanned.

[0035] For example, the initial task may include or be defined by the diagnostic objective of the imaging scan (e.g., the reason for referencing the imaging scan to the patient). Exemplary diagnostic objectives may include diagnosing the presence (or absence) of cerebral hemorrhage, diagnosing the presence (or absence) of liver lesions, determining the presence or extent of a spinal fracture, and performing organ segmentation for planned use in radiation therapy. Each diagnostic objective may target a specific anatomical structure or a set of anatomical structures, and thus may specify the type of scan / examination to be performed. For example, the diagnosis of cerebral hemorrhage may target the brain and thus specify the performance of a non-contrast head scan; the diagnosis of liver lesions may target the liver and thus specify the performance of an abdominal scan, and specifically, a contrast-enhanced multiphase liver study may be specified at different timings; the diagnosis of spinal fractures may target the neck and / or back and thus specify the performance of a full non-contrast spinal scan; and organ segmentation may target the entire abdominal region (or even the whole body) and thus specify the performance of a non-contrast whole-body scan. Each different scan type may have a set of associated scan parameters that specify how the scanner is controlled during the scan. For example, for computed tomography (CT) scans, each different scan type can specify the specific tube current (mA), tube voltage (kV), and gantry rotation speed of the CT scanner during the scan.

[0036] Continuing at 210, method 200 determines whether an initial task is associated with a secondary task. The initial task specifies how imaging data can be acquired, while the secondary task specifies how the imaging data can be processed. As a non-limiting example, a secondary task may include automated lesion detection, organ classification, segmentation, or any type of post-processing method that can be applied to the imaging data. Secondary tasks are typically clinical actions that must be performed during post-scanning of the imaging prescription. Some examples of tasks associated with post-scanning include post-processing images by applying a post-processing application.

[0037] In some examples, a specific initial task may always specify one or more secondary tasks to be performed in conjunction with the initial or primary task. Additionally or alternatively, the initial task instruction received at 205 may specify one or more secondary tasks. For example, the initial task may not specify secondary tasks, but the operator of the imaging system may select secondary tasks when instructing the initial task. As a non-limiting example, the initial task may specify a brain scan, and secondary tasks may include lesion detection. Since lesion detection may not be necessary for every brain scan, the operator of the imaging system may optionally select secondary tasks including lesion detection for the initial task of the brain scan in specific cases where lesions may be suspected. Additionally or as an alternative to the operator manually selecting secondary tasks, in some examples, method 200 may automatically determine secondary tasks based on the initial task and subject metadata such as the subject's EMR.

[0038] If the initial task is not associated with a secondary task (“No”), method 200 proceeds to 212. At 212, method 200 performs a scan according to the initial task instructions. Method 200 may further reconstruct and output one or more images from the imaging data acquired during the scan. Since no secondary task is associated with the initial task, the imaging data acquired at 212 is not streamed to the ECS. Method 200 then terminates.

[0039] However, referring again to 210, if the initial task is associated with a secondary task (“Yes”), then method 200 continues to 215. At 215, method 200 outputs a secondary task indication to ECS 110. ECS 110 performs post-processing of the imaging data based on the secondary task indication.

[0040] At 220, method 200 begins scanning the subject with scanner 105 to acquire imaging data according to the initial task instructions. For example, method 200 begins scanning the subject according to a scanning protocol associated with the initial task. As a non-limiting example, the scanning protocol may be selected by protocol engine 106.

[0041] During the scan that begins at 220, method 200 synchronizes the imaging system 101 to the ECS 110 based on the state of the scanner 105. To this end, at 225, method 200 evaluates the state of the scanner 105. At 230, method 200 determines whether the scanner 105 is in a critical state. The scanner 105 can be in a critical state when asynchronous external operation may negatively affect its ability to meet basic safety requirements. For example, the scanner 105 may be in a critical state when active data acquisition is in progress, such as when the X-ray source is on and image data is being stored to a disk or memory storage location. If the imaging system is synchronized with the ECS during such a period (so that imaging data can be sent from the imaging system to the ECS), it may result in scan failure (e.g., due to data loss, delay, or other problems), and the patient will need to be rescanned, thus exposing the patient to additional radiation doses. Primarily, the CT scanner is in a critical state when the X-ray source is on or during a rapid acquisition sequence. Another example of a critical state is when the system performs an intervention procedure, where both the scanning of data storage and the image display operations are time-sensitive and essential for safety.

[0042] If the scanner is in a critical state (“Yes”), method 200 proceeds to 232, where method 200 continues scanning. More specifically, method 200 continues scanning the subject but does not transmit the acquired imaging data to ECS 110.

[0043] However, referring again to 230, if the scanner is not in a critical state (“No”), method 200 continues to 233, wherein method 200 transmits the acquired imaging data to ECS 110 for post-processing. ECS 110 processes the transmitted imaging data according to a secondary task.

[0044] Method 200 proceeds from both 232 and 233 to 235. At 235, method 200 determines whether the scan is complete. If the scan is not complete (“No”), method 200 proceeds back to 237, where method 200 continues scanning. Method 200 proceeds to 225 to re-evaluate the scanner status. Therefore, method 200 continuously evaluates the status of scanner 105 during scanning to determine whether scanner 105 is in a critical state, and only streams or transmits the imaging data acquired during scanning to ECS 110 if scanner 105 is not in a critical state. In other words, unless scanner 105 is in a critical state, method 200 continuously streams the imaging data to ECS 110 as it is acquired during scanning. In this way, ECS 110 can receive and process imaging data simultaneously with scanning. Furthermore, when scanner 105 is operating in a critical state, the imaging data is not corrupted by transmission during the critical state, and the operation of imaging system 101 is not disturbed.

[0045] Once method 200 determines at 235 that the scan is complete (“Yes”), method 200 proceeds to 240. At 240, method 200 reconstructs one or more images from the acquired imaging data. As a non-limiting example, method 200 may use any suitable iterative or analytical image reconstruction algorithm to reconstruct one or more images.

[0046] At 245, method 200 receives decision support from ECS 110. The decision support includes the results of one or more processing algorithms applied to imaging data that is streamed to ECS 110 during scanning at 233. For example, if the examination is a brain hemorrhage examination performing a head scan, the decision support output may include an indication of whether hemorrhage was detected. If the examination is a liver lesion examination scanning the liver, the decision support output may include whether a lesion was detected, and if a lesion was detected, the decision support output may include an indication of the size, shape, location, etc., of the lesion.

[0047] At point 250, method 200 outputs one or more images and decision support. For example, method 200 may output one or more images and decision support to a display device for display to the operator of imaging system 101. Additionally or alternatively, for example, method 200 may output one or more images and decision support to a mass storage device for later viewing, or to a Picture Archiving and Communication System (PACS) for viewing at a remote workstation. Method 200 then terminates.

[0048] Therefore, the method for the imaging system includes performing a scan of the subject to acquire imaging data, transmitting the imaging data during the scan to a computing device communicatively coupled to and located outside the imaging system, receiving decision support output from the computing device, which is computed by the computing device from the imaging data, and displaying the image reconstructed from the imaging data and the decision support output. In this way, deep learning techniques can be used to provide decision support in parallel with the acquisition of imaging data, thereby reducing the time required for informed diagnosis.

[0049] Automating the analysis of clinical images using deep learning (DL) techniques can greatly simplify and improve clinical diagnoses made by physicians using such images. However, preparing datasets to train DL algorithms can be difficult and time-consuming, as images and corresponding diagnoses must be manually compiled and prepared for input into the algorithm. Furthermore, such training datasets are unnecessarily limited in terms of the amount of potentially usable information related to scans and that can be utilized by DL algorithms. Therefore, the following section discusses… Figure 3 The imaging system 101 outputs data suitable for input to DL algorithms (such as DL application 116 in ECS 110). The output data includes scan data, image data, patient EMR data, scanner type (and other scanner metadata), scan protocol, decision support output, and any clinical diagnoses associated with the scan. For this purpose, the imaging system can retrieve data from the Hospital Information System (HIS) and / or Radiology Information System (RIS) and the EMR of a given patient (to obtain patient data and clinical outcomes). The output data can be used by DL algorithms to optimize / personalize scan protocols for different patients and image quality goals, improve decision support, and potentially automate clinical diagnoses.

[0050] Figure 3 A high-level flowchart illustrating an exemplary method 300 for generating deep learning training data using an imaging system, according to one embodiment, is shown. Specifically, method 300 involves training a deep learning model using all information that may potentially be related to the quality of the reconstructed image and its analysis. (About...) Figure 1The system and components described in method 300 are described herein, but it should be understood that the method can be implemented with other systems and components without departing from the scope of this disclosure. Method 300 can be implemented as executable instructions in the non-transitory memory 104 of imaging system 101 and executed by the processor 103 of imaging system 101.

[0051] Method 300 begins at 305. At 305, method 300 receives a selection of a scanning protocol. The scanning protocol may be selected manually by an operator via user interface 107, or may be selected automatically, for example, via protocol engine 106.

[0052] At 310, method 300 retrieves subject metadata for the subject to be scanned from one or more external databases. The subject metadata may include at least a subset of information relating to the subject, and therefore may include, but is not limited to, demographic information and medical history. Method 300 may, for example, retrieve subject metadata from one or more databases 130 via network 120, including one or more of the HIS, RIS, and EMR databases.

[0053] At 315, method 300 performs a scan of the subject according to a scanning protocol to acquire imaging data. For example, method 300 may control scanner 105 to scan the subject, wherein the scanning protocol selected at 305 specifies the control parameters of scanner 105. At 320, method 300 reconstructs an image from the imaging data acquired during the scan at 315. Method 300 may reconstruct the image using any suitable image reconstruction algorithm depending on the modality of the imaging system.

[0054] At position 325, method 300 transmits the image reconstructed at position 320 and / or the imaging data acquired at position 315 to the ECS 110. The ECS 110 processes the image and / or imaging data using one or more DL algorithms to generate decision support outputs. While transmitting imaging data... Figure 3 The process is described as occurring after the scan, but it should be understood that in some examples, method 300 may transfer imaging data during the scan at 315, allowing the ECS 110 to process the imaging data simultaneously with the scan at 315, as described above regarding... Figure 2 The subject of discussion.

[0055] ECS 110 processes image and / or imaging data during or after scanning to generate decision support output. At 330, method 300 receives decision support output computed for the image and / or imaging data from ECS 110 using a learning model (e.g., a deep learning algorithm, such as a neural network).

[0056] At 335, method 300 displays the image and decision support output. Both the image and the decision support output can be displayed via a display device, such as that of the imaging system 101. Depending on the type of decision support output, the decision support output can be overlaid on or displayed side-by-side with the image.

[0057] At 340, method 300 receives an outcome decision relating to the displayed image and decision support output. The outcome decision includes, for example, a clinical diagnosis made by a physician or radiologist based on the displayed image and decision support output. Additionally or alternatively, the outcome decision may include ground truth relating to the decision support output. For example, if the decision support output includes the detection or classification of an object (such as a lesion) in the image, the ground truth used for the decision support output may include an indication that the detection or classification was correct or incorrect.

[0058] At 345, method 300 updates the training dataset with imaging data, images, scanning protocols, subject metadata, decision support outputs, outcome decisions, and system metadata. Since the training dataset can be located remotely from the imaging system 101, rather than in the imaging system's non-transitory memory 104, in some examples, updating the training dataset may include aggregating the imaging data, images, scanning protocols, subject metadata, decision support outputs, outcome decisions, and system metadata into a single case to be added to the training dataset. System metadata includes information characterizing the imaging system 101 itself, such as the model number of the imaging system 101, the manufacturing date of the imaging system 101, etc.

[0059] At point 350, method 300 transfers the updated training dataset to ECS 110 to update the learning model used to generate the decision support output received at point 330. In this way, additional data that can influence the performance of the learning model, such as subject metadata, system metadata, scanning protocols, and clinical diagnoses made by physicians based on reconstructed images and / or decision support outputs, can be used to improve the performance of the learning model. Method 300 then concludes.

[0060] Therefore, the method for the imaging system includes performing a scan of the subject according to a scanning protocol to acquire imaging data; displaying images and decision support associated with the images, the images being reconstructed from the imaging data, and the decision support being computed using a learning model and the imaging data; and updating the training dataset for the learning model with the imaging data, images, scanning protocol, subject metadata describing the subject, decision support, outcome decisions related to the images and decision support, and system metadata related to the imaging system.

[0061] As imaging systems become more powerful and complex, users may find their operation too cumbersome. Radiologists and technicians must be trained to select and prepare the correct scan protocol for each patient, who may also be positioned differently within the imaging system for a given protocol. If any part of the scan preparation is incorrect, the quality of one or more images obtained may be too poor for clinical use, necessitating a re-scan. Furthermore, image analysis must be performed by the radiologist / physician, thus potentially slowing the turnaround time for clinical reporting. (See below for more information.) Figure 4 Furthermore, the imaging system 101 may therefore include a “quick mode” that utilizes the extended processing power provided by the ECS 110 and the more sophisticated DL application 116 to automate as many parts of the imaging process as possible. Hospital information system, radiology information system, and electronic medical record data are used by the protocol engine 106 to select and personalize a scanning protocol for a given patient; in some examples, this scanning protocol may be assisted by deep learning. The patient is automatically positioned based on the imaging protocol and with the assistance of the camera. The camera further enables the determination of the correct scanning range for the patient, allowing for further adjustments and personalization of the scanning protocol for the patient. Scanning and post-processing / decision support are performed automatically by the imaging system 101 and the ECS 110. The DL application 116, executed by multiple processors 113 of the ECS 110, provides automated processing of the acquired data and at least a draft evaluation of the reconstructed images.

[0062] Figure 4 A high-level flowchart illustrating an exemplary method 400 for a shortcut mode of an imaging system according to one embodiment is shown. Specifically, method 400 involves controlling the imaging system to scan a subject with minimal operator intervention or input. About Figure 1 The system and components described in method 400 are described herein, but it should be understood that the method can be implemented with other systems and components without departing from the scope of this disclosure. Method 400 can be implemented as executable instructions in non-transitory memory 104 of imaging system 101 and non-transitory memory 115 of ECS 110, and can be executed by processor 103 of imaging system 101 and multiple processors 113 of ECS 110.

[0063] Method 400 begins at 405. At 405, method 400 receives an indication of a shortcut mode for scanning. Method 400 may receive the indication of a shortcut mode for scanning, for example, via a user interface 107 of the imaging system 101. In some examples, the user interface 107 may include a dedicated button, switch, or another mechanism for indicating the desired use of a shortcut mode of the imaging system 101.

[0064] At 410, method 400 receives an identifier of the subject to be scanned. In some examples, method 400 may receive the identifier of the subject to be scanned via user interface 107. For example, an operator of imaging system 101 may manually enter the identifier of the subject to be scanned. In other examples, method 400 may automatically identify the subject to be scanned. As an exemplary and non-limiting example, method 400 may acquire a facial image of the subject via camera 108, and facial recognition technology may be used to identify the subject based on the image.

[0065] At 415, method 400 retrieves the EMR of the subject identified at 410, for example, by accessing one or more databases 130 via network 120. At 420, method 400 determines the initial or primary task based on the EMR. As discussed above, the primary task may indicate the clinical context of the scan and thus indicate what type of scan should be performed. Method 400 may determine the primary task from the EMR, which may include a physician's prescription for a specific type of scan.

[0066] At 425, method 400 determines a scanning protocol for the primary task. For example, method 400 may input the primary task into the protocol engine 106 of imaging system 101 to determine the scanning protocol. In other examples, the primary task may be associated with a specific scanning protocol. Furthermore, the scanning protocol may be determined or adjusted based on the subject. For example, the scanning protocol may specify more or less radiation dose based on the subject's age and size.

[0067] At 430, method 400 positions the subject within the imaging system. In some examples, method 400 may determine the subject's position on a movable stage relative to the imaging system, for example, by processing real-time images of the subject captured by camera 108. Method 400 may control a stage motor controller to adjust the stage, and thus the subject's position, such that the region of interest to be scanned is within the imaging area of ​​the imaging system (e.g., positioned within a gantry between the radiation source and the radiation detector).

[0068] At 435, method 400 determines the scanning range for the subject. Method 400 may determine the scanning range for the subject based on real-time images captured by camera 108. For example, different subjects have different body sizes, so the scanning range should be adjusted accordingly. Therefore, method 400 may evaluate the images captured by camera 108 to determine the size and proportions of the subject, and may subsequently set an appropriate scanning range for scanning the ROI of the subject. At 440, method 400 uses the scanning range determined at 435 to adjust the scanning protocol.

[0069] At 445, method 400 performs a scan of the subject according to the adjusted scanning protocol to acquire imaging data. For example, method 400 controls scanner 105 to scan the subject according to the adjusted scanning protocol. At 450, method 400 outputs the imaging data to ECS 110. (As mentioned above regarding...) Figure 2 In some examples, as discussed, imaging data can be output to the ECS 110 for processing during scanning. The ECS 110 uses a learned model to process the imaging data to generate decision support output. At 455, method 400 receives the decision support output from the ECS 110.

[0070] Continuing at 460, method 400 reconstructs the image from the imaging data acquired at 445. Method 400 can reconstruct the image from the imaging data using any suitable image reconstruction technique. Although Figure 4 Image reconstruction is described as occurring after the imaging data is output to the ECS 110; however, it should be understood that in some examples, the reconstructed image may be transferable to the ECS 110, and the decision support output received at 455 may be generated by the ECS 110 based on the reconstructed image rather than the original imaging data. At 465, method 400 outputs the image and decision support output to one or more of the following: a display device for display, a mass storage device for subsequent retrieval and viewing, and a PACS. Method 400 then terminates.

[0071] Therefore, the method for the imaging system includes receiving an identifier of a subject to be scanned, automatically determining a personalized scanning protocol for the subject, automatically performing a scan of the subject according to the personalized scanning protocol to acquire imaging data, and displaying images and decision support, which are automatically generated from the imaging data.

[0072] As mentioned above, post-processing techniques used in imaging systems are regularly improved over time. However, once an imaging system is installed, it is difficult to update it to include improved algorithms and new features. (See this article for more information.) Figure 5 As further discussed above, the application repository enables the remote deployment of software applications for imaging systems. Figure 1As discussed, imaging system 101 and / or ECS 110 may be coupled to application store 140 via network 120. In some examples, imaging system 101 or ECS 110 may retrieve new or updated applications 142 from application store 140. Alternatively, application store 140 may push applications 142 to ECS 110 and / or imaging system 101. Furthermore, as discussed below, some applications may be compatible only with specific combinations of imaging system 101 and ECS 110. For example, different applications may be displayed / deployed to a newer high-power imaging system coupled to a low-power ECS, relative to an outdated low-power imaging system coupled to a high-power ECS.

[0073] Figure 5 A high-level flowchart illustrating an exemplary method 500 for managing applications on an edge computing system (ECS) according to one embodiment is shown. Specifically, method 500 involves retrieving new or updated applications from external storage for processing image and / or imaging data. (About...) Figure 1 The method 500 describes the system and components described herein, but it should be understood that the method can be implemented with other systems and components without departing from the scope of this disclosure. The method 500 can be implemented as executable instructions in the non-transitory memory 115 of the ECS 110 and can be executed by one or more of the plurality of processors 113 of the ECS 110.

[0074] Method 500 begins at 505. At 505, method 500 transmits an access request to a storage library, such as storage library 140. The access request may include identifiers of imaging system 101 and ECS 110. Storage library 140 includes multiple applications that may or may not be compatible with one or more of imaging system 101 and ECS 110. Therefore, storage library 140 determines, based on the identifiers of imaging system 101 and ECS 110, which of the multiple applications is compatible with a given combination of imaging system 101 and ECS 110.

[0075] At 510, method 500 receives a list of compatible applications from storage 140. The list of compatible applications may include a subset of multiple applications stored in storage 140.

[0076] At 515, method 500 receives a selection of an application from a list of compatible applications. For example, an operator of imaging system 101 can view a list of compatible applications and select a desired application from the list via user interface 107. This selection can thus be transmitted from imaging system 101 to ECS 110. In other examples, ECS 110 may include its own user interface for enabling application selection, and thus the selection of an application can be received via said user interface.

[0077] Additionally or alternatively, the selection of applications can be automatic. For example, if the storage vault 140 includes updated versions of applications already installed on the ECS 110, method 500 can automatically select the updated version of the application from a list of compatible applications.

[0078] At 520, method 500 retrieves the selected application from storage 140 and installs it locally in non-transitory storage 115. Therefore, Figure 1 The DL application 116 in the non-transitory memory 115 of the ECS 110 depicted may include an application retrieved from the storage library 140.

[0079] At 525, method 500 uses an application to generate decision support computation from imaging data received by imaging system 101. For example, if the application includes a segmentation application, method 500 can segment the image reconstructed from the imaging data acquired using imaging system 101, and the image segmentation includes decision support computation. At 530, method 500 outputs the decision support computation to imaging system 101. Then method 500 terminates.

[0080] Therefore, a method for an ECS that is communicatively coupled to an imaging system and located outside the imaging system includes: receiving a list of compatible deep learning applications from an application storehouse communicatively coupled to the ECS via a network; retrieving a deep learning application selected from the list from the application storehouse; receiving imaging data from the imaging system; processing the imaging data with the deep learning application to generate decision support computation; and outputting the decision support computation to the imaging system.

[0081] Figure 6An exemplary CT system 600 configured to allow for rapid and iterative image reconstruction is illustrated. Specifically, the CT system 600 is configured to image a subject 612 (such as a patient, inanimate object, one or more manufactured parts) and / or foreign objects (such as dental implants, stents, and / or contrast agents present in the body). In one embodiment, the CT system 600 includes a gantry 602, which may further include at least one X-ray radiation source 604 configured to project an X-ray radiation beam 606 for imaging the subject 612. Specifically, the X-ray radiation source 604 is configured to project X-rays 606 toward a detector array 608 positioned on opposite sides of the gantry 602. Although Figure 6 Only a single X-ray radiation source 604 is depicted, but in some embodiments, multiple radiation sources may be used to project multiple X-rays 606 to collect projection data corresponding to the subject 612 at different energy levels.

[0082] Although CT systems have been described by way of example, it should be understood that the techniques of the present invention can also be useful when applied to images acquired using other imaging modalities, such as tomographic X-ray imaging, MRI, C-arm angiography, etc. The present invention discussion of CT imaging modalities provides only one example of a suitable imaging modality.

[0083] In some implementations, the CT system 600 is communicatively coupled to an edge computing system (ECS) 610, which is configured to process projection data using deep learning (DL) technology. For example, as discussed above, the CT system 600 may transmit projection data to the ECS 610 as it is acquired, and the ECS 610 may then process the projection data to provide decision support along with images reconstructed from the projection data.

[0084] In some known CT imaging system configurations, the radiation source projects a fan-shaped beam, which is collimated to lie in the XY plane of the Cartesian coordinate system and is often referred to as the "imaging plane." The radiation beam passes through the object being imaged, such as a patient or subject.612 The beam, after being attenuated by the object, strikes an array of radiation detectors. The intensity of the attenuated radiation beam received at the detector array depends on the attenuation of the beam by the object. Each detector element in the array generates a separate electrical signal, which is a measure of the beam attenuation at the detector location. Attenuation measurements from all detectors are acquired individually to produce a transmission distribution.

[0085] In some CT systems, a gantry is used to position the radiation source and detector array within the imaging plane and rotate it around the object to be imaged, causing the angle at which the radiation beam intersects the object to continuously change. A set of radiation attenuation measurements (i.e., projection data) from the detector array at a single gantry angle is referred to as a “view.” A “scan” of the object comprises a set of views acquired at different gantry angles or viewpoints during a single rotation of the radiation source and detector. It is conceivable that the benefits of the methods described herein extend to medical imaging modalities beyond CT; therefore, as used herein, the term “view” is not limited to the uses described above with respect to projection data from a single gantry angle. The term “view” is used to mean a data acquisition whenever multiple data acquisitions from different angles (whether from CT, PET, or SPECT acquisitions) are present, and / or any other modality (including modalities yet to be developed) and their combinations in fusion implementations.

[0086] In axial scanning, the projected data is processed to reconstruct an image corresponding to a two-dimensional slice taken through the object. One method for reconstructing an image from a set of projected data is known in the art as filtered backprojection (FBP). Transmission and emission tomography reconstruction techniques also include statistical iterative methods such as maximum likelihood expectation maximization (MLEM) and ordered subset expectation reconstruction techniques, as well as iterative reconstruction techniques. This method converts attenuation measurements from the scan into an integer called the “CT number” or “Hunsfield unit,” which is used to control the brightness of the corresponding pixel on the display device.

[0087] To reduce overall scan time, a "spiral" scan can be performed. To perform a spiral scan, the patient is moved while acquiring data from a predetermined number of slices. Such systems generate a single spiral from a cone-beam spiral scan. The spiral, marked by the cone beam, produces projection data from which an image in each predetermined slice can be reconstructed.

[0088] As used herein, the phrase "reconstructed image" is not intended to exclude embodiments of this disclosure in which data representing an image is generated rather than a visual image. Therefore, as used herein, the term "image" broadly refers to both a visual image and the data representing a visual image. However, many embodiments generate (or are configured to generate) at least one visual image.

[0089] Additionally or alternatively, the CT system 600 can be communicatively coupled to the "cloud" network 620 via communication with the ECS 610.

[0090] Figure 7 It shows something similar to Figure 6 An exemplary imaging system 700 of the CT system 600. The imaging system 700 may include the features described above. Figure 1The imaging system 101 is described above. In one embodiment, the imaging system 700 includes a detector array 608 (see [link to image]). Figure 6 The detector array 608 also includes a plurality of detector elements 702 that together sense an X-ray beam 606 passing through a subject 704 (such as a patient) (see [link to relevant documentation]). Figure 6 To acquire the corresponding projection data. Therefore, in one embodiment, the detector array 608 is fabricated in a multi-slice configuration including multiple rows of cells or detector elements 702. In such a configuration, one or more additional rows of detector elements 702 are arranged in parallel for acquiring projection data.

[0091] In some embodiments, the imaging system 700 is configured to traverse different angular positions around the subject 704 to acquire desired projection data. Therefore, the gantry 602 and the components mounted thereon can be configured to rotate about a center of rotation 706 to acquire projection data, for example, at different energy levels. Alternatively, in embodiments where the projection angle relative to the subject 704 varies over time, the mounted components can be configured to move along a generally arcuate path rather than along a circumference.

[0092] As the X-ray radiation source 604 and detector array 608 rotate, detector array 608 collects data from the attenuated X-ray beam. The data collected by detector array 608 undergoes preprocessing and calibration to adjust the data to represent the line integral of the attenuation coefficient of the scanned subject 704. The processed data is typically referred to as the projection.

[0093] In dual-energy or multi-energy imaging, two or more sets of projection data of the imaging object are typically obtained using an energy-resolved detector array 608 at different tube peak kilovolt (kVp) levels (which change the peak and spectrum of incident photons including the emitted X-ray beam) or alternatively at a single tube kVp level or spectrum.

[0094] The acquired projection dataset can be used for Base Material Decomposition (BMD). During BMD, the measured projections are converted into a set of density line integral projections. The density line integral projections can be reconstructed to form density maps or density images for each corresponding base material (such as mappings of bone, soft tissue, and / or contrast agents). The density maps or density images can then be correlated to form a volumetric rendering of the base material (e.g., bone, soft tissue, and / or contrast agent) within the imaging volume.

[0095] Once reconstructed, the underlying material image generated by the imaging system 700 reveals the internal features of the subject 704, expressed in terms of the density of the two underlying materials. Density images can be displayed to illustrate these features. In conventional methods of diagnosing medical conditions (such as disease states) and more generally medical events, radiologists or physicians will consider a hard copy or display of the density image to identify features of interest. Such features may include lesions, size, and shape of specific anatomical structures or organs, as well as other structural features that will be discernible in the image based on the individual physician's skill and knowledge.

[0096] In one embodiment, the imaging system 700 includes a control mechanism 608 to control the movement of components, such as the rotation of the gantry 602 and the operation of the X-ray radiation source 604. In some embodiments, the control mechanism 708 further includes an X-ray controller 710 configured to provide power and timing signals to the X-ray radiation source 604. Additionally, the control mechanism 708 includes a gantry motor controller 712 configured to control the rotational speed and / or position of the gantry 602 based on imaging requirements.

[0097] In some embodiments, control unit 708 also includes a data acquisition system (DAS) 714 configured to sample analog data received from detector element 702 and convert the analog data into digital signals for subsequent processing. The data sampled and digitized by DAS 714 is transferred to a computer or computing device 716. In one example, computing device 716 stores the data in a storage device 718, such as a mass storage device. For example, mass storage device 718 may include hard disk drives, floppy disk drives, optical disc read / write (CD-R / W) drives, digital versatile optical disc (DVD) drives, flash memory drives, and / or solid-state storage drives.

[0098] Additionally, the computing device 716 provides commands and parameters to one or more of the DAS 714, X-ray controller 710, and rack motor controller 712 for system operation, such as data acquisition and / or processing. In some embodiments, the computing device 716 controls system operation based on operator input. The computing device 716 receives operator input, such as commands and / or scan parameters, via an operator console 720 operably coupled to the computing device 716. The operator console 720 may include a keyboard (not shown) and / or a touchscreen to allow the operator to specify commands and / or scan parameters.

[0099] Although Figure 7Only one operator console 720 is shown, but more than one operator console may be coupled to the imaging system 700, for example, to input or output system parameters, request inspections, and / or view images. Furthermore, in some embodiments, the imaging system 700 may be coupled via one or more configurable wired and / or wireless networks (such as the Internet and / or VPNs) to multiple monitors, printers, workstations, and / or similar devices, located locally or remotely, either within an institution or hospital or in entirely different locations.

[0100] In one implementation, for example, the imaging system 700 includes or is coupled to a Picture Archiving and Communication System (PACS) 724. In an exemplary implementation, the PACS 724 is further coupled to a remote system (such as a radiology information system, a hospital information system) and / or coupled to an internal or external network (not shown) to allow operators in different locations to supply commands and parameters and / or obtain access to image data.

[0101] The computing device 716 operates the table motor controller 726 using operator-supplied and / or system-defined commands and parameters, which in turn controls the table 728, which may include a motorized table. Specifically, the table motor controller 726 moves the table 728 to properly position the subject 704 within the frame 602 to acquire projection data corresponding to the target volume of the subject 704.

[0102] As previously described, the DAS 714 samples and digitizes the projection data acquired by detector element 702. Subsequently, the image reconstructor 730 uses the sampled and digitized X-ray data to perform high-speed reconstruction. Although Figure 7 Image reconstructor 730 is shown as a separate entity; however, in some embodiments, image reconstructor 730 may be part of computing device 716. Alternatively, image reconstructor 730 may not be present in imaging system 700, and alternatively, computing device 716 may perform one or more functions of image reconstructor 730. Furthermore, image reconstructor 730 may be located locally or remotely and may be operatively connected to imaging system 700 using wired or wireless networks. Specifically, one exemplary embodiment may use computing resources in a "cloud" network cluster for image reconstructor 730.

[0103] In one embodiment, image reconstructor 730 stores the reconstructed image in a storage device or mass storage device 718. Alternatively, image reconstructor 730 transmits the reconstructed image to computing device 716 to generate usable patient information for diagnosis and evaluation. In some embodiments, computing device 716 transmits the reconstructed image and / or patient information to display 732, which is communicatively coupled to computing device 716 and / or image reconstructor 730.

[0104] The various methods and processes further described herein may be stored as executable instructions in non-transitory memory on a computing device within the imaging system 700. For example, the image reconstructor 730 may include such executable instructions in non-transitory memory and may apply the methods described herein to reconstruct an image from scan data. In another embodiment, the computing device 716 may include instructions in non-transitory memory and may apply the methods described herein at least partially to the reconstructed image after receiving it from the image reconstructor 730. In yet another embodiment, the methods and processes described herein may be distributed throughout the image reconstructor 730 and the computing system 716.

[0105] In one embodiment, the display 732 allows an operator to evaluate the anatomical structures of the image. The display 732 may also allow the operator, for example via a graphical user interface (GUI), to select the volume of interest (VOI) and / or request patient information for subsequent scanning or processing.

[0106] The technical advantage of this disclosure is that it transmits imaging data to an external computing system in parallel with the acquisition of imaging data for processing. Another advantage is that it pauses the transmission of imaging data to the external computing system when the imaging system is in a critical state. Yet another advantage is the ability to display reconstructed images and automatically generated decision support.

[0107] In one embodiment, the system includes an imaging system comprising at least a scanner and a processor configured to reconstruct images from data acquired during scanning of a subject via the scanner; and a computing device communicatively coupled to the imaging system and located externally to the imaging system, the computing device being configured to generate decision support computations based on the data, wherein the imaging system transmits data to the computing device during scanning.

[0108] In a first example of the system, the imaging system also includes a user interface, wherein a scanning protocol for scanning is determined based on a primary task input to the processor via the user interface. In a second example of the system, optionally including the first example, the processor transfers secondary tasks based on the primary task to a computing device, and the computing device generates decision support computations based on the secondary tasks. In a third example of the system, optionally including one or more of the first and second examples, the imaging system also includes a display device, wherein the processor is configured to display images and decision support computations via the display device. In a fourth example of the system, optionally including one or more of the first to third examples, the imaging system transfers data only during scanning when the imaging system is not in a critical state. In a fifth example of the system, optionally including one or more of the first to fourth examples, the computing device includes a deep learning model in non-transitory memory, wherein the computing device inputs data to the deep learning model to generate decision support computations.

[0109] In another embodiment, the method for an imaging system includes performing a scan of a subject to acquire imaging data, transmitting the imaging data during the scan to a computing device communicatively coupled to and located outside the imaging system, receiving decision support output from the computing device, the decision support output being calculated by the computing device from the imaging data, and displaying an image reconstructed from the imaging data and the decision support output.

[0110] In a first example of the method, the method further includes evaluating the state of the imaging system during scanning, wherein transmitting imaging data to the computing device includes transmitting imaging data to the computing device when the state of the imaging system is not critical, and not transmitting imaging data to the computing device when the state of the imaging system is critical. In a second example of the method, optionally including the first example, the method further includes receiving an instruction for a primary task and transmitting instructions for secondary tasks related to the primary task to the computing system. In a third example of the method, optionally including one or more of the first and second examples, the method further includes determining a scanning protocol for scanning based on the primary task, wherein the computing system computes decision support output based on the secondary tasks. In a fourth example of the method, optionally including one or more of the first to third examples, the computing system computes decision support output during scanning. In a fifth example of the method, optionally including one or more of the first to fourth examples, displaying the image and decision support output includes displaying decision support output superimposed on the image. In a sixth example of the method, optionally including one or more of the first to fifth examples, the computing device computes decision support output by inputting imaging data into a deep learning model, which outputs decision support output. In a seventh example, which optionally includes one or more of the first to sixth examples, the method further includes receiving underlying facts about the decision support output via a user interface of the imaging system, and transmitting the underlying facts to a computing device to update the deep learning model.

[0111] In another embodiment, the imaging system includes an X-ray source that emits an X-ray beam toward a subject to be imaged; a detector that receives X-rays attenuated by the subject; a data acquisition system (DAS) operatively connected to the detector; and a computing device operatively connected to the DAS and configured with executable instructions in a non-transitory memory, which, when executed, cause the computing device to: control the X-ray source to perform a scan of the subject; during the scan, receive projection data via the DAS and transmit the projection data to an edge computing system (ECS) communicatively coupled to and located outside the imaging system; receive a decision support output generated by the ECS based on the projection data; and output the decision support output and an image reconstructed from the projection data.

[0112] In a first example of the imaging system, the imaging system further includes a display device, to which decision support output and an image are output. In a second example of the imaging system optionally including the first example, the computing device is further configured with executable instructions in non-transitory memory that, when executed, cause the computing device to reconstruct an image from the projection data. In a third example of the imaging system optionally including one or more of the first and second examples, the computing device is further configured with executable instructions in non-transitory memory that, when executed, cause the computing device to receive an image reconstructed by the ECS from the projection data. In a fourth example of the imaging system optionally including one or more of the first to third examples, the imaging system further includes an operator console coupled to the computing device and configured to receive input from an operator of the imaging system, wherein the computing device is further configured with executable instructions in non-transitory memory that, when executed, cause the computing device to receive instructions for a primary task via the operator console and select a scanning protocol for scanning based on the primary task. In a fifth example, which optionally includes one or more of the first to fourth examples of the imaging system, the computing device is further configured with executable instructions in non-transitory memory that, when executed, cause the computing device to output instructions for a secondary task to the ECS, the secondary task being associated with the primary task, wherein the decision support output is generated by the ECS based on the secondary task.

[0113] Figure 8 This illustrates a ratio of the capabilities of one embodiment for expanding an imaging system or other device 802. Figure 1 A more detailed block diagram of an exemplary system 800 is provided. System 800 may include multiple imaging systems 802, such as any suitable non-invasive imaging system, including but not limited to computed tomography (CT) imaging systems, positron emission tomography (PET) imaging systems, magnetic resonance (MR) imaging systems, ultrasound systems, and combinations thereof (e.g., multimodal imaging systems such as PET / CT or PET / MR imaging systems), or other devices 802, such as advanced workstations (AW), PACS, and mobile client devices for visualizing images, such as tablets, iPads, smartphones, iPhones, etc. The multiple imaging systems and other devices 802 may be communicatively coupled to an edge computing system 804 via a network. The multiple imaging systems and other devices 802, as well as the edge computing system 804, may be communicatively coupled to a cloud network 806. Communication between the multiple imaging systems and other devices 802, ECS 804, and cloud network 806 is secure.

[0114] ECS 804 may also be referred to as an on-premises cloud and may include data 808 from various inputs and sources (including EMR, HIS / RIS data, overall data, etc.), which is communicatively coupled to DL interface 810 and DL training 812. ECS 804 may also include at least one edge host 814, which may include at least one edge platform 816 and an application store 818 with multiple applications 820.

[0115] Cloud network 806 may include data 822 from various inputs and sources, as well as data from DL training 824. Additionally or alternatively, cloud network 806 may include an application store with multiple applications 826. Cloud network 806 may also include an artificial intelligence (AI) interface for generating learning models or other models that can be used with system 800.

[0116] Data from system 800 can be shared and / or stored across multiple imaging systems and other devices 802, ECS 804, and cloud network 806. Post-processing can be performed on data sent to ECS before being sent to DICOM. Quantization and segmentation can be performed on the imaging system before being sent to PACS.

[0117] Some examples offer kernel processing capabilities organized into units or modules that can be deployed in multiple locations. Off-device processing can be leveraged to provide micro-clouds, mini-clouds, and / or global clouds. For example, a micro-cloud provides a one-to-one configuration with an imaging equipment console, offering ultra-low latency processing (e.g., outlining) for customers without cloud connectivity. A mini-cloud is deployed on a customer's network, offering low-latency processing for customers who, for example, prefer to keep their data internal. A global cloud is deployed throughout the customer's organization to enable high-performance computing and management of operationally superior IT infrastructure.

[0118] In some other examples, one or more off-device processing engines (e.g., acquisition engines, reconstruction engines, diagnostic engines, etc., and their associated deployed deep learning network devices, etc.) may be included in system 800. For example, examination purpose, electronic medical record information, heart rate and / or heart rate variability, blood pressure, weight, prone / supine, head-in or foot-in visual assessment, etc., can be used to determine one or more acquisition settings, such as default field of view (DFOV), center, spacing, orientation, contrast agent injection rate, contrast agent injection timing, voltage, current, etc., thereby providing a “one-click” imaging device. Similarly, for example, kernel information, slice thickness, slice spacing, etc., can be used to determine one or more reconstruction parameters, including image quality feedback. Acquisition feedback, reconstruction feedback, etc., can be provided to the system design engine to provide real-time (or substantially real-time (considering processing and / or transmission delays)) health analysis for the imaging device, such as represented by one or more digital models (e.g., deep learning models, machine models, digital twins, etc.). One or more digital models can be used to predict the component health status of the imaging device in real-time (or substantially real-time (considering processing and / or transmission delays)).

[0119] Figure 9 A flowchart illustrating a method 900 for generating decision support outputs using one or more deep learning (DL) applications is shown. (About...) Figure 1 The method 900 describes the system and components described herein, but it should be understood that the method can be implemented with other systems and components without departing from the scope of this disclosure. The method 900 can be implemented as executable instructions in the non-transitory memory 115 of the ECS 110 and can be executed by one or more of the plurality of processors 113 of the ECS 110.

[0120] At point 905, the examination type and secondary task are received for decision support output related to the imaging scan. As explained above, when performing an imaging scan to diagnose a patient's condition, the scanner operator can input the initial task or primary task (as described above regarding...). Figure 2 (as explained above), or the initial / primary task can be obtained from the patient's EMR (as mentioned above). Figure 4 (As explained). The initial / primary task may include an indication of the examination type and may include associated secondary tasks. The examination type may include the target anatomical structure to be scanned and certain features of the scan used to perform the examination (such as examination for brain hemorrhage, examination for liver lesions, etc.). Associated secondary tasks may indicate decision support that will be generated and output along with the images acquired during the imaging scan. Decision support may include hemorrhage detection, lesion detection, organ segmentation, etc.

[0121] At point 910, select the application to be executed to generate decision support output. The selected application can be a deep learning application (DL application) that utilizes imaging information sent from the scanner (e.g., as described above regarding...). Figure 2 The information (as explained) serves as input to the DL model. The DL model then uses the DL model and the input imaging information to generate decision support output. The DL application can be selected based on the secondary task and examination type. For example, if the examination is a brain examination to detect the presence of hemorrhage, a brain hemorrhage DL application can be selected. If the examination is a liver scan to detect the presence of lesions, a liver lesion DL application can be selected. In some examples, the same DL application can be applied to more than one examination type. For example, a lesion detection DL application can be selected for both liver lesion examination and kidney lesion detection. In other examples, a separate DL application can be selected for each different examination type / secondary task combination. In such examples, different DL applications can be selected for liver and kidney lesion detection examinations.

[0122] Appropriate DL applications can be selected using suitable mechanisms. In some implementations, the ECS may store a lookup table in memory that indexes DL applications based on examination type and secondary task. Other mechanisms for selecting DL applications are possible. For example, instead of the ECS selecting the appropriate DL application, the operator may select the desired DL application and send the indication of the selected DL application, along with the examination type and secondary task, to the ECS. In some implementations, the selection of a particular DL application may automatically trigger a sequence of other DL applications to be triggered for preprocessing or postprocessing of data for the primary task. For example, selecting a DL application for liver lesion detection may trigger an anatomical structure localization DL application, which may be executed to reduce the acquired images to include only the liver.

[0123] At point 915, a request for additional input may optionally be sent. Additional input may include subject and / or scanner metadata or other information that can be used by the selected DL application to generate decision support output. Therefore, a request for additional input may be sent to an EMR, PACS, or other database that stores the additional input. For example, as described above regarding... Figure 3In addition to the images / imaging data acquired during the patient's imaging scan, the DL application may utilize various information as input to the corresponding DL model, including but not limited to patient information (e.g., demographic data, medical history) and scanner information. Scanner information may include the type of scanner (such as a CT scanner or ultrasound), imaging category / type (such as contrast-enhanced imaging, non-contrast-enhanced imaging, Doppler, B-mode), scanner settings (such as tube current and tube voltage), etc. Additional input may also include the patient's laboratory test results, previous diagnostic imaging scan results, and complementary images from other imaging modalities. In some embodiments, the additional input requested and / or used by the selected DL application may be based on the examination type and secondary tasks. For example, the ECS may store a lookup table that stores additional inputs requested based on the examination type and secondary tasks. As a non-limiting example, if the examination type is a non-contrast head CT scan to be analyzed by the brain hemorrhage detection DL application, the requested additional input may include laboratory reports, neurophysiological examination reports, the X-ray tube kVp used during the scan, and / or the type of CT scan. As another non-limiting example, if the examination type is a multiphasic liver study using contrast agents administered at different timings to be analyzed by the liver lesion detection module, additional inputs may include the scan type and X-ray tube kVp used during the scan, contrast agent timing, volume of contrast agent injected, cardiac ejection fraction, and / or previous CT studies for the patient.

[0124] At position 920, receive images and / or imaging data from the scanner. (As mentioned above...) Figure 2 and Figure 3As explained above, the scanner may send the reconstructed images and / or raw imaging data to the ECS during and / or after the imaging scan. At 925, a decision support output is generated using the selected DL application. To generate the decision support output, the selected DL application may input the received images / image data, along with any additional information specified by the DL application, into a DL model executed by the DL application. The DL model can then output decision support. As explained above, decision support may include organ segmentation, anatomical structure identification, indications of whether bleeding, lesions, fractures, etc., are detected. At 930, the decision support output is sent to the requesting device, such as the scanner that sent the imaging information, and / or other devices, such as PACS, care provider devices, or other suitable devices. As explained above, the decision support output may be displayed together with the reconstructed images from the imaging scan or otherwise presented. The decision support output may support the diagnosis or determination of clinical findings via the reconstructed images. The decision support output and the reconstructed images may be analyzed by one or more clinicians (such as radiologists). One or more clinicians may accept or reject the decision support output. For example, if the decision support output includes indications of lesions on the subject's liver, the clinician can agree to the lesion detection and accept the output, or disagree that the identified lesions are indeed lesions and reject the output. In an example where the decision support output includes organ segmentation, the clinician can edit or update the contours defining the organs, where the contours are generated by the DL application. Furthermore, the clinician can generate a final report with detailed and (if indicated) edited findings (e.g., updated contours, rejected decision support, etc.). The final report can be stored in the subject's EMR.

[0125] At 935, receiving (e.g., from an EMR database) may include a final report with edited results. The final report may be analyzed by an ECS (e.g., by the ECS's training data aggregator module) to identify the inspection type and DL application used to generate the decision support included in the report, which may be tagged / stored along with the final report. At 940, the tagged report (e.g., the final report, including edited results where appropriate, and tagged with inspection type and DL application) is placed in training, testing, and / or validation datasets. Each dataset may be stored on an ECS. Furthermore, each dataset may be specific to a DL application. For example, each DL application may have a specific training dataset, testing dataset, and validation dataset. Thus, tagged reports may be placed in datasets based on the DL application used to generate the decision support listed in the report. At least in some embodiments, within a dataset used for a specific DL application, tagged reports may be randomly or semi-randomly assigned to one of the training, testing, and / or validation datasets.

[0126] At 945, method 900 includes determining whether the training dataset includes a threshold number of reports. The threshold number of reports can be a number sufficient to accurately reflect the functionality of the DL model executed by the DL application, such as 100 reports. Therefore, if the training dataset used for a given DL application reaches 100 reports, the answer at 945 includes "yes"; otherwise, the answer at 945 includes "no". If the answer at 945 is "no", method 900 returns. If the answer is "yes", method 900 proceeds to 950 to retrain the DL application using the new data in the dataset. In this way, the DL model can be fine-tuned by retraining with new data. The updated weights can be deployed locally for site-tuned models. For example, a DL model executed by a DL application on an ECS may include weighted connections, decision trees, etc., and the DL model can be retrained so that the weights are updated to better fit the new data. As more data is collected, the DL model can become more accurate. By retraining the DL model locally (e.g., on an ECS) rather than globally via a central device such as the cloud, site-specific preferences for deploying DL models can be maintained. For example, a particular medical facility might prefer an aggressive lesion detection approach to ensure fewer false negatives, while different facilities might prefer a more conservative approach to reduce false positives. Such preferences can be established and maintained by allowing local retraining of the appropriate DL model. Furthermore, local retraining of the DL model can be used to tailor the model to specific patient demographics in certain regions of the world. For example, if the majority of patients in a hospital are obese, the DL model might need to be tuned to the imaging characteristics of that patient demographic. However, in at least some examples, a subset or all of the data and / or updated weights can be sent, along with data from other sites, to a central device (e.g., the cloud) to create an updated global model, as shown at 955. Data sent back to the cloud can include images and other specific datasets that can be used to train a particular model. The global model can also be sent back to the locally stored DL application model.

[0127] As used herein, elements or steps listed in the singular and beginning with the word "a" or "an" should be understood to not exclude a plurality of said elements or steps unless such exclusion is explicitly stated. Furthermore, references to "an embodiment" of the invention are not intended to be construed as excluding the existence of additional embodiments that also include the referenced features. Moreover, unless explicitly stated to the contrary, embodiments that "comprise," "include," or "have" elements or multiple elements having a particular characteristic may include additional such elements that do not have that characteristic. The terms "comprise" and "in..." are used as concise linguistic equivalents to the corresponding terms "comprising" and "wherein". Furthermore, the terms "first," "second," and "third," etc., are used merely as notations and are not intended to impose numerical requirements or a particular order of position on their objects.

[0128] This written description uses examples to disclose the invention, including the best mode, and also enables those skilled in the art to practice the invention, including making and using any device or system and performing any included methods. The scope of patentability of the invention is defined by the claims and may include other examples that would occur to those skilled in the art. Such other examples are intended to fall within the scope of the claims if they have structural elements that are not indistinguishable from the literal language of the claims, or if they include equivalent structural elements that have minor differences from the literal language of the claims.

Claims

1. A system comprising: An imaging system comprising at least a scanner and a processor, wherein the scanner is configured to scan a subject to acquire imaging data, and the processor is configured to reconstruct an image from the imaging data acquired during a scan of the subject via the scanner; and A computing device, communicatively coupled to the imaging system and located outside the imaging system, is provided. Specifically, the imaging system transmits the imaging data to the computing device during the scanning process only when the imaging system is not in a critical state. The computing device is configured to apply post-processing algorithms to the imaging data while the imaging system is performing a scan to generate decision support computation, wherein the decision support computation includes clinically diagnostic results obtained by one or more post-processing algorithms applied to the imaging data.

2. The system of claim 1, wherein the imaging system further includes a user interface, wherein a scanning protocol for the scan is determined based on a primary task input to the processor via the user interface.

3. The system of claim 2, wherein the processor transmits secondary tasks based on the primary task to the computing device, and wherein the computing device generates the decision support computation based on the secondary tasks.

4. The system of claim 1, wherein the imaging system further comprises a display device, wherein the processor is configured to display the image and the decision support calculation via the display device.

5. The system of claim 1, wherein the computing device includes a deep learning model in a non-transitory memory, wherein the computing device inputs the imaging data into the deep learning model to generate the decision support computation.

6. A method for an imaging system, comprising: Perform a scan of the subject to acquire imaging data; During the scan, the state of the imaging system is evaluated; During the scan, when the state of the imaging system is not critical, the imaging data is transmitted to a computing device to allow the computing device to apply post-processing algorithms to the imaging data to generate decision support output while the imaging system is performing the scan, wherein the decision support output includes clinically relevant results obtained by one or more post-processing algorithms applied to the imaging data; when the state of the imaging system is critical, the imaging data is not transmitted to the computing device, wherein the computing device is communicatively coupled to the imaging system and located outside the imaging system. The decision support output is received from the computing device, the decision support output being calculated by the computing device from the imaging data; as well as The image reconstructed from the imaging data and the decision support output are displayed.

7. The method of claim 6, further comprising receiving an instruction for a primary task and transmitting an instruction for a secondary task related to the primary task to the computing device.

8. The method of claim 7, further comprising determining a scanning protocol for the scan based on the primary task, wherein the computing device computes the decision support output based on the secondary task.

9. The method of claim 6, wherein the computing device computes the decision support output during the scan.

10. The method of claim 6, wherein displaying the image and the decision support output includes displaying the decision support output superimposed on the image.

11. The method of claim 6, wherein the computing device computes the decision support output by inputting the imaging data into a deep learning model, and the deep learning model outputs the decision support output.

12. The method of claim 11, further comprising receiving basic facts about the decision support output via a user interface of the imaging system, and transmitting the basic facts to the computing device to update the deep learning model.

13. An imaging system, comprising: An X-ray source that emits an X-ray beam toward the subject to be imaged; A detector that receives the X-rays attenuated by the subject; A data acquisition system (DAS) is operatively connected to the detector; and A computing device, operatively connected to the DAS and configured with executable instructions in non-transitory memory, which, when executed, cause the computing device to: Control the X-ray source to perform a scan of the subject; During the scan, projection data is received via the DAS and transmitted to the edge computing system (ECS) only when the imaging system is not in a critical state. This allows the ECS to apply post-processing algorithms to the projection data to generate decision support outputs while the imaging system is performing the scan. The decision support outputs include clinically relevant results obtained by one or more post-processing algorithms applied to the imaging data. The ECS is communicatively coupled to the imaging system and located outside the imaging system. Receive the decision support output generated by the ECS based on the projected data; as well as Output the decision support output and the image reconstructed from the projected data.

14. The imaging system of claim 13, further comprising a display device, wherein the decision support output and the image are output to the display device.

15. The imaging system of claim 13, wherein the computing device is further configured with executable instructions in a non-transitory memory, the executable instructions, when executed, causing the computing device to reconstruct the image from the projection data.

16. The imaging system of claim 13, wherein the computing device is further configured with executable instructions in a non-transitory memory, the executable instructions, when executed, causing the computing device to receive from the ECS the image reconstructed by the ECS from the projection data.

17. The imaging system of claim 13, further comprising an operator console coupled to the computing device and configured to receive input from an operator of the imaging system, wherein the computing device is further configured with executable instructions in non-transitory memory, which, when executed, cause the computing device to receive instructions for a primary task via the operator console and select a scanning protocol for the scan according to the primary task.

18. The imaging system of claim 17, wherein the computing device is further configured with executable instructions in a non-transitory memory, the executable instructions, when executed, causing the computing device to output an indication of a secondary task to the ECS, the secondary task being associated with the primary task, wherein the decision support output is generated by the ECS based on the secondary task.

Images