Combination of Imaging and Reporting in Medical Imaging

Abstract

The present invention relates to the combination of imaging and reporting in medical imaging. Since the ultimate output of medical imaging is a radiology report, the quality of which depends to a large extent on the radiologist, there is a need for an integrated system for both medical imaging and reporting. Combine imaging and radiology reporting. Combine image acquisition, image reading, and reporting, thereby allowing feedback on the reading to control the acquisition, such that the final report is more comprehensive. Clinical findings that are normally associated with the report can be automatically used for feedback for further or continuous acquisition without the radiologist. Clinical identification can be used to determine what image processing to perform for reading, and / or to provide raw (i.e., non-reconstructed) scan data from the imaging system for image processing integrated with report generation.

Description

Technical Field

[0001] This embodiment relates to medical imaging and reporting from imaging. Background Art

[0002] Medical reports, such as radiology reports, are primarily written communication between radiologists, medical professionals, and patients. These reports typically contain complex anatomical, technical, and medical information, including images, measurements, and textual notations (e.g., key findings) that provide an analysis or summary of the pathology represented in the images. Their comprehensive interpretation is of great value for the diagnosis, prognosis, and treatment of diseases.

[0003] Currently, the processes of image acquisition and image reporting are separate. The role of a medical imaging scanner is mainly to produce high-quality diagnostic images, while the role of a radiologist is to interpret those images and produce a radiology report with key findings. Due to the focus on value-based care and increased productivity, there is a greater emphasis on quantifying and improving the "value" of the overall imaging examination. In recent years, there have been significant developments in image acquisition to standardize image protocols for consistent high-quality images across multiple clinical sites. This focus has not addressed inefficiencies in reporting or inconsistencies in reports. Summary of the Invention

[0004] As an introduction, the preferred embodiments described below include methods, systems, instructions, and computer-readable media for combined imaging and reporting. Since the final output is a radiology report, the quality of which depends largely on the radiologist, there is a greater need for an integrated system for both medical imaging and reporting. Combine imaging and radiology reporting. Combine image acquisition, image reading, and reporting, thereby allowing feedback on the reading to control the acquisition, such that the final report is more comprehensive. Clinical findings typically associated with the report can be automatically used for feedback for further or continuous acquisition without the radiologist. Clinical identification can be used to determine what image processing to perform for reading, and / or provide raw (i.e., non-reconstructed) scan data from the imaging system for image processing integrated with report generation.

[0005] In a first aspect, a method for imaging and generating a radiology report in a medical system is provided. A medical imaging scanner scans a patient. An image processor receives the clinical identification of the patient. Image processing is selected based on the clinical identification. The image processor performs image processing on first scan data from the scan based on the selected image processing. Information is fed back to the medical imaging scanner based on the image processing. The medical imaging scanner re-scans based on the feedback information. The image processor generates the radiology report in response to the first scan data, the feedback information, and / or second scan data from the re-scan. The radiology report has narrative text characterizing the patient, and the radiology report is outputted.

[0006] In a second aspect, a system for imaging and generating a radiology report is provided. A medical imager is configured to scan a patient. The configuration is for clinical applications. A processor is configured to receive scan data from the medical imager, apply image processing to the scan data, determine clinical findings from the image processing, control the medical imager based on the clinical findings, and generate the radiology report from the clinical findings. An interface is configured to output the radiology report.

[0007] In a third aspect, a method for imaging and generating a radiology report in a medical system is provided. First and second scan data are received from first and second medical scanners. The first and second scan data represent first and second patients and are not reconstructed into a three-dimensional object space. First and second clinical indication labels are obtained for the first and second scan data. Image processing of the first and second scan data is performed, including reconstruction. Based on the first and second clinical indication labels being different, the image processing of the first scan data is different from the image processing of the second scan data. First and second radiology reports are generated from the information provided by the image processing. The first and second radiology reports are outputted. Since the acquisition and reading are combined to generate the report, the report can be generated without displaying any images from the first and second scan data to a person after receiving, passing through image processing, and generating.

[0008] The invention is defined by the following claims, and any content in this section should not be regarded as a limitation to those claims. Other aspects and advantages of the invention are discussed in conjunction with the preferred embodiments below, and they can be claimed separately or in combination later. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] The components and the drawings are not necessarily drawn to scale, but rather the emphasis is placed on illustrating the principles of the invention. Further, in the drawings, the same reference numerals designate corresponding parts throughout the different views.

[0010] Figure 1is an embodiment of a system for performing imaging and generating a medical imaging examination report;

[0011] Figure 2 is another embodiment of a system for performing imaging and generating a medical radiology report;

[0012] Figure 3 illustrates an example interaction for generating a medical imaging examination report; and

[0013] Figure 4 is a flowchart of an embodiment of a method for performing imaging and generating a radiology report in a medical system. Detailed Description

[0014] A comprehensive medical imaging and reporting device or process scans a subject and produces a clinical report as a final output. As Figure 1 shown, the comprehensive medical imaging and reporting device includes a medical imaging device 22 and a fulfillment center or processor 24, and the fulfillment center or processor 24 is used to interact to generate a medical imaging examination report 26. Compared with a typical medical imaging scanner that produces medical images as its final product, this comprehensive device produces a medical imaging examination report that describes the main findings in the imaging examination of a patient. The fulfillment center 24 is a medical image reading and reporting infrastructure. Different from a remote radiology operation independent of imaging, the fulfillment center 24 is specifically configured to operate in a manner inherently interconnected with the medical imaging device 22. Feedback can be used to control imaging based on ongoing clinical findings from imaging. Reconstruction can be performed using feedforward by the fulfillment center 24 instead of the medical imaging device 22, thereby allowing reconstruction based on clinical findings. The fulfillment center 24 can use clinical identifications from the medical imaging device 22 to select image processing as iterative or interactive imaging and continue report reading.

[0015] Figure 2 shows a block diagram of an embodiment of a system for imaging a patient and generating a radiology report. The system provides combined imaging, reading, and reporting. Due to the inherently interconnected arrangement between imaging and reading, (1) feedback can be used to control imaging due to clinical findings; (2) scan data can be provided for clinical identification for reading based on clinical identification instead of only performing image-based radiology reading; and / or (3) non-reconstructed data can be provided for reconstruction based on reading instead of reading from images without control over the reconstruction used. In one embodiment, the system implements Figure 4 the method and / or Figure 3 the arrangement.

[0016] The system includes one or more medical imagers 22 and a fulfillment center shown as a processor 24. Other parts of the system include an interface 27, a medical record database 29, and a display 28. Additional, different, or fewer components may be provided. For example, a user interface or input device may be provided on the medical imager 22 and / or for the processor 24. In another example, a network or network connection is provided, such as for networking different components (e.g., the medical imager 22 with the processor 24 and / or the processor 24 with the database 29).

[0017] The interface 27, the processor 24, and / or the display 28 are part of a server, a workstation, or a computer. In one embodiment, the interface 27, the processor 24, and / or the display 28 are a server or a workstation. The medical record database 29 is part of a computer separate from the processor 24, such as in a cloud-hosted electronic health record or electronic medical record system.

[0018] The medical imager 22 and the processor 24 are at different facilities, such as far from each other. The medical imager 22 and the processor 24 are located in different buildings, cities, states, or countries. For example, the processor 24 is connected to multiple medical imagers 22 via a high-speed computer network, so the processor 24 is a server or operates in the cloud. In an alternative embodiment, the processor 24 (i.e., the fulfillment center) may be located at the site with the medical imager 22, such as in the same building or facility, but in different rooms. In other alternative embodiments, the processor 24 is part of the medical imager 22 or in the same housing as the medical imager 22. Each medical imager 22 includes a separate processor 24, or one medical imager 22 implements the processor 24 for use by that imager 22 and other imagers 22.

[0019] In the case where the interconnection between the imager 22 and the processor 24 is disconnected, the imager 22 can continue to operate. Scan data (e.g., images) can be processed into images that can be visually interpreted and routed to a picture and archiving system for later reading and / or report generation. In the absence of interconnection and communication for combined imaging, reading, and reporting, the hybrid system operates according to traditional systems for generating images, reading images, and reporting.

[0020] The medical imager 22 is a magnetic resonance (MR), computed tomography (CT), X-ray, ultrasound, or nuclear medicine (e.g., positron emission tomography or single photon computed tomography) scanner. In other embodiments, the medical imager 22 is a multimodal device, such as a combination of nuclear medicine and X-ray or CT. In other embodiments, invasive, other non-invasive, or minimally invasive imaging systems are used.

[0021] The medical imager 22 is configured to scan a patient. The same imager 22 can be used to scan different patients at different times. Other imagers 22 can be used to scan other patients.

[0022] To scan a patient, the medical imager 22 is configured by settings or values of scan parameters. The same imager 22 can operate differently based on the settings. For example, voltage, collimator settings, pulse settings, bias, spatial position, speed, travel path, pulse sequence, timing, coils to be used, amplitude, focus, and / or other parameters control the physics for scanning the patient. Different modalities have different scan parameters. Other parameters control the subsequent processing of the acquired data, such as filtering, amplification, reconstruction, and / or detection.

[0023] Different settings are used for different clinical applications. For example, the field of view and / or focus position for a chest scan are different from those for a lower torso or whole body scan. For different applications, different settings are used for the parameters of the same modality. The application can be a scan type by region and / or by lesion (e.g., verification for a specific disease). A technician configures the medical imager 22 for a given patient based on the instructions of a consulting physician or the selection of a clinical application. This configuration can use a drop-down menu or other menu structures that allow selection of a clinical application and / or adjustment or input of one or more settings.

[0024] In some cases, a general clinical application is selected. For example, a patient complains of chest pain. A cardiac imaging application is selected. A later finding may be an arterial blockage, so a more specific clinical application for vascular imaging can be used later. Alternatively, a more specific clinical application is specified initially without confirmation of the expected disease, location, or severity based on imaging findings.

[0025] For combined reading and reporting, the medical imager 22 is configured to output a clinical identification or indication label to the processor 24. The clinical identification label can be a clinical application or an indication of a clinical application (e.g., the settings for the scan). In other embodiments, the clinical identification label is a symptom, a measurement from the patient, or other patient information indicating the reason for imaging. The medical imager 22 transmits the clinical identification label to the processor 24 for automatic generation of findings for the patient.

[0026] The medical imager 22 is configured to output scan data to the processor 24. The scan data is data obtained from scanning at any processing stage. For example, data without reconstruction is provided. For CT, the data can be detector measurements for multiple projections that have not been reconstructed into values for specific spatial locations. For MR, the data can be k-space data prior to Fourier transform for determining values for specific spatial locations. For nuclear imaging, the data can be line-of-response values prior to tomography for assigning specific spatial locations. As another example, reconstructed data is provided. Filtering, detection, scan conversion, and / or other image processing may or may not be applied to the data for transmission to the processor 24. The medical imager 22 provides image data (e.g., scan data) as data obtained from scanning, which has any amount of processing for generating an image. The image data can be formatted for display, such as RGB values, or can be in a scan format (e.g., scalar values).

[0027] The processor 24 of the fulfillment center is a general-purpose processor, a control processor, a digital signal processor, an application-specific integrated circuit, a field-programmable gate array, or other hardware processor for image processing. The processor 24 is part of a computer, a workstation, a server, or other device configured to apply image processing, derive patient-specific findings, and / or generate a radiology report 26. The processor 24 can be a network of computing devices, such as multiple computers or servers. The processor 24 is configured by software, hardware, and / or firmware.

[0028] The processor 24 is configured to interface with one or more medical imagers 22. The processor 24 is the second subunit of an integrated device for imaging and report generation. Raw and / or processed medical imaging data (e.g., scan data) is received to produce a clinical report 26 as an output. Other information, such as clinical indication tags and / or patient information, may also be received.

[0029] The processor 24 is configured to apply image processing to the scan data. Any type of image processing can be performed. For example, reconstruction, scan quality assurance, filtering, denoising, detection, segmentation, classification, quantification, and / or prediction are applied. The image processing provides imaging quality, patient-specific measurements or quantification, image data that is more responsive to or indicative of a given condition, location information, dimension information, type information, recommended clinical decisions, expected outcomes or progress, and / or other information that can be used for reading and reporting. Natural language processing can be implemented to generate natural text for the radiology report 26 from the image processing results and / or the scan data.

[0030] Image processing uses manually coded processes (such as algorithms) and / or the application of one or more machine-learned networks. The fulfillment center implemented by the processor 24 can be fully automated or semi-automated, assisted by artificial intelligence (AI) algorithms, to perform various image processing tasks, such as quality assurance, reconstruction, filtering and denoising, detection, segmentation, classification, quantification, prediction, natural language processing, etc. of imaging examinations.

[0031] Figure 3 An example semi-automated method or arrangement is shown. The processor 24 applies an automated image processing algorithm or a machine-learned network 30. The output of the image processing is reviewed 32 by a human (such as a radiologist) for consistency, accuracy, and / or errors. Then a report generation algorithm 34 is applied to generate a radiology report 26. The human may be in the loop of the fulfillment center to ensure the quality of all or part of the findings and / or reports. In an alternative embodiment, no human review 32 is provided except that the final radiology report 26 is reviewed by a radiologist, a patient, or a consulting physician. A fully automated report generation is provided based on the interaction between the imager 22 and the processor 24.

[0032] The image processing 30, report generation 34, and / or other processes implemented by the processor 24 use the inherent interconnection with the medical imager 22. The processor 24 is configured to select the image processing to be applied based on the interconnection. The clinical application label indicates the type of examination being performed and / or the reason for the examination (e.g., symptoms). A specific set of image processing can be automatically invoked at the fulfillment center based on the clinical indication label. A look-up table is used. Alternatively, a machine-learned network (e.g., a deep machine-learned network 25) is trained to select the image processing and / or the settings for the image processing based on the clinical indication label, scan data, patient information, and / or other information from the imager 22 and / or other sources. The clinical indication label is used to classify and route the patient's examination. The image processing to be performed depends on the type of examination being performed on a specific medical imaging device as indicated by the clinical indication label. For example, the clinical indication label includes symptoms of chest pain and / or a cardiac imaging application scanned with ultrasound. The image processing for this symptom and / or application is selected, such as image processing for detecting stenosis, problematic valve movement, and / or abnormal heart wall movement.

[0033] In other embodiments, no clinical indication label is used. The scan data is processed for image processing to determine the image processing to be performed, such as applying a machine-learned network (e.g., a deep machine-learned network 25) to the scan data and / or the header information with the imaging data frames to select the image processing.

[0034] This inherent interconnection can be used in another way. In typical use, the medical imager 22 performs reconstruction based on the selected imaging settings, such as settings selected for a specific application or scan type with or without considering patient characteristics. Given the interconnection, the medical imager 22 provides the raw scan data or the scan data before reconstruction to the processor 24. As part of the application of image processing, the processor 24 is configured to perform reconstruction. The type of reconstruction or the reconstruction settings to be used can be based on a look-up table from clinical indication tags, clinical findings, and / or other information. Alternatively, a machine-learned network (e.g., the deep machine-learned network 25) correlates the input data to output the reconstruction settings or the type of reconstruction. The image processing algorithm at the fulfillment center can operate directly on the raw data or the scan data before reconstruction and / or can perform reconstruction. At this time, no medical images are present at or shown by the medical imager 22. Clinical findings based on image processing occur before any human visualization of the images from the scan data.

[0035] The processor 24 is configured to determine clinical findings. As part of report generation or image processing, the clinical findings are identified. Clinical findings are a medical view that combines segmentation, detection, and / or classification. Clinical findings provide severity, location, relationship to symptoms, and / or lesion or disease type. For example, the clinical finding is a detected and classified calcified atherosclerotic plaque that results in moderate stenosis in the mid-segment.

[0036] Clinical findings are based on a look-up table and / or a machine-learned network. The results from image processing, patient information, clinical indication tags, and / or scan data are used to derive one or more clinical findings. For example, a natural language processing system using one or more machine-learned networks for text generation is used to provide clinical findings. Clinical findings can be used to guide other image processing, report generation, and / or re-scanning.

[0037] Another use of the inherent interconnection can be to control the medical imager 22 based on clinical findings and / or results from image processing. By applying a machine-learned network (e.g., deep machine learning neural network 25) and / or a manually programmed algorithm, the processor 24 is configured to associate input information (e.g., scan data, results of other image processing such as physical quantities, clinical findings, and / or patient data) to an output (e.g., scan settings, scan type, and / or need for rescan). The processor 24 outputs feedback in the form of scan settings or image processing parameters to the medical imager 22. The settings are for redoing a scan, changing a currently ongoing scan, and / or for performing a different scan. For example, the clinical finding is based on a general heart scan. A further scan using a contrast agent or different settings is to be performed based on the clinical finding to focus on blood vessels or lesions. In an alternative embodiment, the control is less direct. For example, the feedback is a request to perform a further scan or change certain aspects of the scan. A technician at the medical imager 22 implements the request.

[0038] In one example, the applied image processing is quality verification. The scan data is applied to a machine-learned network that is trained to output a quality indication. Based on clinical indication tags, the quality verification (e.g., algorithm and / or network) is selected or adjusted to verify the quality of the imaging type. In the presence of sufficient contrast, sufficient resolution, sufficient signal-to-noise ratio, limited artifacts, and / or other quality indications, the scan data can be used for further image processing and / or clinical findings. In the case where the scan data quality is insufficient (such as based on a threshold measurement), the processor 24 uses the available information to provide settings for another scan. Other image processing can indicate the need or desire for different or additional scan data, such as imaging on another cardiac cycle to better sample.

[0039] The processor 24 is configured to generate a radiology report from the results of image processing. The radiology report includes one or more clinical findings. It can also include quantification, segmentation, detection, and / or other image processing results. It can include a table of measurement results, images, patient information, clinical indication tags, recommended actions, possible treatments, and / or other information. In the case of performing a rescan, results from different scans and / or image processing can be provided. Alternatively, the radiology report is generated from the scan data and image processing of the most recent or most specific scan.

[0040] The generated radiology report can include free text or narrative text. More structure can be provided, such as narrative text in a section communicating clinical findings to other departments being structured to include additional information. In an alternative embodiment, the radiology report is a structured report without narrative text, such as presenting clinical findings as a symptom list, detected and classified diseases, classified severity, and / or segmented or detected locations.

[0041] In one embodiment, the processor 24 is configured to generate clinical findings and / or a radiology report using a machine-learned neural network. A natural language processing system is applied. The processor 24 applies the machine-learned network 25 and / or other natural language processing. The machine-learned network 25 implemented by the processor 24 facilitates generating text for clinical findings and / or a radiology report. The processor 24 is configured to use the network 25 trained from a corpus of radiology terms and / or reports, a plain language corpus, and / or one or more word-embedding linguistic models to perform the generation based on input scan data, image processing results, and / or measurement results.

[0042] The natural language processing system includes a deep machine learning network. Other natural language processing tools may also be included. Computational linguistics (e.g., linguist rules), information retrieval, and / or knowledge representation can be part of the natural language processing system.

[0043] The deep machine learning network facilitates any part of the natural language processing system, such as analyzing, parsing, extracting, retrieving evidence, and / or generating clinical findings. One network can be trained to perform all these actions. Separate networks can be trained for each action. Some actions may not use any machine-learned network. The deep machine learning network is any neural network known now or developed in the future. Generally, deep learning uses a neural network with raw data or data not converted to other features as input and ground truth. Learn the relationship between the values of the input raw data and the ground truth. Deep learning can be used without including specific linguistic rules. The deep neural network processes the input via multi-layer feature extraction to produce features for the output. Deep learning provides features for generating the output. In an alternative embodiment, other machine learning is used, such as Bayesian networks, probabilistic boosting trees, or support vector machines.

[0044] For natural language processing, the deep machine learning network can be a recurrent neural network, a convolutional network, an attention model, and / or a long short-term memory network. Any architecture can be used. Other deep learning sparse autoencoder models can be trained and applied. Machine training is unsupervised when learning the features to use and how to classify given an input sample (i.e., feature vector). Learn the combination of information in the raw data that indicates the ground truth, and learn the output given the combination of input information.

[0045] The trained network is stored in a memory. Store the trained artificial intelligence (i.e., machine learning network). The result of training is a matrix or other model. The model represents the knowledge learned through machine training using deep learning. Other machine learning network representations can be used, such as a hierarchical structure of matrices or other non-linear models.

[0046] Once trained, the machine-learned network is applied by a machine such as a computer, a processor, or a server. The machine uses the input data for the patient (i.e., scan data, clinical indication labels, results from image processing, patient information, and / or information derived therefrom) and the machine-learned network to generate outputs such as clinical findings or a radiology report 26 with clinical findings.

[0047] Other information can be generated to be included in the radiology report or for other outputs. For example, reimbursement codes are automatically generated. The codes are provided based on a look-up table of clinical indication labels, scan data, patient information, and / or other information. Alternatively, the codes are provided by training a machine learning network.

[0048] The processor 24 can create multiple reports for different end consumers (consultants, patients, and radiologists) for the same examination of the same patient. Train the machine-learned network to output different narrative texts, report content, report structure, and / or other radiology report information for different end users or viewers.

[0049] In another embodiment, the processor 24 is configured to identify similar cases and / or provide statistical information. The processor 24 uses clinical findings, image processing results (e.g., quantification), patient information, and / or other information to identify medical records for similar patients. Use the identified medical records to calculate statistical similarities and / or differences. The statistical information and / or list of similar patients can be output separately from the report 26 or included in the report 26.

[0050] In another embodiment, the processor 24 is configured to prioritize the determination of clinical findings and / or the generation of reports. Since the processor 24 can communicate with multiple different medical imagers 22, priorities are provided among the imagers 22. Even for one imager 22, priorities can be provided among patients. The order of reception can be used. Using priorities determined by an application based on manual programming or a machine learning network, clinical findings or radiology reports 26 can be generated for one patient before another patient, even if the other patient was received later. The processor 24 is configured to identify emergency room patients, trauma-related patients, and / or patients more likely to have a rescan based on the available data. For such patients, clinical findings and / or radiology reports 26 are generated with a higher priority than for other patients.

[0051] Referring again to Figure 2 , the interface 27 is a communication port, such as an Ethernet card, or another computer network interface. In other embodiments, the interface 27 is a user interface, such as a user input device (e.g., keyboard, mouse, touchpad, touchscreen, and / or trackball). The interface 27 can be a bus, a chip, or other hardware for receiving and / or outputting information such as scan data, clinical indication labels, patient information, clinical findings, scan settings, and / or radiology reports 26.

[0052] The interface 27 is configured to output the radiology report 26. For example, the radiology report 26 is output to the database 29, the medical imager 22, or a computer. The output from the fulfillment center is a clinical report, which can be automatically sent back to one or more locations, such as an electronic health record or an electronic medical record system, a consulting physician, and / or the patient.

[0053] The medical record database 29 is a random access memory, a system memory, a cache memory, a hard disk drive, an optical medium, a magnetic medium, a flash drive, a buffer, a database, a combination thereof, or other currently known or later developed memory devices for storing radiology reports 26, patient information, clinical findings, scan settings, clinical indication labels, and / or the deep machine learning network 25. The medical record database 29 is part of a computer associated with the processor 24 or the medical imager 22, or a separate or remote database accessible via a computer network.

[0054] The medical record database 29 or other memory is alternatively or additionally a non-transitory computer-readable storage medium that stores data representing instructions executable by the programming processor 24 and / or the medical imager 22. Instructions for implementing the processes, methods, and / or techniques discussed herein are provided on a non-transitory computer-readable storage medium or memory, such as a cache, buffer, RAM, removable media, hard drive, or other computer-readable storage medium. Non-transitory computer-readable storage media include various types of volatile and non-volatile storage media. In response to one or more sets of instructions stored in or on the computer-readable storage medium, the functions, acts, or tasks illustrated in the figures or described herein are performed. The functions, acts, or tasks are independent of the particular type of instruction set, storage medium, processor, or processing strategy and may be performed by software, hardware, integrated circuits, firmware, microcode, etc., operating alone or in combination. Similarly, the processing strategy may include multiprocessing, multitasking, parallel processing, etc.

[0055] In one embodiment, the instructions are stored on a removable media device for reading by a local or remote system. In other embodiments, the instructions are stored at a remote location for transmission over a computer network or over a telephone line. In yet other embodiments, the instructions are stored within a given computer, CPU, GPU, or system.

[0056] The display 28 is a monitor, LCD, projector, plasma display, CRT, printer, or other device now known or later developed for displaying the radiology report 26, clinical findings, and / or patient images. The display 28 receives output from the processor 24, the database 29, or the interface 27. The processor 24 formats the data for display (e.g., maps to RGB values) and stores the image in a buffer, thereby configuring the display 28. The display 28 uses the image in the buffer to generate an image for viewing. The image includes charts, alphanumeric text, anatomical scans, and / or other information. The display 28 is located at the medical imager 22, fulfillment center (e.g., the processor 24), the physician's computer, or another location.

[0057] Since the processor 24 is interconnected with the medical imager 22, the focus is on generating a radiology report rather than just generating an image or just reading and reporting in the case of a given previously acquired image. The communication not only allows for the continuous generation of images, reading, and reporting. As a result, the radiology report and the corresponding clinical findings can be more complete, more accurate, and less likely to result in additional appointments or examinations for the patient.

[0058] The use of a deep or other machine learning network 25 can make the response time of the fulfillment center (e.g., processor 24) more rapid. Due to the application network rather than a series of manually programmed calculations, the computer operates faster to provide clinical findings and / or radiology reports 26.

[0059] Figure 4 An embodiment of a method for imaging a patient in a medical system and generating a radiology report is shown. The imaging, findings, and reporting portions of the processing chain are joined together such that the final radiology report is more likely to be completed based on a given examination of the patient and less likely to require a follow-up examination to complete the diagnosis.

[0060] The method is implemented by Figures 1-3 one or another of the systems of. For example, the method is implemented by a computer, a server, a medical imager, or other processor. In action 40, the medical imager performs a scan and may rescan in feedback. The image processor (e.g., server) receives in action 42, selects in action 44, reconstructs in action 45, performs image processing in action 46, generates in action 48, controls the scan based on feedback from the medical imager, and outputs to a display device in action 49. Different devices may be used.

[0061] Additional, different, or fewer actions may be provided. For example, alternatively, some of the reconstruction from raw data in action 45 and / or the image processing in action 46 at the server that performs action 45 as part of action 40 at the image processor. As another example, clinical identification is not received in action 42 or not used in action 44. As yet another example, feedback from action 46 and / or action 48 is not used to control the scan in action 40. Actions for configuration or communication may be provided.

[0062] Actions 44, 45, 46, and / or 48 use one or more manually coded algorithms and / or one or more machine-learned networks. Different learned networks or classifiers are used for each action, but a common network (e.g., a multi-task or cascade network) or a network formed by other networks may be used to implement one action or a combination of two or more actions. Any type of machine learning and corresponding machine-learned network may be used, such as deep learning with a neural network architecture. Probabilistic boosting trees, support vector machines, Bayesian networks, or other networks may be used. One or more AI-based algorithms (e.g., machine-learned networks) may be applied to the same task, and a consensus decision (e.g., an average) may be included in the final report. For example, a fulfillment center may run several lesion detection algorithms on the same study and report the results in a composite manner.

[0063] The actions are performed in the order shown (e.g., from top to bottom or in numerical order) or in some other order. For example, actions 46 and 45 are performed in reverse order or simultaneously. Action 45 may be performed before action 44. Action 48 may be performed as part of action 46. Feedback of the scan of action 40 from either of action 46 or 48 is shown. The resulting repetition may exclude action 42 and / or action 45.

[0064] In action 40, a medical imaging scanner scans a patient. The medical imaging scanner is any medical imager, such as a CT, MR, nuclear medicine, or ultrasound scanner.

[0065] The patient is scanned along a plane or in a volume. Energy such as x-rays, electromagnetic pulses, acoustic, or other energy may be transmitted into the patient. Energy that passes through the patient and is detected and / or a response to the energy is received or detected from the patient. Alternatively, the scan relies on emission from within the patient and the emission is measured. The detected response, energy, or emission is the raw scan data.

[0066] As part of the scan, the raw scan data may be processed. For example, reconstruction is applied. Reconstruction may determine the response of the patient at positions within the patient, such as for voxels or pixels in a volume or plane of the patient. For MR, k-space data is reconstructed using a Fourier transform. For CT, projection images from different directions relative to the patient are reconstructed by computed tomography. For nuclear medicine, emissions detected along lines of response are reconstructed by tomography.

[0067] Other post-reception processing may be provided, such as filtering, denoising, segmentation, reformatting, rendering, scan conversion, color mapping, and / or magnification. Scan data from any point along this processing path from the raw scan data to the rendered, color-mapped image is output by the scan.

[0068] The scan uses settings based on a pre-booked or prescribed scan type. A physician indicates a clinical application or indication. A technician configures the medical imaging scanner based on the application or indication. Alternatively, the medical imaging scanner self-configures the scan using an application or indication with or without other patient information.

[0069] The scan data, scan settings, patient information, and / or clinical identification (e.g., application or indication) are passed to a fulfillment center or an image processor. A request, scan type (e.g., for different and / or more specific clinical applications), or scan parameter settings may be retrieved from the fulfillment center.

[0070] In operation 42, an image processor of the fulfillment center receives a clinical identification (e.g., an application or indication) for a patient. Scan data, patient information, and / or scan settings may also be received. This information is provided via a computer network. Alternatively, the information is received by loading from a memory.

[0071] In one embodiment, the image processor and the medical imaging scanner are located in different buildings or facilities. For example, the image processor is one or more servers operated by a different entity than the hospital or medical practice operating the medical imaging scanner. The same fulfillment center may be used for multiple medical imaging scanners, such as scanners owned and operated by different hospitals. The image processor receives the same or different types of information for different medical imaging scanners, such as receiving different clinical identifications corresponding to different scans for different patients or reasons, as well as different scans. Alternatively, the image processor and the medical imaging scanner are in the same building or facility.

[0072] The clinical identification may be an indication of symptoms or other reasons for the scan. The clinical identification may be an application, such as a scan type. The clinical identification is a marker or label, but may be a scan setting or other information from which an application or indication may be derived.

[0073] In one embodiment, the received scan data is not reconstructed. The image processor is to perform the reconstruction. This may allow the reconstruction type and / or reconstruction settings to be responsive to clinical findings or other image processing that is not available at the medical imaging scanner, at least during that scan. By reconstructing the scan data into a three-dimensional object space based on more information, the reconstruction may result in an image or other scan data that is better adjusted for clinical needs or reporting.

[0074] In operation 44, the image processor selects image processing. Different image processing may be used for different cases. Different settings (e.g., different filter kernels) within the same image processing may be used for different cases. Different groups and / or sequences may be used for different cases. Instead of relying on user-set configurations, the image processing is selected based on current standards, expert input, and / or other knowledge bases.

[0075] The image processor uses clinical recognition to select image processing. For example, clinical recognition is for cardiac scans or identifying pain related to heart problems. The selected image processing includes filtering, denoising, stenosis detection, detection of heart wall abnormalities, severity classification, and / or quantification of cardiac function (e.g., fractional flow reserve and / or stroke volume). In another example, clinical recognition is for fetal scans or identifying a standard 3-month pregnancy examination. The selected image processing includes heartbeat detection, fetal detection, length and / or volume quantification, and / or health classification. Symptoms, clinical applications, scan settings, other clinical recognition, and / or other information are used to select appropriate image processing.

[0076] In operation 46, the image processor performs the selected image processing in the selected sequence. Different image processing for different patients is performed independently. The scan data is processed based on the selected image processing. Other information can be used for image processing, such as clinical recognition, scan settings, and / or patient information. Image processing suitable for the clinical recognition is performed.

[0077] In one embodiment, the selected image processing includes reconstruction. The received scan data is projection data, k-space data, or response line data that does not represent specific voxels or positions within the patient's body. The image processor performs reconstruction to estimate scalar values at different positions within the patient's body from the scan data. Different reconstructions can be performed for different patients and corresponding clinical indications.

[0078] Other image processing before or after the initial reconstruction can indicate the type and / or settings for the reconstruction. The reconstruction can be repeated. For example, clinical findings indicate that specific types of information may be important, such as the change in heart wall thickness over time. Reconstruction with better or different motion compensation can be used to provide greater resolution, contrast, and / or motion indication for the heart wall. By redoing the reconstruction, the resulting scan data can better represent the information of interest. By using an automated process in a fulfillment center, better information can be provided without relying on the user to identify the differences in the need for different reconstructions.

[0079] Other image processing that can be applied includes filtering, denoising, detection, classification, and / or segmentation from the scan data. Detection can allow for the detection of anatomical structures for quantification and / or detection of the presence of lesions within the patient's body. Filtering and denoising can provide information more targeted to the information of interest. Segmentation can be used for quantification.

[0080] Another form of image processing can be the determination of clinical findings. Clinical findings can include the detection of lesions, lesion locations, and severity. It can include recommended actions and / or comparison information. Clinical findings can be determined using a machine-learned network that is trained to provide clinical findings from, for example, radiology reports, given input scan data, patient information, clinical identifications, and / or information from other image processing (e.g., physical quantities or detections). In one embodiment, clinical findings are provided as part of action 48. Alternatively, clinical findings are performed separately and subsequently used to generate a radiology report.

[0081] Yet another form of image processing is quality verification. The quality of the scan data can be tested, such as measuring signal-to-noise ratio, contrast level, and / or resolution. Artifact presence, interfering anatomy, and / or other quality verifications can be performed.

[0082] Based on one or more results and / or information of the image processing generated for the radiology report in action 48, the image processor can feed back information to the medical imaging scanner. The feedback can be a request to repeat the same scan (e.g., in response to poor quality), a request for a different scan (e.g., in response to clinical findings), or scan settings for the scan (e.g., in response to quality verification or clinical findings). The feedback allows for the collection of better or additional scan data to improve the quality of the ultimately generated radiology report.

[0083] In response to the feedback, the medical imaging scanner rescans the patient in action 40. The rescan can be part of a continued scan or a restart of the scan. The patient is scanned using the scan settings and / or the requested feedback. Actions 42, 44, 45, and / or 46 can be repeated for the scan data from the rescan.

[0084] The rescan can include scanning the same part or region of the patient and / or scanning a different part or region of the patient. For example, the same scan (e.g., a heart scan) is performed again, but with one or more different settings (e.g., amplitude). In another example, the first scan is of the liver, and based on the information extracted from the liver scan, the second scan for the rescan is of another organ.

[0085] In action 48, the image processor generates a radiology report. The radiology report is generated using the scan data, the output from the image processing (e.g., measurements or physical quantities), patient information, clinical identifications, and / or other information. Different radiology reports are generated for different patients, such as generating radiology reports for different patients from different scan data. Even when the same clinical identification is provided for different patients, differences in lesions reflected in different scan data and / or patient information still result in different radiology reports for different patients.

[0086] Generate one or more clinical findings for a radiology report. The clinical findings can be in the form of free text or narrative text. For example, generate clauses or sentences indicating a lesion, the severity of the lesion, and the location of the lesion as a summary opinion. A machine-learned network, such as a deep machine-learned text generator, generates the clinical findings of the radiology report. Natural language processing, which at least partially includes a deep machine-learned network or a machine-learned natural language network, generates narrative text from input data. Apply a neural network that has been trained from a sample of patient radiology reports, scan data, and measurements reported by the patient. Different deep machine-learned text generators can be used for different clinical identifications. Use the clinical identifications to select algorithms and / or machine-learned networks to generate the radiology report, thereby allowing training based on a corpus specific to the clinical indication. Alternatively, the text generator is generic for different clinical identifications.

[0087] The radiology report can be structured. Different information from image processing is inserted into different fields of the structured report. In other embodiments, the radiology report or a portion of the report is unstructured. Input narrative text.

[0088] The radiology report can be generated as part of a combined imaging-to-report system without displaying any images for one, multiple, or all patients. After receiving the scan data until image processing and report generation, a person does not view any images from the scan data. A person such as a technician operating a medical imager can view the images during the scan or may not view the images. A person at a fulfillment center can view the images and / or other information, or may not view. The first person to view after sending the scan data to the fulfillment center can be a physician who views the radiology report and any included images.

[0089] In the case of a rescan, the content of the radiology report can be based on the initial scan data and / or the scan data from the rescan. Clinical findings from different scan data can be included, such as general clinical findings based on the initial scan data and more detailed or specific clinical findings based on the scan data from the rescan. One clinical finding can be determined based on the scan data from multiple scans.

[0090] In operation 49, the image processor outputs the radiology report. Different radiology reports are output for different patients and / or medical imaging scanners. The output is to a display, an electronic medical record database, a consulting physician, a patient, a computer network interface, and / or a memory.

[0091] Although the present invention has been described above with reference to various embodiments, it should be understood that many changes and modifications can be made without departing from the scope of the present invention. Therefore, the foregoing detailed description is to be considered illustrative rather than restrictive, and it should be understood that the following claims, including all equivalents, are intended to define the spirit and scope of the present invention.

Claims

1. A method for imaging in a medical system and generating a radiology report, the method comprising: A medical imaging scanner scans a patient; An image processor receives the clinical identification of the patient, the image processor being in a building different from the medical imaging scanner; Select image processing based on the clinical identification; The image processor performs image processing on first scan data from the scan based on the selected image processing; Feedback information to the medical imaging scanner based on the image processing; The medical imaging scanner re-scans based on the feedback information; The image processor generates a radiology report in response to the first scan data, the feedback information, and / or second scan data from the re-scan, the radiology report having narrative text characterizing the patient; And Output the radiology report, wherein the medical imaging scanner is configured to output clinical identification to a processor, wherein image processing includes determining clinical findings, the clinical findings being a medical opinion combining segmentation, detection, and / or classification, wherein feeding back the information includes feeding back scan settings for the medical imaging scanner, the scan settings being based on the clinical findings, and wherein the medical imaging scanner is used to scan the patient and the image processor is used to generate the radiology report.

2. The method according to claim 1, wherein the image processor and a machine-learned text generator are in a building different from the medical imaging scanner, wherein the image processor receives the first scan data and the clinical identification from the medical imaging scanner, and wherein the image processor performs the selection.

3. The method according to claim 2, wherein scanning includes scanning with a computed tomography system, a magnetic resonance system, or a nuclear medicine system, and wherein performing image processing on the first scan data includes receiving the first scan data as projection data, k-space data, or response line data and reconstructing it to object space based on the clinical identification.

4. The method according to claim 1, wherein, Image processing includes filtering, denoising, detecting, classifying, and / or segmenting from the first scan data.

5. The method according to claim 1, wherein Generating includes generating a machine-learned natural language network.

6. The method according to claim 1, wherein image processing includes determining a first clinical finding with a machine-learned network, and wherein generating the clinical report includes generating the clinical report with the first clinical finding and a second clinical finding, the second clinical finding being from image processing of second scan data from the re-scan.

7. The method according to claim 1, wherein image processing includes verifying the quality of the first scan data and the second scan data, and wherein feeding back the information includes feeding back a request to perform a re-scan based on the quality verification.

8. The method according to claim 1, wherein receiving the clinical identification includes receiving symptoms, and wherein selecting includes selecting based on the symptoms.

9. The method according to claim 1, wherein, Generating the radiology report includes generating narrative text including lesions, lesion severity, and lesion location.

10. The method according to claim 1, wherein, Outputting includes outputting to an electronic medical record database, a consulting physician, and / or the patient.

11. A system for performing imaging and generating a radiology report, the system comprising: A medical imager configured to scan a patient, the configuration being for clinical applications; A processor configured to receive scan data from the medical imager, apply image processing to the scan data, determine clinical findings from the image processing, control the medical imager based on the clinical findings, and generate the radiology report from the clinical findings, the processor being in a different building from the medical imager; And An interface configured to output the radiology report, Wherein the medical imager is configured to output a clinical identification or indication label to the processor, wherein the processor is configured to control the medical imager with scan settings based on the clinical findings, the clinical findings being a medical view combining segmentation, detection, and / or classification, and wherein the medical imager is used to scan the patient and the processor is used to generate the radiology report.

12. The system according to claim 11, wherein, The processor is a server at a facility different from the medical imager.

13. The system according to claim 11, wherein, The processor is configured to apply image processing as reconstruction, scan quality assurance, filtering, denoising, detection, segmentation, classification, quantification, and / or prediction.

14. The system according to claim 11, wherein, The processor is configured to control the medical imager with scan settings based on the clinical findings.

15. The system according to claim 11, wherein the medical imager is configured to output a clinical identification label for the clinical application, and wherein the processor is configured to apply image processing selected based on the clinical identification label.

16. The system according to claim 11, wherein the medical imager is configured to provide scan data from the scan without reconstruction, and wherein the processor is configured to perform reconstruction as part of the application of the image processing.

17. The system according to claim 11, wherein The interface includes a computer network interface for output to a medical record database.

18. A method for imaging and generating a radiology report in a medical system, the method comprising: Receiving first and second scan data from first and second medical scanners, the first and second scan data representing first and second patients and not being reconstructed into a three-dimensional object space; Obtaining first and second clinical indication labels for the first and second scan data; Performing image processing on the first and second scan data, including reconstruction, with the image processing of the first scan data being different from that of the second scan data based on the first and second clinical indication labels being different, the image processing being in a different building from the first and second medical scanners; Generating first and second radiology reports from the information provided by the image processing; And Outputting the first and second radiology reports, Wherein the first and second medical scanners are configured to output first and second clinical indication labels, wherein the image processing includes determining clinical findings, the clinical findings being a medical view combining segmentation, detection, and / or classification, wherein the first and second medical scanners are controlled with scan settings based on the clinical findings, and wherein the first and second medical scanners are used to scan the patient and the image processing is used to generate the radiology report.

19. The method according to claim 18, wherein Generation is performed including after receiving and passing through the image processing and the generation without displaying any images from the first and second scan data to a person.

Images