System and method for display of multi-date and multi-modal images

Abstract

Methods and systems for multi-date, multi-modality imaging displays are provided herein. In one example, a method for displaying a plurality of images in a graphical user interface (GUI) includes obtaining one or more medical images and data seen thereof from a database (902); time ordering (904) the obtained one or more medical images and the seen data thereof; and displaying the one or more medical images and at least one corresponding seen data in respective viewports of the GUI based on the selected GUI configuration, wherein the one or more medical images are comparable images (910).

Description

TECHNICAL FIELD

[0001] Embodiments of the subject matter disclosed herein relate to medical imaging displays, and more particularly to display of multi-date and multi-modality follow-up imaging displays. BACKGROUND

[0002] Medical images, typically stored and transmitted as digital files in Digital Imaging and Communications in Medicine (DICOM) format, are displayed via display devices, often in a graphical user interface of a workstation or Picture Archiving and Communication System (PACS). Medical images, such as images of internal anatomy of a human subject (e.g., a patient) acquired by computed tomography (CT), magnetic resonance imaging (MRI), ultrasound, etc., are used for diagnosis, monitoring, screening, and more by physicians and other care providers in a medical setting. Physicians and other care providers use data of medical images, including sequence numbers, patient orientation data, etc., to assist in evaluation and diagnosis, which are often displayed as annotations overlaid within the graphical user interface. SUMMARY

[0003] In one example, a computing device includes a display screen, the computing device configured to display a plurality of image views within a multi-image graphical user interface (GUI) on the display screen, the plurality of image views displaying respective medical images of a patient, and additionally configured to display one or more finding views within the GUI, each finding view displaying finding data corresponding to an image displayed within one of the plurality of image views, wherein the finding data for each respective medical image of the patient is obtained from a database when the database is in an unactivated state, and wherein the respective medical images are comparable images of one or more imaging modalities.

[0004] In another example, a method for displaying a plurality of images in a graphical user interface (GUI) includes obtaining one or more medical images and their finding data from a database, time-sorting the obtained one or more medical images and their finding data, and displaying the one or more medical images in respective image views of the GUI and the finding data of at least one of the one or more medical images in at least one finding view of the GUI, wherein the display of the respective image views and the at least one finding view is based on a selected GUI configuration, wherein the one or more medical images are comparable images of one or more imaging modalities.

[0005] In another example, a system includes one or more imaging systems configured to acquire medical imaging data of a patient, wherein the one or more imaging systems are coupled to a database configured to store the medical imaging data and to a computing device configured with instructions stored in a non-transitory memory executable by a processor that, when executed, cause the processor to: display a graphical user interface (GUI) including one or more image viewports and one or more view viewports, wherein each image viewport displays a medical image of the medical imaging data and each view viewport displays view data corresponding to a respective medical image displayed within one of the one or more image viewports, wherein the one or more image viewports and the one or more view viewports are arranged within the GUI according to a first GUI configuration; display one or more view elements within each of the one or more view viewports, wherein each of the one or more view elements corresponds to an object of the medical imaging data, wherein the one or more view elements display information related to the object including a percent change in volume of the object; and modify the GUI from the first GUI configuration to a second GUI configuration in response to a user input, wherein in the first GUI configuration, a set of comparable images are displayed in respective image viewports and in the second GUI configuration, a subset of the set of comparable images are displayed in respective image viewports.

[0006] It should be appreciated that the above Brief Summary is provided merely for purposes of summarizing some embodiments of the application so as to provide a basic understanding of the application. The above Brief Summary is not intended to identify key or essential features of the claimed subject matter, nor is it intended to limit the scope of the claimed subject matter in any way. Furthermore, the claimed subject matter is not limited to implementing any of the embodiments discussed in the above Summary. BRIEF DESCRIPTION OF DRAWINGS

[0007] The patent or application file contains at least one drawing executed in color. Copies of this patent or patent application publication with color drawing(s) will be provided by the Office upon request and payment of the necessary fee.

[0008] The application will be better understood by reading the following description of non-limiting embodiments, with reference to the appended drawings, in which:

[0009] Figure 1 A block diagram of an example computing system is shown;

[0010] Figure 2 An example of a graphical user interface (GUI) in a first configuration is shown;

[0011] Figure 3 An example of the GUI in a second configuration is shown;

[0012] Figure 4 A second example of a GUI of Figure 3 is shown;

[0013] Figure 5 An example of a GUI in a third configuration is shown;

[0014] Figure 6 A second example of a GUI of Figure 5 is shown;

[0015] Figure 7 A third example of a GUI of Figure 5 is shown;

[0016] Figure 8 A fourth example of a GUI of Figure 5 is shown;

[0017] Figure 9 A flowchart illustrating a method for displaying a multi-image GUI is shown; and

[0018] Figure 10 A flowchart illustrating a method for modifying the display of a multi-image GUI is shown. DETAILED DESCRIPTION

[0019] The following description relates to various embodiments for follow-up medical imaging displays. In particular, systems and methods are provided for generating and displaying multi-date and multi-modality imaging studies within a single GUI for follow-up comparison. One or more images can be displayed within the GUI in a selected configuration based on how the user prefers to interact with the presented data.

[0020] Hospitals and other clinical facilities can provide computing systems with graphical user interfaces (GUIs) for displaying patient medical images to care providers and other users. Medical images can be displayed in a suitable format, such as Digital Imaging and Communications in Medicine (DICOM), on a display device (e.g., a screen) of a care provider device. Chronic diseases, such as cancer and neurodegenerative disorders, require long-term follow-up, including repeated or serial imaging studies over time. Health care professionals assess disease evolution and / or treatment response over months / years in response to various treatments and interventions, such as medication, surgical resection, radiation therapy, and chemotherapy, for example, the evolution of the size of a tumor over time. Assessments over time inform medical decision making and patient care. To properly assess, an overview and precise comparison between imaging studies is used, including, for example, both a 1 : 1 comparison between the most recent study and a previous study or between the most recent study and a first performed study and a multi-study comparison over a longer time period.

[0021] However, opening multiple images for comparison while allowing efficient and accurate comparison is cumbersome and time consuming. Additionally, multiple types of imaging can be used for disease assessment, such as computed tomography (CT), magnetic resonance imaging (MRI), positron emission tomography (PET), x-ray, ultrasound, mammography, and more, all of which can be presented separately in some systems. A care provider can have to individually find and open multiple studies, manually resize windows to fit within a defined screen size, and separately scroll through slices of 3D images.

[0022] As one example, a patient with monitored colon cancer who has undergone numerous types of studies over months or years can undergo an abdominal CT after a medical intervention (e.g., surgical resection, chemotherapy, etc.). A care provider treating the patient can open the abdominal CT and separately open the most recent prior CT, the most recent prior PET scan, and a first performed abdominal CT taken prior to the medical intervention to assess current disease status. Opening each of the images can be opened in separate windows independent of one another. The care provider can then have to individually resize each of the separate windows as they desire. For example, the care provider can set the size of the abdominal CT to half the size of the screen of the care provider device and set the size of the first performed abdominal CT to the second half of the size of the screen in order to make a 1 : 1 comparison of the two studies to compare, for example, the initial size of a tumor to the current size of the tumor. The care provider can then have to separately resize the size of the other one of the opened images to allow comparison between the three studies. The care provider can also have to manually scroll through the image slices to allow side-by-side comparison of similar sections (e.g., to assess tumor size at the same coordinates over time). Altogether, this image window opening, resizing, and manual comparison between images causes an undesirable amount of manual clicking by the care provider, thereby increasing the overall time taken and decreasing efficiency. Additionally, separately opening individual studies and interacting with them can increase the overall processing power required by the system, thereby decreasing the efficiency of the computing device.

[0023] The methods and systems provided herein detail GUIs that display multiple medical images in various configurations. The GUIs described herein can include one or more image viewports, each of which displays a medical image therein. The medical images in each of the viewports can be interacted with individually and / or together, as will be described herein. The GUIs can also include one or more seen viewports that display information corresponding to one or more of the displayed images. In a first GUI configuration, the image viewports can be configured in a grid, and one seen viewport can be displayed. The displayed seen viewport can correspond to a selected one of the displayed images. In a second GUI configuration, the image viewports can be configured horizontally, and corresponding seen viewports for each of the displayed images can be configured horizontally, e.g., below the displayed image viewports. The imaging viewports and seen viewports can be arranged in a longitudinal, chronological manner. In a third GUI configuration, two image viewports can be arranged side-by-side to allow for a 1 : 1 comparison. Corresponding seen viewports for the two displayed images can also be displayed within the GUI in the third configuration. A user can switch between each of the available GUI configurations based on a desired view, clinical application, and / or intended use. Additionally, in the third comparison configuration, the user can switch between the displayed images and the seen in a chronological manner or can pin a selected study to compare it to different other studies. Additionally, studies can be grouped within the third GUI, e.g., studies performed at substantially the same clinical time, and the grouped images can be displayed with another single image for comparison between the two.

[0024] The GUIs as presented herein can allow a user to visualize all or some of the comparable imaging studies within the same window. In some examples, the window size can be related to the aspect ratio of the window in which the GUI resides. As such, each of the displayed imaging viewports can be displayed as the same size as each other, allowing for comparable views of the anatomy within the displayed images. Additionally, the user can select a particular seen to be displayed within the seen viewports, and all of the displayed images can simultaneously show the particular seen (e.g., via comparable image slices). Thus, the GUIs as presented herein reduce the time a user spends opening individual images, resizing the image window, and scrolling through image slices.

[0025] Reference is now made to Figure 1 , which shows an example of a computing system 100. The computing system 100 includes a computing device 102, which can include a server, a personal computer, a workstation, a mobile device (e.g., a cellular phone, a smart phone, a computing tablet, etc.), or any other type of computing device as illustrative and non-limiting examples.

[0026] The computing device 102 includes a processor 104 configured to execute machine readable instructions stored in a memory 106. The processor 104 can be a single core or multi-core processor, and programs executed thereon can be configured for parallel or distributed processing. In some embodiments, the processor 104 can optionally comprise separate hardware components spread across two or more devices, which can be remotely located and / or configured for cooperative processing. In some embodiments, one or more aspects of the processor 104 can be virtualized and executed by remotely accessible networked computing devices configured in a cloud computing configuration.

[0027] As noted, the computing device 102 also includes a memory 106. The memory 106 can include non-transitory memory, volatile memory, mass storage, local storage, and / or the like or some combination thereof. In some examples, the memory 106 can include components disposed on two or more devices, which can be located remotely and / or configured for cooperative processing. In some embodiments, one or more aspects of the memory 106 can include remotely accessible networked storage configured in a cloud computing configuration. The processor 104 and the memory 106 can be coupled, for example, via a communication bus 118.

[0028] The computing device 102 can also include an interface 120 communicatively coupled to the processor 104 and the memory 106 via the communication bus 118. The interface 120 can be implemented by one or more of any type of interface standard, such as an Ethernet interface, a Universal Serial Bus (USB), a BLUETOOTH interface, a Near Field Communication (NFC), and / or a PCI express interface.

[0029] The computing device 102 can also include one or more output devices 122 communicatively coupled to the processor 104 and the memory 106 via the interface 120. The output devices 122 can include, for example, one or more display devices. Such display devices can include one or more display devices utilizing almost any type of technology (e.g., light emitting diode (LED), organic light emitting diode (OLED), liquid crystal display (LCD), cathode ray tube (CRT) display, in-plane switching (IPS) display, touchscreen, etc.). In some examples, the output devices 122 can include a computer monitor configured to display various types and styles of medical information, including medical images. The output devices 122 can be combined with the processor 104, the memory 106, and / or the user input devices 124 in a common housing, or can be peripheral display devices, and can include a monitor, a touchscreen, a projector, or other output devices known in the art that can enable a user to view decision support output (e.g., alerts) in accordance with one or more examples of the current disclosure, and / or interact with various data stored in the memory 106.

[0030] The computing device 102 can also include one or more user input devices 124 coupled to the processor 104 and the memory 106 via the interface 120. The user input devices 124 can include, for example, one or more of a touchscreen, a keyboard, a mouse, a touchpad, a motion-sensing camera, a microphone, or other devices configured to enable a user to interact with and manipulate data within the computing device 102.

[0031] The interface 120 can also include a communication device, such as a transmitter, a receiver, a transceiver, a modem, a home gateway, a wireless access point, and / or a network interface, to facilitate the exchange of data with external machines (e.g., computing devices of any kind) via a network 126. For example, communication can occur via an Ethernet connection, a digital subscriber line (DSL), a telephone line, a coaxial cable system, a satellite system, a direct-broadcast satellite system, a direct-broadcast satellite system, a straight-line-of-site wireless system, a cellular telephone system, etc. As a non-limiting example, Figure 1One or more caregiver devices 134 communicably coupled to the computing device 102 are shown. Each caregiver device can include a processor, memory, communication module, user input device, display (e.g., screen or monitor), and / or other subsystems (similar to the processes, memory, communication module, user input device, and output device of the computing device 102) and can be in the form of a desktop computing device, laptop computing device, tablet, smart phone, or other device. Each caregiver device can be adapted to send and receive encrypted data and display medical information, including medical images, in a suitable format such as DICOM or other standard. As will be explained in greater detail below, the caregiver devices can display the GUIs described herein on respective display screens.

[0032] The imaging analysis system 140 can obtain, analyze, and display one or more medical images of one or more patients based on instructions stored in the memory 106. The imaging analysis system 140 can include an imaging analyzer 142, a findings module 144, and a GUI configuration module 146. The imaging analysis system 140 can be communicably coupled to a database 150. The database 150 can store medical information, including medical images and data thereof, obtained from a picture archiving and communication system (PACS) 152 and / or one or more imaging systems 154. The imaging systems 154, such as CT systems, MRI systems, PET systems, etc., can include imagers configured to acquire medical imaging data and can be configured to reconstruct medical images from the medical imaging data. The medical images can be stored via the PACS 152 or directly via the database 150.

[0033] The imaging analyzer 142 can analyze the obtained medical imaging data and determine one or more parameters of the corresponding medical images. In some examples, the imaging analyzer 142 can organize the medical images by patient, date, scan type, imaging protocol, etc. The imaging analyzer 142 can obtain, generate, or otherwise determine image dates, image series, patient orientation of images, or more. The imaging analyzer 142 can also include instructions for generating one or more reformatting and / or rendering from the obtained medical images, such as multi-planar reformatting (MPR), maximum intensity projection (MIP), etc. The resulting reformatting and / or rendering can be displayed in a GUI via the caregiver devices 134. The imaging analyzer 142 can also generate and display annotation data of respective image data and image slices thereof within an image viewport, such as image series, date, etc. The imaging analyzer 142 can thus define comparable studies based on imaging protocols (e.g., scan range, protocol name, etc.), where the comparable studies include data of similar regions of the body, similar anatomical structures, etc.

[0034] The findings module 144 can obtain and / or generate data for one or more findings in each of the obtained imaging studies. For example, the findings module 144 can obtain and / or generate data for the size of a lesion at various slice locations based on the generated segmentation data. The findings module 144 can also display the obtained and / or generated findings data within the GUI based on the current GUI configuration. For example, the findings module 144 can display a findings viewport displaying each of the display image viewports in a horizontal configuration. The configuration of the GUI can be determined and generated by the GUI configuration module 146.

[0035] The GUI configuration module 146 can determine the location and size of the image viewports and findings viewports based on the selected GUI configuration. As previously described, the GUI can display in a grid configuration, a horizontal configuration, or a comparison configuration. In the grid configuration, one or more image viewports are displayed in a grid, and one findings viewport is displayed as a side panel, as will be described with respect to Figure 2 In the horizontal configuration, comparable images are displayed horizontally in the image viewports in chronological order, and a findings viewport is displayed for each of the display image viewports, as will be described with respect to Figure 3 and Figure 4 In the comparison configuration, two images are displayed side-by-side in the image viewports, and corresponding findings viewports for the two display images are displayed side-by-side in a side panel, as will be described with respect to Figures 5 to 8 The GUI configuration module 146 can also determine the selected image, whether the images have been grouped, and whether the images have been pinned. Additionally, the GUI configuration module 146 can arrange the findings viewports so as to align comparable findings elements corresponding to different display images. The GUI configuration module 146 can also determine the relative size settings for the various image and findings viewports based on the GUI window size, the number of viewports, the selected GUI configuration, etc. The GUI configuration module 146 can thus automatically adjust the viewport size and dimensions in response to changes in the GUI window size based on the aspect ratio of the GUI window. The size of the display images can thus always remain proportional and similar regardless of changes in the GUI window size. Additionally, since the GUI configuration module 146 automatically adapts the size settings of the viewports, the user does not need to do so manually, saving time and increasing efficiency.

[0036] Turning now to Figure 2The first GUI 200 is shown in a grid configuration. In some examples, the grid configuration can be a default GUI configuration. The first GUI 200 can be displayed on a care provider device within one or more image viewports 204 and a see view 206 within which a set of comparable images are displayed. Each of the images can correspond to a specified patient. The first GUI 200 can also include a patient demographics subheader 202 that includes information such as patient name, patient date of birth, patient medical identification number, patient gender, and the like. Additionally, the first GUI 200 can include a close element 260 of the see view 206 that, when selected, can remove the see view 206 from display. The configuration of the first GUI 200 can be manually changed from the grid configuration to a comparison configuration via selection of a comparison configuration element 252 or to a horizontal configuration via selection of a horizontal configuration element 254.

[0037] Each of the one or more image viewports 204 can display an image within. In some examples, at any point in time, when the image data set is a 3D data set, the displayed image can be an image slice of the imaging data set. In other examples, the displayed image can be one of a plurality of 2D views of the imaging data set (e.g., in the case of a multi-view x-ray imaging study). Additionally, each of the image viewports can display an annotation overlay within that identifies data of the imaging study such as acquisition date, sequence, imaging sequence, current view (e.g., axial vs. sagittal vs. coronal), and the like. For example, the first image viewport 210 that displays a first image can include an annotation overlay 222.

[0038] Each of the image viewports can have a header row, an identifying outline, and an identifying icon. As one example, the first image viewport 210 can include a header row 226 including a title of the imaging study displayed therein, a date on which the imaging study was acquired, and a study description, a first identifying outline 220, and a first identifying icon 238. In some examples, the identifying outline and the identifying icon of each image viewport can be displayed in a matching color, thereby identifying the imaging study by the color displayed. For example, the first identifying outline 220 and the first identifying icon 238 can be displayed in a first color, such as green. The second image viewport 212 displaying a second image can include a second identifying outline 214 and a second identifying icon 216, both of which can be displayed in a second color, such as orange, that is different from the first color. Additionally, in some examples, when the first image viewport 210 is a “current” (e.g., most recently acquired) imaging study displayed within the first GUI 200, the first identifying icon 238 can be a first type of icon, such as a tear drop identifying the image of the first image viewport 210 as the current study. The other identifying icons of the other imaging viewports can be a second type of icon, such as a square with a letter therein. Displaying the letter can correspond to a chronological order of the imaging studies, with the second most recent study labeled as “A,” the third most recent study labeled as “B,” and so on.

[0039] Additionally, each of the one or more image viewports 204 can be selectable. In the displayed grid configuration, the seen viewport 206 can correspond to a selected image viewport. For example, the first image viewport 210 can be the selected image viewport of the one or more image viewports 204. Thus, the seen viewport 206 can correspond to the first image viewport 210. Additionally, each of the one or more image viewports 204 can include a second outline indicating whether the viewport is the selected viewport. For example, the first image viewport 210 can include a highlighted second outline 242, while the second image viewport 212 can include a non-highlighted second outline 244 indicating that the first image viewport 210 is selected and the second image viewport 212 is not. Additionally, the selected image viewport (in this case, the first image viewport 210) can display a scroll bar 224 via which a user can manually scroll through the image slices. In some examples, scrolling through the image slices of the selected image viewport can also cause the other displayed images to match the current image slice (or the most comparable image slice). In this way, scrolling through the image slices of the selected study can be equivalent to scrolling through the selected slices of all of the displayed images. In other examples, scrolling through the selected image slices can be performed independently of the other displayed images.

[0040] In the first GUI configuration, the seen viewport 206 can include an image study subheading 214 that includes identifying information for the corresponding imaging study (in this case, the imaging study of the first image viewport 210). The seen viewport 206 also includes one or more seen elements 240. Each of the seen elements 240 can correspond to an object that appears within the corresponding imaging study, such as a lesion, a tumor, an area of inflammation, etc. For example, a first seen element 228 can be displayed within the seen viewport 206. The first seen element 228 and other seen elements displayed therein can include various types of information related to the corresponding object 246 as well as one or more selectable elements. For example, the first seen element 228 can include image series information, image slice information, and abbreviated object information, such as information for one or more metrics (such as size or standardized uptake value (SUV)). This information can include the metric itself as well as a percentage change compared to a reference image. As one non-limiting example, the abbreviated object information can indicate a size of the object, including a total volume of the object 246 as well as a percentage change compared to a reference imaging study.

[0041] As one example, the first image can be a current study, and the second image can be a most recent prior study (e.g., a longitudinally adjacent study) of the first image. When the object information includes size information, as Figure 2 As shown, the abbreviated object size 232 of the first seen element 228 can indicate the total volume of the object 246 as well as a percentage change compared to a reference image. However, it should be understood that other object information metrics can be displayed within for different types of objects. In some examples, the abbreviated object size 232 displayed within the first seen element 228 can be color-coded based on the percentage change. For example, a negative percentage change (e.g., a statistically significant decrease in object size) can be displayed as green, a neutral percentage change (e.g., no statistically significant change) can be displayed as yellow, and a positive percentage change (e.g., a statistically significant increase in object size) can be displayed as red. In this way, a user can easily determine the change in a given metric (e.g., size of a lesion, SUV of a lesion, etc.) at a glance. For example, for a size metric, a user can easily determine whether a lesion or other type of object is increasing in size or decreasing in size and by how much, thereby reducing the time a user spends looking for matching images and individually measuring lesion size.

[0042] The abbreviated object size 232 can have a corresponding object color 248 also displayed within the first seen element 228. When the first seen element 228 is selected, the color of the object color 248 can correspond to the display color of the object within one or more of the image viewports 204. For example, when the first seen element 228 is selected, the object 246 to which the first seen element corresponds can be highlighted within one or more of the image viewports 204 in the color of the object color 248. The one or more image viewports 204 displaying one or more highlighted objects 246 can correspond to the images in which the object 246 is present.

[0043] As noted above, each seen element can include one or more selectable elements. For example, within the first seen element 228, the one or more selectable elements can include a hide element 262, an expand element 264, and an additional information element 266. The hide element 262, when selected, can remove the first seen element 228 from display within the seen viewport 206. The expand element 264, when selected, can trigger display of additional information for the object 246, including the minimum radius, the maximum radius, the minimum radio density (e.g., in Hounsfield Units (HU)), the maximum radio density, etc.

[0044] In some examples, an object can be present in one image and not present in another image, such as after a surgical resection or chemotherapy. In such examples, the corresponding seen element can indicate that no corresponding seen was found. For example, the second seen element 270 of the seen viewport 206 includes an indication that no corresponding seen was found in place of the percentage change amount. Additionally, the second seen element 270 can be shaded, indicating that the seen element would correspond to the absence of the object in the corresponding image.

[0045] The selected image viewport can be changed manually via user selection of another image viewport. Alternatively, the selected image viewport can be changed via user selection of the next element 234 or the previous element 236. The next element 234, when selected, can select the next image viewport based on the date of acquisition of the corresponding image (e.g., the next later acquired image). The previous element 236, when selected, can select the previous image viewport based on the date of acquisition of the corresponding image (e.g., the next earlier acquired image). Accordingly, the seen viewport 206 can also update to display information related to the newly selected image viewport. As one example, selection of the previous element 236 can deselect the first image viewport 210 and the second image viewport 212 can be selected. The seen viewport 206 can thus display information related to the second image displayed within the second image viewport 212. In this way, information related to multiple images acquired across multiple dates and with multiple modalities can be viewed within the same GUI.

[0046] One or more image viewports 204 in the grid configuration can be configured with a shape and size that fit within the window of the first GUI 200. Additionally, the shape and size of the viewports can depend on how many comparable studies are available for display. For example, six image viewports are displayed within the first GUI 200. However, if four comparable images are available, the size of the viewports can be larger than when six comparable images are available. Additionally, in examples where the number of available viewports is too small to fill each viewport in the assigned grid size, one or more viewports can be larger than the other viewports included in the grid. In some examples, the arrangement of images within the image viewports can be in decreasing chronological order from left to right, or right to left, per row, depending on user preference.

[0047] In this way, the first GUI 200 in the first configuration can allow the user to see each of the comparable studies in the set of comparable studies at the same time without having to manually resize individual windows. This reduces the time spent by the user in setting up the viewports and can also reduce the processing power used by the computing device by reducing the number of interactions the user has with the viewports.

[0048] Turning now to Figure 3 , a second GUI 300 is shown. Similar to the first GUI 200, the second GUI 300 can be displayed on a care provider device. The second GUI 300 can be a horizontal configuration. In some examples, the second GUI 300 can be displayed in response to a user selection of a horizontal configuration element within a GUI in another configuration. For example, the first GUI 200 as described above can be in a grid configuration and can include a horizontal configuration element 254. The horizontal configuration element 254, when selected, can initiate the second GUI 300.

[0049] The second GUI 300 can include a patient demographics subheading 302 that displays the patient name, patient birth date, patient identification number, and patient gender. The second GUI 300 can also include one or more image viewports 304 and one or more findings viewports 306. Each of the one or more findings viewports 306 can correspond to and be vertically aligned with one of the one or more image viewports 304. Similar to the first GUI 200, the second GUI 300 can include a close element 340 of the one or more findings viewports 306 that, when selected, can remove the findings viewport 306 from display. The configuration of the second GUI 300 can be manually changed from the horizontal configuration to a comparison configuration via selection of a comparison configuration element 344 or to a grid configuration via selection of a grid configuration element.

[0050] As referenced above Figure 2As described, each of the one or more image viewports can display an image therein, can have a header, an identifying contour, and an identifying icon. Each of the one or more seen viewports can similarly include a header element identifying which image study it corresponds to. The header element can also include the same identifying icon as the corresponding image viewport. For example, the first image viewport 308 can correspond to the first seen viewport 310. The first image viewport 308 can include a header 309 including information such as study modality, study description, and acquisition date, as well as an identifying icon. The first seen viewport 310 can include a corresponding header 312 including information such as study modality and acquisition date, as well as an identifying icon. The first image displayed within the first image viewport 308 can be the current image and thus its identifying icon can be the current study icon (e.g., a tear drop shaped pinned icon), thereby visually indicating that the first image is the current study.

[0051] Similar to the first GUI 200, each of the seen viewports 306 can include one or more seen elements. For example, the first seen viewport 310 can include one or more seen elements 314, including a first seen element 316, each displaying information related to an object, including a total volume of the object in the corresponding image. Each of the one or more seen elements 314 can be linked to a seen element of an adjacent seen viewport. For example, the second image viewport 350 can be directly adjacent (e.g., in a temporal chronological sense and in some examples visually) to the first image viewport 308 and can correspond to a second seen viewport 352, which can be directly adjacent to the first seen viewport 310. The second seen viewport 352 can include one or more seen elements 354, each of which can be aligned with one or more seen elements 314. A plurality of linking arrows 342 can be arranged between the first seen element 314 and the second seen element 354. The plurality of linking arrows 342 can include a linking arrow between each aligned seen element. The linking arrows can point in a direction of increasing acquisition date. The linking arrows 342 can visually indicate to a user in a displayed percentage change which object measurements are being compared. For example, the linking arrows can extend from a second seen element 356 of the one or more seen elements 354 of the second seen viewport 352 to the first seen element 316 of the first seen viewport 310. As previously described, the first seen element 316 can include a percentage change amount, and the percentage change amount can reference the percentage change compared to the information of the object displayed in the linked second seen element 356.

[0052] As described with respect to the first GUI 200, the second GUI 300 can include a plurality of image viewports 306 and a plurality of seen viewports 306. The image viewports 306 can each display an image of a corresponding image study. The seen viewports 306 can each display one or more seen elements 314, each of which can be linked to a corresponding seen element of an adjacent seen viewport. The linking arrows 342 can be arranged between each adjacent seen element. The linking arrows 342 can include a linking arrow between each aligned seen element. The linking arrows 342 can point in a direction of increasing acquisition date. The linking arrows 342 can visually indicate to a user in a displayed percentage change which object measurements are being compared. For example, the linking arrows can extend from a second seen element 356 of the one or more seen elements 354 of the second seen viewport 352 to the first seen element 316 of the first seen viewport 310. As previously described, the first seen element 316 can include a percentage change amount, and the percentage change amount can reference the percentage change compared to the information of the object displayed in the linked second seen element 356. Figure 2As described, each of the image views 304 and each of the seen elements of the seen views 306 can be selectable elements. The seen elements of the seen views 306 can be linked to the corresponding image views, whereby selection of a seen element also selects the corresponding image view. For example, the first seen element 316 of the first seen view 308 is selected, and the first image view 310 can also be selected. Additionally, the object to which the first seen element 316 corresponds can be highlighted within each of the display images (e.g., a segmentation mask can be superimposed on each display image).

[0053] In some examples, the percent change of the object can be displayed within each of the seen views except for the seen view corresponding to the first acquired image. For example, the fourth imaging view 326 can correspond to the fourth seen view 320. The fourth imaging view 326 can display the first acquired image of the comparable images displayed within the second GUI 300. Additionally, the fourth seen view 320 can include a flag icon 324 in its caption. The flag icon 324 can indicate that the corresponding image is a reference image. In some examples, the reference image can be the first acquired image in which one or more objects are recorded. For example, an imaging study can be performed for a patient's symptoms, and this imaging study can first show a lesion. This imaging study can thus become the reference study for all subsequently acquired comparable images. In other examples, the reference image can not be the first acquired image, depending on the clinical application. For example, a pre-treatment imaging study can be designated as the reference study in order to make pre-treatment versus post-treatment comparisons. The fourth seen view 320 can include a third seen element 328 of the object corresponding to the first seen element 316 and the second seen element 356. The third seen element 328 can not include the percent change amount, but can instead indicate a baseline object metric 330, such as a baseline object size when the corresponding image is the reference image. As a point of comparison, in a grid configuration, the seen view can additionally include a flag icon when the reference image is selected.

[0054] Additionally, each of the sub-titles of each of the one or more seen viewports 306 can include selectable elements, including a hide element and an additional action element. For example, the third image viewport 332 with the identifying icon 334 can correspond to a third seen viewport 360 with a sub-title 362. The sub-title 362 can include the same identifying icon 336 as the identifying icon 334 of the third image viewport 332. The sub-title 362 can additionally include a hide element 322. The hide element 322, when selected via a first user input such as a mouse click, can trigger removal of the third image viewport 332 and the third seen viewport 360 from display within the second GUI 300. The hide element 322 can also be selected via another type of user input, such as a hover that displays a pop-up window indicating the result of the first user input (e.g., indicating that the viewport will be hidden upon selection of the hide element).

[0055] The one or more image viewports 304 can be arranged horizontally along a first portion of the second GUI 300 in a chronologically ordered manner. The one or more image viewports 304 can include all available comparable imaging studies for a patient. In some examples, a comparable imaging study can be an imaging study that includes similar patient anatomy. For example, Figure 3 The illustrated one or more image viewports 304 include acquired images of an abdomen. The comparable images can include images of various modalities, such as a CT abdomen, a CT abdomen / pelvis, an MRI abdomen, and a PET / CT abdomen, all of which can be considered comparable studies that image the anatomy of an abdomen.

[0056] The one or more seen viewports 306 can be arranged horizontally along a second portion of the second GUI 300. As Figure 3 and Figure 4 illustrated, the first portion can be an upper portion, and the second portion can be a lower portion arranged below the one or more image viewports 304. However, it should be understood that in other examples, seen viewports can be arranged in the upper portion, and image viewports can be arranged in the lower portion.

[0057] When the set of images and seen viewports are hidden, the second GUI 300 can automatically adapt to expand the size of each of the remaining image viewports and each of the remaining seen viewports to fit within the GUI window. In this way, the GUI can be configured to adapt the window size of each of the viewports based on the aspect ratio and size of the GUI window, the number of viewports, and the GUI configuration.

[0058] Turning to Figure 4 , the second GUI 300 is also illustrated. As Figure 4 illustrated, the second GUI 300 can be responsive to selection of the hide element 322 via a first user input (e.g., a mouse click) to remove the third image viewport 332 and the third seen viewport 360 from display within the second GUI 300.Figure 3 user selection of the hidden element 322. After the third imaging viewport and the third seen viewport are hidden, the second GUI 300 can display each of the remaining imaging viewports and seen viewports in a similar manner as prior to being hidden. As noted, the second GUI 300 can be configured to resize each of the remaining viewports to fit within the same GUI window size.

[0059] After the third imaging viewport and the third seen viewport are hidden, the second GUI 300 can continue to display its identifying icon 336. The identifying icon 336 can be a selectable element that, when selected via a user input such as a mouse click, can trigger the display of the corresponding seen viewport and imaging viewport (e.g., thereby causing the second GUI 300 to return to the view shown in FIG. 3B). Figure 3 Additionally, in the case that the third seen viewport is hidden, a link arrow 410 can be displayed between the next seen viewport 420 and the previous seen viewport 320 (e.g., the fourth seen viewport 320) of the third image. Additionally, a dashed line 412 can be positioned as a placeholder between the next seen viewport and the previous seen viewport, indicating the presence of an image therebetween.

[0060] In some examples, the percentage change displayed within the next seen viewport 420 can be related to the previous seen viewport 320, skipping the seen of the third seen viewport when the third seen viewport is hidden. In other examples, the percentage change displayed within the next seen viewport 420 can be related to the hidden third seen viewport, the dashed line 412 indicating the presence of the hidden seen viewport.

[0061] Turning now to Figure 5 , a third GUI 500 is shown. Similar to the first GUI 200 and the second GUI 300, the third GUI 500 can be displayed on a care provider device. The third GUI 500 can be a comparison configuration (e.g., a vertical configuration). In some examples, the third GUI 500 can be displayed in response to a user selection of a comparison configuration element within a GUI that is in another configuration. For example, the first GUI 200, as described above, can be a grid configuration and can include a comparison configuration element 252. The comparison configuration element 252, when selected, can initiate the third GUI 500. In some examples, the second GUI 300 can include a similar comparison configuration element (e.g., comparison configuration element 344) that, when selected, can also initiate the third GUI 500.

[0062] The third GUI 500 can include a patient demographics subheading 502 that displays the patient name, patient birth date, patient identification number, and patient gender. The third GUI 500 can also include two or more image viewports 504 and two or more findings viewports 510 that correspond to a subset of the set of comparable images. Each of the two or more findings viewports 510 can correspond to one of the two or more image viewports 504. As referenced above with respect to the first GUI 200 and the second GUI 300, each of the two or more image viewports can display an image therein, can have a title, an identifying outline, and an identifying icon. Each of the two or more findings viewports can similarly include a title element that identifies which image study it corresponds to. The title element can also include the same identifying icon as the corresponding image viewport. For example, a first image viewport 506 can correspond to a first findings viewport 512. The first image viewport 506 can include a field title that includes information such as the study modality, study description, and acquisition date, as well as an identifying icon. The first findings viewport 512 can include a corresponding field title 520 that includes information such as the study modality and acquisition date, as well as the identifying icon. Figure 2 As described, each of the two or more image viewports can display an image therein, can have a title, an identifying outline, and an identifying icon. Each of the two or more findings viewports can similarly include a title element that identifies which image study it corresponds to. The title element can also include the same identifying icon as the corresponding image viewport. For example, a first image viewport 506 can correspond to a first findings viewport 512. The first image viewport 506 can include a field title that includes information such as the study modality, study description, and acquisition date, as well as an identifying icon. The first findings viewport 512 can include a corresponding field title 520 that includes information such as the study modality and acquisition date, as well as the identifying icon.

[0063] Similar to the first GUI 200 and the second GUI 300, each of the findings viewports 510 can include one or more findings elements that indicate data for the corresponding subject, such as the subject’s total volume. For example, the first findings viewport 512 can include one or more findings elements, including a first findings element 524. Each of the one or more findings elements can be linked to a findings element of an adjacent findings viewport. For example, a second image viewport 508 can be adjacent to the first image viewport 506 and can correspond to a second findings viewport 514, which can be adjacent (e.g., both visually and chronologically) to the first findings viewport 512. The second findings viewport 514 can include one or more findings elements, each of which can be aligned with one or more findings elements of the first findings viewport 512. A plurality of linking arrows 518 can be arranged between the first findings element 512 and the second findings element 514. The plurality of linking arrows 518 can include a linking arrow between each of the aligned findings elements. The linking arrows can point in a direction of increasing acquisition date. The linking arrows 518 can visually indicate to the user in a displayed percentage change which subject measurements are being compared.

[0064] For example, a link arrow can extend from a second seen element 534 of the one or more seen elements of the second seen viewport 514 to a first seen element 524 of the first seen viewport 512. The first seen element 524 can include a percent change amount 532, as previously described, and the percent change amount 532 can reference a percent change in a given metric (e.g., object size, SUV, etc.) of the corresponding object in the first image compared to the second image to which the linked second seen element 534 corresponds. The second seen element 534 can also include a percent change amount 536, which can be based on a comparison to the next most recent study of the seen element that is not currently displayed within the third GUI 500.

[0065] Additionally, each of the seen elements can be color-coded, as previously described, and the corresponding object can be highlighted within each of the displayed images with the corresponding color when selected. Additionally, each of the seen elements can include one or more selectable elements. For example, the first seen element 524 includes a hide element 526 and an expand element 528, which, when selected, removes the first seen element 524 from display within the first seen viewport 512. As shown, when selected, the expand element 528 can expand the first seen element 524 to display additional information 530 of the object to which the first seen element 524 corresponds, including a minimum radius measurement, a maximum radius measurement, a minimum radio density, a maximum radio density, a mean radio density, and a standard deviation of radio density. In some examples, the expansion of the first seen element 524 can also trigger the expansion of the second seen element 534 to which the first seen element 524 is linked. In this way, a user can directly compare one or more parameters of an object between different imaging studies without having to open multiple windows or individually interact with the images. Figure 4

[0066] ​The third GUI 500 can also include a display image change element 516 that includes a next element and a previous element. When no image is pinned, the next element, when selected via user input, can trigger a shift in the images and views in the viewports to display the next previously acquired image for comparison with the second image. Specifically, the image displayed within the second image viewport 508 can shift to be displayed within the first view viewport 506, and the image displayed within the first view viewport 506 will be removed from display. A third image can be displayed within the second image viewport 508, replacing the image that moved to the first view viewport 506. Similarly, the information presented within the first view viewport 512 can be removed from display, the information presented within the second view viewport 514 can shift to be displayed within the first view viewport 512, and information for a third image can be presented within the second view viewport 514. In this way, the next element can allow for viewing of subsequent images for 1 : 1 comparison with adjacent images. The previous element can have a similar, but opposite, effect whereby the images and views displayed in the viewports can shift to display a recently acquired image for comparison with an adjacent previously acquired image (e.g., the first image). Thus, a user can easily switch between images for comparison over time.

[0067] Additionally, the field title 520 of the first view viewport 512 can additionally include a pin element 522. The pin element 522, when selected via user input, can pin the imaging study currently displayed within the first imaging viewport 506 and the first view viewport 512 to the third GUI 500. Thus, when a user changes which images and information thereof are displayed, for example, via the display image change element 516, only the images in the second image viewport 508 and the information in the second view viewport 514 can shift, while the image currently displayed in the first image viewport 506 and the corresponding information displayed within the first view viewport 512 can remain unchanged. In this way, a user can individually compare a pinned image study (e.g., its images and views thereof) to each of the other available comparable images, rather than just to adjacent imaging studies.

[0068] For example, Figure 6 A third GUI 500 is shown in which a first imaging study is pinned. While the third GUI 500 as shown displays a first image and a second image in the viewports, and their views, where the first and second images are adjacent to each other in time order, Figure 5 A third GUI 500 is shown in which a first imaging study is pinned. While the third GUI 500 as shown displays a first image and a second image in the viewports, and their views, where the first and second images are adjacent to each other in time order, Figure 6 A third GUI 500 is shown in which a first imaging study is pinned. While the third GUI 500 as shown displays a first image and a second image in the viewports, and their views, where the first and second images are adjacent to each other in time order, Figure 6As shown, in the case where the first imaging study is pinned, the user can switch the display of different images in the second image viewport 508 via the display image change element 516. In the case of a study that is not pinned, the display images can always be adjacent to each other in time order. As Figure 6 As shown, in the case where the study is pinned, the display images can not necessarily be adjacent to each other in time order.

[0069] For example, the first image viewport 506 can display a first image of the pinned study, and the second image viewport 508 can display a third image. In some examples, the first view viewport 512 can remain unchanged compared to that described with respect to Figure 5 However, when the second image viewport 508 displays the third image in response to a user selection of the third imaging study (e.g., via one or more selections of the next element of the image change element 516), the second view viewport 514 can indicate the view of the third image. The view elements of the second view viewport 514 can remain linked to the view elements of the first view viewport 512, as previously described. For example, the second view viewport 514 can include a third view element 634 that indicates information 636 of the object to which the first view element 524 and the third view element 634 correspond, such as total volume and percent volume change.

[0070] When the display images and their views are not adjacent in time order, as Figure 6 depicted, in the case where the first study is pinned, a dividing line 610 can be displayed between the first view viewport 512 and the second view viewport 514. The dividing line 610 can indicate that one or more imaging studies are available between the currently displayed first image and the third image. The linking arrows 518 can link the view elements as previously noted, however the dividing line 610 can indicate to the user that the currently displayed third image is not longitudinally adjacent to the first image.

[0071] Pinning the study allows the user to individually compare the selected image and its view to each of the other available comparable images as desired. For example, the pinned study can be a recently acquired imaging study, and the user can first compare one or more metrics of the object in the first imaging study (such as size and SUV) to the recently acquired imaging study. The user can then switch to display a second previously acquired imaging study, and can compare the first imaging study to the second previously acquired imaging study. In this way, the user is able to directly compare not only images that are adjacent in time order, but also images that are not adjacent in time order, allowing for comparisons over a longer time period within the same single GUI.

[0072] Turning now to Figure 7 again showing the third GUI 500, similar toFigure 5 .like Figure 7 The third GUI 500 shown displays the first image in the first image window 506 and the second image in the second image window 508. Comparison GUI configurations allow users to group one or more imaging studies. For example, some imaging studies may be considered to have occurred “at the same time” from a clinical perspective, despite having different dates or timestamps. For instance, a patient might undergo a CT scan on a first date for symptom assessment, and then, based on the findings of that CT scan, might undergo an MRI for additional characterization of those findings. CT and MRI scans may be acquired on different dates, but because the time interval between them is not clinically significant, there is no medical intervention between them, and / or there is no significant change in the findings between them, the two studies can be considered together. For example, in the presented scenario, both the CT and MRI scans could be considered pre-intervention studies, and therefore, from a clinical perspective, the care provider could consider them to have been acquired “simultaneously.”

[0073] In this scenario, the user can group the study within the GUI. For example, the second image displayed in the third GUI 500 can be grouped with the third image. As previously noted, the second viewport 514 may include a subheading with an additional action element 702. The additional action element 702 can trigger the display of a first pop-up menu 720 when selected via user input. The first pop-up menu 720 may include multiple selectable action elements. For example, the first pop-up menu 720 may include selectable elements that can set the second image as a reference image, a baseline image, and / or a minimum point image (e.g., an image where one or more objects in the comparable images are at their minimum values). The pop-up menu 720 may also include a grouping element 706.

[0074] When grouping element 706 is selected, it triggers the display of a second pop-up menu 708. The second pop-up menu 708 may include a list of available imaging studies, with which second images can be grouped. Selecting a study from the list triggers the system to group the second study with the selected study. For example, a user selection of listed study 710 triggers the grouping of the second study with listed study 710 (e.g., a third image).

[0075] In response to the grouping of studies as described above, the third GUI 500 can be displayed together with the grouped studies, such as... Figure 8 As shown. Figure 8As shown, when including grouped studies, the third GUI 500 can include a first image viewport 506 and a grouped image viewport 820. Similarly, when including grouped studies, the third GUI 500 can include a first findings viewport 512 and a grouped findings viewport 802. The grouped image viewport 820 can be a second image viewport 508 in which more than one image is displayed, as will be further explained. The grouped findings viewport 802 can be a second findings viewport 514 in which more than one image's findings data is displayed, as will be further explained.

[0076] For example, the grouped image viewport 820 (e.g., the second image viewport 508 when the studies are grouped) can include a first grouped image viewport 822 and a second grouped image viewport 824. As one example, the first grouped image viewport 822 can display the second image displayed within the second image viewport 508, and the second grouped image viewport 824 can display a third image of the listed studies 710 that is grouped together with the second image.

[0077] The grouped image viewports 822, 824 can be arranged vertically within the third GUI 500 so as to occupy the same space as the second image viewport 508. Thus, the GUI can configure the size of the grouped image viewports and the display annotations, display images, etc. within them. Each of the first and second grouped image viewports 822, 824 can be individually selectable and / or collectively selectable. When selected, the user can interact with the selected image viewport, including scrolling through image slices of the corresponding image, selecting on various elements such as annotation elements, etc.

[0078] The grouped findings viewport 802 can include a subheading 804. The subheading 804 can indicate the study modality / protocol and date of each study that is grouped (such as the second and third images). In some examples, the grouped studies can have the same or similar subject and subject size to one another, and thus the data reported in one or more findings elements within the grouped findings viewport 802 can be shared data.

[0079] As one example, the grouped findings viewport 802 can include a first grouped findings element 806. The first grouped findings element 806 can include data corresponding to the subject of the first findings element 524, e.g., size information 808, including measured volume and percent volume change. Depending on the clinical situation, other metrics such as SUV can also be included in some examples. In some examples, the first grouped findings element 806 can separately display size information for each of the grouped imaging studies, whether or not there is a statistically significant difference between their sizes.

[0080] Similar to other seen elements described herein, the first grouped seen element 806 can include an expansion element 812 that, when selected or triggered as selected (e.g., via user selection of the expansion element 528 of the first seen element 524 of the first seen viewport 512), triggers display of additional information 810 of the object. The additional information 810 can directly correspond to the additional information 530 of the first image to allow comparison between the object in the first image and the object in the grouped image.

[0081] When the third GUI 500 includes a grouped imaging study, as shown in Figure 8 the display image change element 516 can allow the user to switch between displays of the imaging study while maintaining the grouping selected by the user. For example, selection of the display of the previous study can trigger display of the fourth image with the second and third images of the grouping. Thus, the grouping of images can group the grouped images such that the grouped images are indicated to the system as one study for display purposes.

[0082] When the third GUI 500 includes a grouped imaging study, as shown in Figure 8 the user can switch to a different GUI configuration, such as the grid configuration shown in Figure 2 or the horizontal configuration shown in Figures 3 to 4 In some examples, when the user switches to a different GUI configuration, the grouped study can remain grouped in the display. In other examples, when the user switches to a different GUI configuration, the grouped study can be split apart and displayed individually. As one example, when switching from the comparison configuration of the third GUI 500 to the horizontal configuration of the second GUI 300, the grouped study can remain grouped, and the display of the horizontal configuration can be adjusted to accommodate display of two image viewports within the size of one of the other image viewports. In this way, the system as presented herein can adapt to various image viewport presentations based on user interaction. As another example, when switching from the comparison configuration of the third GUI 500 to the grid configuration of the first GUI 200, the grouped study can be split apart and displayed individually. When the grouped study is displayed individually, such as in the grid configuration, the grouping can be visually displayed via shared identification icons and identification border colors. The splitting of the grouped study can not change the grouping of the study within the memory of the stored memory, such that when switching back to the comparison configuration, the study can again be displayed as grouped.

[0083] The presented GUIs thereby allow the user to selectively switch between various configurations of the GUIs as presented herein based on desired use. For example, the comparison configuration can allow direct comparison of two selected images and corresponding seen, while Figure 3 and Figure 4The horizontal configuration shown can allow for longitudinal comparison of images and findings over a longer period of time. Thus, as one non-limiting example, the horizontal configuration can allow a user to view changes in the size of a lesion from the earliest acquired image to the most recently acquired image, while the comparison configuration can allow a user to view changes in the size of a lesion from the most recently acquired image to a specifically selected previously acquired image. Each configuration can have clinical uses for different users, and thus the system provided herein can be tailored for different use case scenarios.

[0084] Each of the available GUI configurations can be defined as a default by a user. For example, a radiologist who can desire to focus on lesion size in directly compared images in order to indicate in their findings whether a lesion is increasing in size, decreasing in size, or remaining stable compared to a most recent prior image can select the comparison configuration as a default. However, an oncologist who can desire to focus on the overall evolution of images and findings, such as lesion size, in order to determine a next course of treatment can select the horizontal configuration as a default. The system and GUI provided herein can thus be flexible and adaptable to various user preferences.

[0085] Turning now to Figure 9 , a flowchart illustrating a method 900 for displaying a multi-date imaging GUI is shown. The method 900 can be performed using the systems and components described above with respect to Figure 1 , for example. The method 900 can be performed according to instructions stored in the memory of one or more processors of a computing device, such as the computing device 102, which is communicatively coupled to a display device configured to display a GUI, such as the first GUI 200, the second GUI 300, and / or the third GUI 500.

[0086] At 902, the method 900 includes obtaining one or more images from a database. The one or more images can be specific to a patient and can include various types of rendered image data, such as MPRs and / or MIPs, as well as image data of various imaging modalities, such as CT, MRI, x-ray, PET, etc. In some examples, the one or more images can be stored in and retrieved from the database in DICOM format. In some examples, the one or more images can be comparable images. For example, a user can define a body region and all comparable images of data including the body region. As one example, a user can define an abdomen as a body region, and comparable images, such as an abdominal CT, an abdominal MRI, an abdominal PET scan, etc. can be obtained from the database. Other methods for selecting metrics for defining comparable images can be used, for example a reference study, such as a most recently acquired study, can be defined, and other images can be compared to the reference study in order to define which studies are comparable.

[0087] The obtained one or more images can be time-ordered based on the acquisition data, as indicated at 904. Time-ordering the one or more images can define longitudinally adjacent images. Time-ordering the one or more images can thus define a next image and a previous image relative to each of the one or more images, as well as a current image (e.g., the most recently acquired image) and a reference image (e.g., the earliest acquired image of the one or more images).

[0088] At 906, the method 900 includes obtaining annotation data and seen data for each of the one or more images. The data for the one or more images can include annotation data information (including acquisition date, image series, image phase, fusion percentage, various image view types and / or rendering modes, orientation type, and patient orientation for each image view type), as well as scan data (including window width, window level, radiotracer / contrast concentration, display field of view (DFOV), etc.). Additionally, the data for the one or more images can include seen data, including data for the one or more objects. As one example, one or more segmentation protocols can be performed when the seen data is obtained to define the presence and data of the one or more objects within each of the one or more images. The seen data can include individual size data, SUV data, etc., for each object within each of the one or more images. For example, the size data can include a maximum radius, a minimum radius, etc., as well as comparative size data across longitudinally adjacent images for each object, such as a percentage change amount between longitudinally adjacent images.

[0089] At 908, the method 900 includes determining a configuration for the image viewport and the seen viewport of the GUI. The GUI can have available preset configurations, such as a grid configuration (e.g., a first configuration of Figure 2 a horizontal configuration (e.g., a second configuration of Figures 3 to 4 a comparison configuration (e.g., a third configuration of Figures 5 to 8 The computing device can have a predetermined default configuration stored in memory. In some examples, the predetermined default configuration, such as the grid configuration, can be the configuration determined for the GUI. In other examples, a user can define preferences within the system for their profile, including a default configuration preference. If a default configuration preference is defined for the user, the configuration determined at 908 can be the preferred default configuration defined by the user. In yet other examples, to determine the configuration, the user can select one of the available configurations.

[0090] At 910, method 900 includes displaying a GUI having one or more image viewports and one or more corresponding viewed viewports according to a determined configuration. As noted above, the determined configuration may be a grid configuration, a horizontal configuration, or a comparison configuration. One or more images may be displayed within the GUI according to the configuration, as noted at 912, and the image view of each of the displayed images may be displayed according to the configuration, as noted at 914.

[0091] As an example, when the determined configuration is a grid configuration, displaying one or more images in a viewport may include an imaging viewport that displays each of the one or more images in a grid arrangement, thereby displaying the images in rows from left to right based on the temporal order of the images, as per [reference to...]. Figure 2 Described. In a grid configuration, the viewport that displays the seen information of the selected image viewport can be displayed as a side panel of the image viewport's grid. When the determined configuration is a horizontal configuration, the image viewport can be displayed in a single row from left to right in chronological order across the first part of the GUI, and the corresponding viewport can be displayed in a single row across the second part of the GUI, as described above. Figures 3 to 4 The viewport described can directly correspond to the image viewport and can be aligned in the display as shown. When the determined configuration is a comparison configuration, the two image viewports can be displayed horizontally adjacent to each other, where the displayed images can default to the two most recently acquired images, or alternatively, images selected by the user, as described in the section on... Figure 5 Described. In a comparison configuration, the viewports corresponding to two image viewports can be displayed horizontally adjacent to each other. Information displayed in the first viewport may correspond to the image displayed in the first image viewport, and information displayed in the second viewport may correspond to the image displayed in the second image viewport. (See reference...) Figure 10 As further described, user interaction with the GUI can be modified by changing the configuration to one of the other available configurations, pinning a study, grouping studies, and / or hiding / unhiding studies.

[0092] The GUI presented herein can therefore display multiple comparable images acquired across multiple dates within a single window. Thus, users can visualize objects within multiple images in a comparable manner without having to individually open each image within the PACS application and individually resize each application window. Additionally, the GUI can be configured to automatically adjust the size of images and viewports to fit the GUI window of a given GUI configuration. For example, a grid configuration can display a number of viewports based on the number of comparable images acquired and can proportionally resize image viewports in response to changes in the GUI window size and / or in response to user interactions that cause the viewports to close.

[0093] Turn nowFigure 10 FIG. 13 shows an example flow diagram illustrating a method for modifying a display of a multi-image GUI. The method 1300 can be performed using the systems and components described above with respect to FIGS. 1-12. For example, the method 1300 can be performed according to instructions stored in the memory of one or more processors of a computing device, such as the computing device 102, that is communicatively coupled to a display device configured to display a GUI, such as the first GUI 200, the second GUI 300, and / or the third GUI 500. It should be appreciated that while the steps of the method 1300 are presented in a particular order, the modification of a GUI as described herein can be performed in any reasonable or practicable order. Figure 1

[0094] At 1002, the method 1000 includes displaying a GUI having an image viewport and a seen viewport according to a first GUI configuration. As described above, the GUI can display a plurality of images acquired over a plurality of dates, and in some cases with multiple modalities within the same window. The first GUI configuration can be a default configuration defined by the system or according to user preferences. The first GUI configuration can be one of a grid configuration, a horizontal configuration, and a comparison configuration.

[0095] At 1004, the method 1000 includes determining whether a first GUI configuration change is requested. A change in GUI configuration can be requested by a user by user selection of an element within the GUI. For example, when the first configuration is a grid configuration, the GUI can include a horizontal configuration element and a comparison configuration element. User selection of the comparison configuration element can indicate to the system that a change from the grid configuration to the comparison configuration is requested. If a GUI configuration change is requested, the method 1000 proceeds to 1006. If a GUI configuration change is not requested, the method 1000 proceeds to 1008 to continue displaying the GUI according to the first GUI configuration. After 1008, the method 1000 can proceed to 1010.

[0096] At 1006, the method 1000 includes displaying the GUI having the image viewport and the seen viewport according to a second GUI configuration. The second GUI configuration can be the GUI configuration selected within the GUI when a configuration change was indicated in the first configuration. As an example, the first GUI configuration can be a grid configuration, and the second GUI configuration can be a comparison configuration.

[0097] At 1010, the method 1000 includes determining whether an imaging study is pinned. When 1010 follows 1008, the first GUI configuration can be a comparison configuration. When 1010 follows 1006, the second GUI configuration can be a comparison configuration. An imaging study can be pinned when a user selects a pin element corresponding to a displayed image. As described with respect to FIG. 12, the pin element can be a pin element of the grid configuration, a pin element of the horizontal configuration, or a pin element of the comparison configuration. Figure 5 ​The pinned element described can be included within the caption of the viewports of the images. If the imaging study is pinned, the method 1000 proceeds to 1012. If the imaging study is not pinned, the method 1000 proceeds to 1014 to continue displaying the GUI in the comparison configuration. After 1014, the method 1000 proceeds to 1016.

[0098] At 1012, the method 1000 includes modifying the GUI to pin the selected imaging study. As described with respect to Figure 6 The pinned study described can pin the images within a given image viewport and pin the views thereof within the corresponding viewports. When the user changes which images are displayed within the GUI, for example, via a display image change element, the pinned study can not be removed from display and the images within the second image viewport and the views thereof can be shifted in response to user input, thereby allowing the user to compare the selected study to different selected studies.

[0099] At 1016, the method 1000 includes determining whether the imaging studies are grouped. Two or more imaging studies can be grouped together based on user selection of a grouping element within the viewports of the selected images, as described with respect to Figure 7 The user can indicate which study or studies are to be grouped with the selected study. If two or more studies are to be grouped, the method 1000 proceeds to 1018. If no grouping is to occur, the method 1000 proceeds to 1020 to continue displaying the GUI without the grouped studies. After 1020, the method 1000 proceeds to 1022.

[0100] At 1018, the method 1000 includes modifying the GUI to group the selected imaging studies. As described with respect to Figure 8 The modified GUI with the grouped studies can include a plurality of grouped image viewports and a grouped viewports indicating the view of one of the grouped images, the views of two of the grouped images individually in different instances or simultaneously in the same instance, or the view of an average of the grouped images, as described with respect to

[0101] At 1020, the method 1000 includes determining whether a second GUI configuration change is requested. Changes in the GUI configuration can be requested by the user by user selection of elements within the GUI. For example, when the second configuration is the comparison configuration, the GUI can include a horizontal configuration element and a grid configuration element. User selection of the horizontal configuration element can indicate to the system that a change from the comparison configuration to the horizontal configuration is requested. If a GUI configuration change is requested, the method 1000 proceeds to 1024. If no GUI configuration change is requested, the method 1000 proceeds to 1026 to continue displaying the GUI as is (e.g., according to the first configuration or the second configuration). After 1026, the method 1000 can proceed to 1028.

[0102] At 1024, the method 1000 includes displaying the GUI with the image viewport and the seen viewport according to a third GUI configuration. The third GUI configuration can be the GUI configuration selected within the GUI when the configuration change was indicated in the second configuration. As an example, the second GUI configuration can be the comparison configuration, and the third GUI configuration can be the horizontal configuration.

[0103] At 1028, the method 1000 includes determining whether the study is hidden. When 1028 follows 1026, the first GUI configuration or the second GUI configuration can be the horizontal configuration, depending on the step previously taken. When 1028 follows 1024, the third GUI configuration can be the horizontal configuration. The study can be hidden in response to a user selection of a hidden element. The hidden element can be displayed within a caption of the seen viewport of a given imaging study, and selection of the hidden element can indicate that the seen viewport and the image viewport of the given imaging study are to be hidden. If the study is hidden, the method 1000 proceeds to 1030. If the study is not hidden, the method 1000 proceeds to 1032 to continue displaying the GUI without hiding the study.

[0104] At 1030, the method 1000 includes displaying the GUI with the study hidden. Displaying the GUI with the study hidden can include altering the display size of the remaining image viewports to fill the GUI window with the study hidden.

[0105] The modifications and interactions with the GUI described herein are exemplary in nature. The systems disclosed herein also provide other actions, including shifting as to which images are displayed, switching between different configurations, interacting with displayed images including image annotations, selecting images, selecting objects within the seen viewport, etc.

[0106] The technical effect of the systems and methods provided herein is to display multiple images within a single GUI, thereby reducing user interaction with the system (e.g., number of clicks, manually resizing the image window, etc.) and saving user time. Additionally, processing power can be reduced, as all comparable studies can be obtained at once, thereby reducing the number of times the database is accessed as compared to the user needing to individually open each image they wish to view. Additionally, the GUI as presented herein can provide information of the findings within the comparable images, thereby allowing the user to easily track the evolution of a lesion over time. Thus, the systems described herein provide a specific improvement in the field of medical systems by allowing the user to view multiple comparable images within the same application, thereby reducing the need for the user to individually open each image to compare these images. As one example, the user can launch the GUI in a horizontal configuration that displays all of the comparable images and their findings in a portrait configuration ordered by time, thereby allowing the user to visualize the object within the displayed images over time and observe the measured size and / or calculated SUV of the object over time.

[0107] Thus, via the systems disclosed herein, comparable images and their findings information can be displayed in an easily visually resolved manner with a reduced amount of time. Patient information including image and findings data can be displayed via small graphical elements with minimal text, which can allow a greater amount of information to be immediately displayed to the user. The displayed data can be stored in various repositories, including PACS, EMRs, etc., and via the GUIs disclosed herein, patient information can be aggregated and displayed for easier viewing. This can reduce cognitive overload and aid in clinical thinking, as patient data is structured in a clinically helpful manner and the need for the user to open images and data, resize the images and data, and otherwise manipulate the images and data is reduced.

[0108] The present disclosure also provides support for a computing device including a display screen configured to display a plurality of image viewports within a multi-image graphical user interface (GUI) on the display screen, the plurality of image viewports displaying respective medical images of a patient, and additionally configured to display one or more finding viewports within the GUI, each finding viewport displaying finding data corresponding to an image displayed within one of the plurality of image viewports, wherein the finding data for each respective medical image of the patient is obtained from a database when the database is in an unactivated state, and wherein the respective medical images are comparable images of one or more imaging modalities. In a first example of the system, the GUI is displayed on the display screen in one of a grid configuration, a comparison configuration, and a horizontal configuration. In a second example of the system, optionally including the first example, in the grid configuration, the plurality of image viewports are arranged in a grid and the images are displayed longitudinally according to a temporal order, and the one or more finding viewports include a finding viewport corresponding to a selected image viewport of the plurality of image viewports. In a third example of the system, optionally including one or both of the first example and the second example, in the comparison configuration, the plurality of image viewports include a first image viewport and a second image viewport, and the one or more finding viewports include a first finding viewport corresponding to the first image viewport and a second finding viewport corresponding to the second image viewport, wherein finding data for a first image displayed within the first image viewport is displayed within the first finding viewport, and finding data for a second image displayed within the second image viewport is displayed within the second finding viewport. In a fourth example of the system, optionally including one or more or each of the first example through the third example, in the horizontal configuration, the plurality of image viewports are arranged side-by-side along a first portion of the GUI, and the one or more finding viewports are arranged side-by-side along a second portion of the GUI, wherein each of the finding viewports corresponds to one of the plurality of image viewports and is vertically aligned therewith to display finding data for an image displayed within the corresponding image viewport. In a fifth example of the system, optionally including one or more or each of the first example through the fourth example, the GUI includes one or more selectable elements configured to switch between the grid configuration, the comparison configuration, and the horizontal configuration. In a sixth example of the system, optionally including one or more or each of the first example through the fifth example, when in the comparison configuration, the GUI includes: a pin element configured to pin a given display image to the GUI when selected for the given display image; and a grouping element configured to trigger display of a pop-up menu listing available images to group the selected display image with when selected for the display image. In a seventh example of the system, optionally including one or more or each of the first example through the sixth example, each finding viewport includes one or more finding elements, each of the one or more finding elements displaying information for an object of the corresponding medical image.

[0109] The present disclosure also provides support for a method for displaying a plurality of images in a graphical user interface (GUI), the method comprising: obtaining one or more medical images and their findings data from a database; time-sorting the obtained one or more medical images and their findings data; and displaying the one or more medical images in respective image viewports of the GUI and the findings data of at least one of the one or more medical images in at least one findings viewport of the GUI, wherein the display of the respective image viewports and the at least one findings viewport is based on a selected GUI configuration, wherein the one or more medical images are comparable images of one or more imaging modalities. In a first example of the method, the image viewports include a first image viewport corresponding to a first findings viewport and a second image viewport corresponding to a second findings viewport, wherein the first image viewport and the first findings viewport correspond to a first color and the second image viewport and the second findings viewport correspond to a second, different color. In a second example of the method, optionally including the first example, the method further comprises modifying the findings data displayed within the at least one findings viewport in response to a user selection of one of a next element and a previous element when the selected GUI configuration is one of a grid configuration and a comparison configuration, wherein the next element triggers display of data corresponding to a next, later-acquired image and the previous element triggers display of data corresponding to a next, earlier-acquired image. In a third example of the method, optionally including one or both of the first example and the second example, each of the respective image viewports is selectable and, when the selected GUI configuration is one of the grid configuration and the comparison configuration, selection of one of the next element and the previous element selects a corresponding image viewport displaying one of the next, later-acquired image and the next, earlier-acquired image, respectively. In a fourth example of the method, optionally including one or more or each of the first example through the third example, the method further comprises pinning an image to the GUI in response to a user selection of a pin element corresponding to the image when the selected GUI configuration is the comparison configuration, wherein the pinned image is not removed from display when one or both of the next element and the previous element are selected while the image is pinned. In a fifth example of the method, optionally including one or more or each of the first example through the fourth example, the method further comprises removing an image viewport and a corresponding findings viewport from the GUI in response to a user selection of a hide element corresponding to the corresponding findings viewport when the selected GUI configuration is the horizontal configuration. In a sixth example of the method, optionally including one or more or each of the first example through the fifth example, each findings viewport includes one or more findings elements, each findings element corresponding to an object identified within one or more of the one or more medical images.In a seventh example of the method, optionally including one or more or each of the first through sixth examples, the selected GUI configuration is one of a predefined default configuration, a user preference default configuration, or a configuration selected via user input.

[0110] The present disclosure also provides support for a system comprising: one or more imaging systems configured to acquire medical imaging data of a patient, wherein the one or more imaging systems are coupled to a database configured to store the medical imaging data and to a computing device configured with instructions stored in a non-transitory memory executable by a processor that, when executed, cause the processor to: display a graphical user interface (GUI) comprising one or more image viewports and one or more seen viewports, wherein each image viewport displays a medical image of the medical imaging data and each seen viewport displays seen data corresponding to a respective medical image displayed within one of the one or more image viewports, wherein the one or more image viewports and the one or more seen viewports are arranged within the GUI according to a first GUI configuration; display one or more seen elements within each of the one or more seen viewports, wherein each of the one or more seen elements corresponds to an object of the medical imaging data, wherein the one or more seen elements display information related to the object including a percent change in volume of the object; and modify the GUI from the first GUI configuration to a second GUI configuration in response to user input, wherein in the first GUI configuration a set of comparable images are displayed in respective image viewports and in the second GUI configuration a subset of the set of comparable images are displayed in respective image viewports. In a first example of the system, each of the one or more seen elements are selectable elements that, when selected, highlight the corresponding object within each respective displayed medical image. In a second example of the system, optionally including the first example, user selection of an image slice displayed within a selected image viewport triggers display of the comparable image slice within each of the other image viewports. In a third example of the system, optionally including one or both of the first and second examples, each of the one or more seen elements includes one or more metrics including a percent change in the one or more metrics compared to a reference image.

[0111] As used herein, an element or step recited in the singular and preceded with the word "a" or "an" should be understood as not excluding plural of said elements or steps, unless explicitly stated that such exclusion applies. Furthermore, references to "one embodiment" of the present application are not intended to be interpreted as excluding the existence of additional embodiments that also incorporate those features. Moreover, unless explicitly stated otherwise, embodiments "comprising" or "including" an element or a plurality of elements having a particular property are intended to include additional such elements not having that property. The terms "including" and "comprising" are used synonymously with the corresponding terms "containing" and "including", respectively. Furthermore, the terms "first", "second", and "third" and the like, merely denote different categories, and do not require or imply a particular order or sequence, unless explicitly stated otherwise.

[0112] This written description uses examples to disclose the application, including the best mode, and also to enable any person skilled in the art to practice the application, including making and using any devices or systems and performing any incorporated methods. The patentable scope of the application is defined by the claims, and can include other examples that occur to those skilled in the art. Such other examples are intended to fall within the scope of the claims if they have structural elements that do not differ from the literal language of the claims, or if they include equivalent structural elements with insubstantial differences from the literal languages of the claims.

Claims

1. A computing device (102) including a display screen (134), the computing device being configured to display a plurality of image viewports (204) within a multi-image graphical user interface (GUI) (200) on the display screen, the plurality of image viewports displaying corresponding medical images of a patient, and additionally configured to display one or more viewing viewports (206) within the GUI, each viewing viewport displaying viewing data corresponding to an image displayed in one of the plurality of image viewports, wherein the viewing data for each corresponding medical image of the patient is obtained from a database (150) when the database is in an inactive state, and wherein the corresponding medical image is a comparable image of one or more imaging modalities.

2. The computing device of claim 1, wherein the GUI (200) is displayed on the display screen (134) in one of a grid configuration, a comparison configuration, and a horizontal configuration.

3. The computing device according to claim 2, wherein, In the grid configuration, the plurality of image viewports (204) are arranged in a grid and the images are displayed longitudinally in chronological order, and the one or more viewports (206) include a viewport corresponding to a selected image viewport among the plurality of image viewports.

4. The computing device according to claim 2, wherein, In the comparison configuration, the plurality of image viewports (504) include a first image viewport (506) and a second image viewport (508), and the one or more viewing viewports (510) include a first viewing viewport (512) corresponding to the first image viewport and a second viewing viewport (514) corresponding to the second image viewport, wherein the viewing data of the first image displayed in the first image viewport is displayed in the first viewing viewport, and the viewing data of the second image displayed in the second image viewport is displayed in the second viewing viewport.

5. The computing device according to claim 2, wherein, In the horizontal configuration, the plurality of image viewports (304) are arranged side by side along a first portion of the GUI (300), and the one or more viewports (306) are arranged side by side along a second portion of the GUI, wherein each of the viewports corresponds to one of the plurality of image viewports and is vertically aligned with it to display the seen data of the image displayed within the corresponding image viewport.

6. The computing device of claim 2, wherein the GUI (200, 300, 500) includes one or more selectable elements (252, 254) configured to switch between the grid configuration, the comparison configuration, and the horizontal configuration.

7. The computing device according to claim 2, wherein, When in the comparison configuration, the GUI (500) includes: a pinning element (522) that pins the given display image to the GUI when selected for a given display image; and a grouping element (706) that, when selected for display images, triggers the display of a pop-up menu listing available images for grouping the selected display image.

8. The computing device of claim 1, wherein each viewport includes one or more view elements (524), each of the one or more view elements displaying information of an object (246) corresponding to a medical image.

9. A method for displaying multiple images in a graphical user interface (GUI), the method comprising: Obtain one or more medical images and their data from the database (902); Time-sorting of one or more medical images and their observed data (904); and The one or more medical images are displayed in the corresponding image viewport of the GUI and the seen data of at least one of the one or more medical images is displayed in at least one seen viewport of the GUI (910), wherein the display of the corresponding image viewport and the at least one seen viewport is based on a selected GUI configuration (908), wherein the one or more medical images are comparable images of one or more imaging modalities.

10. The method according to claim 9, wherein the image viewport includes a first image viewport (506) corresponding to a first viewing port (512) and a second image viewport (508) corresponding to a second viewing port (514), wherein the first image viewport and the first viewing port correspond to a first color, and the second image viewport and the second viewing port correspond to a second different color.

11. The method of claim 10, further comprising, when the selected GUI configuration is one of the grid configuration and the comparison configuration, modifying the seen data displayed in the at least one seen viewport in response to a user selection of one of the next element (234) and the previous element (236), wherein the next element triggers the display of data corresponding to the next later acquired image, and the previous element triggers the display of data corresponding to the next earlier acquired image.

12. The method of claim 11, wherein each of the respective image viewports is selectable, and when the selected GUI configuration is one of the grid configuration and the comparison configuration, the selection of one of the next element (234) and the previous element (236) respectively selects to display the corresponding image viewport of one of the next later acquired image and the next earlier acquired image.

13. The method of claim 11, further comprising, when the selected GUI configuration is the comparison configuration, pinning the image to the GUI in response to a user selection of a pinning element (522) corresponding to the image, wherein, When the next element (234) and the previous element (236) or both are selected while the image is pinned, the pinned image is not removed from the display.

14. The method of claim 10, further comprising, when the selected GUI configuration is a horizontal configuration, removing the image viewport and the corresponding viewport from the GUI in response to a user selection of a hidden element corresponding to the corresponding seen viewport (322).

15. The method of claim 9, wherein each viewed viewport includes one or more viewed elements, each viewed element corresponding to an object identified within one or more medical images in the one or more medical images (246).

Images