Techniques for updating graphical user interfaces

Abstract

The disclosure relates to techniques for updating graphical user interfaces. A processor-implemented method includes receiving user input from one or more user input devices, the user input corresponding to a selected element of a displayed graphical user interface hijacked by one or more software having functionality provided by an undisplayed interface. The processor-implemented method also includes determining one or more corresponding target functions of the non-displayed interface based on the selected element; executing the one or more corresponding target functions via the one or more software hijacks; and displaying one or more changes to the displayed graphical user interface based on the execution of the one or more corresponding target functions.

Description

Technical Field

[0001] The subject matter disclosed herein relates to medical imaging devices, and more particularly to techniques for updating a graphical user interface of a medical device in order to change or improve an interface or appearance associated with a pre-existing or outdated interface. Background Art

[0002] The subject matter discussed in this section should not be assumed to be prior art simply because it is mentioned in this section. Similarly, issues mentioned in this section or related to the subject matter provided as background should not be assumed to have been previously recognized in the prior art. The subject matter in this section merely represents different approaches, which themselves may also correspond to specific implementations of the claimed technology.

[0003] Medical imaging devices may be used to image anatomical targets, such as organs and soft tissues in the human body, as well as non-human targets, such as packages or luggage, manufactured items undergoing quality control, etc. For example, in addition to ultrasound imaging of humans, animals, etc., medical imaging devices may also be used for applications such as ultrasound / acoustic sensing, nondestructive evaluation (NDE), ultrasound therapy (e.g., high-intensity focused ultrasound (HIFU)), etc.

[0004] Imaging equipment (such as medical imaging equipment) may include a user interface (UI) to facilitate the specific implementation or configuration of imaging protocols or sessions, the observation of real-time or previously acquired image data and / or the accurate and effective diagnosis and treatment of patients. The UI may present, for example, various imaging modalities (ultrasound, X-ray), patient information and control options to clinicians. In addition, the UI may include various graphic components, including layouts, windows, buttons, icons and menus that may facilitate the navigation of the UI. Over time, and as technology changes, UI design, layout or selectable option feature sets may become obsolete or otherwise unsuitable or different from other types of interfaces utilized by clinicians. However, due to the limitations of existing software functionality, the costs associated with specific implementations, etc., the update of such existing imaging systems may be difficult to achieve. Summary of the invention

[0005] The embodiments disclosed herein are not intended to limit the scope of the claimed subject matter, but rather these embodiments are intended only to provide a brief overview of possible embodiments. In fact, the present disclosure may include a variety of forms that may be similar to or different from the embodiments described below.

[0006] In one embodiment, a processor-implemented method includes receiving user input from one or more user input devices, the user input corresponding to a selected element of a displayed graphical user interface having one or more software hijacks of functionality provided by a non-displayed interface. The processor-implemented method also includes: determining one or more corresponding target functions of the non-displayed interface based on the selected element; executing the one or more corresponding target functions via the one or more software hijacks; and displaying one or more changes to the displayed graphical user interface based on the execution of the one or more corresponding target functions.

[0007] In another embodiment, a computer-readable medium includes processor-executable code that, when executed by a processor, causes the processor to receive user input from one or more user input devices, the user input corresponding to a selected element of the displayed graphical user interface having one or more software hijacks of functionality provided by a non-displayed interface. The computer-readable medium also includes processor-executable code that, when executed by the processor, causes the processor to: determine one or more corresponding target functions of the non-displayed interface based on the selected element; execute the one or more corresponding target functions via the one or more software hijacks; and display one or more changes to the displayed graphical user interface based on the execution of the one or more corresponding target functions.

[0008] In yet another embodiment, a system includes a first computing device configured to: display a graphical user interface; receive user input from one or more user input devices, the user input corresponding to a selected element of the displayed graphical user interface having one or more software hijacks of functionality provided by an undisplayed interface; and send an indication of the selected element. The system also includes a second computing device configured to: receive the indication of the selected element; determine one or more corresponding target functions of the undisplayed interface based on the selected element; instruct a third computing device to execute the one or more corresponding target functions via the one or more software hijacks; receive output associated with the execution of the one or more corresponding target functions; and instruct the first computing device to change one or more elements of the displayed graphical user interface based on the output. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] These and other features, aspects and advantages of the present invention will be better understood when the following detailed description is read with reference to the accompanying drawings, in which like characters refer to like parts throughout the several views, and in which:

[0010] Figure 1 is a pictorial representation of an imaging system, such as a CT imaging system, according to aspects of the present disclosure;

[0011] Figure 2 According to various aspects of the present disclosure Figure 1 A block diagram of the imaging system in FIG.

[0012] Figure 3 is a block diagram of an updated graphical user interface system according to an embodiment of the present disclosure;

[0013] Figure 4 According to the embodiments of the present disclosure, Figure 1 An illustration of an updated graphical user interface of a medical imaging device of an imaging system of the present invention;

[0014] Figure 5 is a block diagram illustrating communication between a web browser and a target application of an updated graphical user interface system according to an embodiment of the present disclosure;

[0015] Figure 6 is a block diagram illustrating communications between components of an updated graphical user interface system using a server according to an embodiment of the present disclosure;

[0016] Figure 7 is a block diagram illustrating communications between multiple web browsers according to an embodiment of the present disclosure;

[0017] Figure 8 is a flow chart of a method for implementing an updated graphical user interface system according to an embodiment of the present disclosure;

[0018] Fig. 9 is a flowchart of a method for launching and updating an updated graphical user interface according to an embodiment of the present disclosure; and

[0019] Fig.10 is an illustration of a client-server architecture for medical image processing and display in accordance with an embodiment of the present technology. DETAILED DESCRIPTION

[0020] One or more specific embodiments will be described below. In order to provide a concise description of these embodiments, not all features of an actual implementation are described in the specification. It should be understood that in the development of any such actual implementation, as in any engineering or design project, many implementation-specific decisions must be made to achieve the developer's specific goals, such as complying with system-related and business-related constraints that may vary from implementation to implementation. In addition, it should be understood that such development efforts may be complex and time-consuming, but are still routine tasks for design, fabrication, and manufacturing for ordinary technicians who benefit from this disclosure.

[0021] Any example or illustration given herein should not be considered as a constraint, restriction or clear definition of any one or more terms utilized by it. On the contrary, these examples or illustrations should be considered as describing about various specific embodiments and are only exemplary. It will be understood by those of ordinary skill in the art that any one or more terms used by these examples or illustrations will cover other embodiments that may or may not be provided together with them or provided elsewhere in this specification, and all such embodiments are intended to be included in the scope of the one or more terms. The language specifying such non-limiting examples and illustrations includes but is not limited to: "for example", "for example", "such as", "such as", "including", "in certain embodiments", "in some embodiments" and "in one embodiment".

[0022] This written description uses examples to disclose the invention, including the best mode, and also to enable any person skilled in the art to practice the invention, including making and using any devices or systems and performing any included methods. The patentable scope of the invention is defined by the claims, and may include other examples that occur to those skilled in the art. Such other examples are intended to fall within the scope of the claims if they have structural elements that do not differ from the literal language of the claims, or if they include equivalent structural elements with insignificant differences from the literal language of the claims.

[0023] For decades, medical imaging devices have been used to non-invasively acquire image data of a subject's internal structures or physiological processes, thereby allowing appropriate medical diagnosis and care to be performed without harming the subject. Examples of such medical imaging techniques include X-ray radiography, computed tomography (CT), magnetic resonance imaging (MRI), positron emission tomography (PET), single photon emission computed tomography (SPECT), mammography, ultrasound, etc. Traditionally, such images can be evaluated (i.e., read) by trained clinicians (such as radiologists), and advances in medical imaging technology have allowed such images to be electronically acquired, processed, accessed, and viewed via a graphical user interface.

[0024] Graphical user interfaces (GUIs) for medical imaging devices can provide several benefits to clinicians, and therefore can be conducive to improving patient care. Graphical user interfaces can provide the benefit of ease of use, allowing clinicians to control medical imaging devices through buttons, sliders, and other visual elements that are directly interpreted and easy to manipulate or otherwise use. Clinicians can, for example, manipulate or otherwise control imaging parameters (such as default or preset imaging protocols, contrast levels, and exposure settings) to customize imaging processes for the patient's specific conditions or for the patient's body type. In addition, graphical user interfaces can provide real-time visualization during the imaging process, so that clinicians can make immediate (i.e., instant) adjustments as needed. Graphical user interfaces can also allow clinicians to edit or annotate medical images, such as once the image is acquired, highlighting the region of interest in the image. In addition, during the image acquisition process or during the image examination process, the graphical user interface can integrate and display patient information (such as data from electronic medical records), and the patient information can be presented together with the medical image.

[0025] Due to various factors, the life of an imaging system can be many years, or even decades. Such factors may include, but are not limited to, the cost or capital investment in the imaging system, the training invested in the use of such a system, the actual physical installation and coverage area of ​​such an imaging system, and correspondingly, the facility modifications that have been made to install the imaging system or that may be required to replace such a system. Therefore, an imaging system that is available and useful can be used for many years. Although such an imaging system may still have a production life, in practice, certain aspects related to the use of the imaging system (such as a graphical user interface or other user interface) may become obsolete and inconsistent with more modern interfaces. For example, interface elements for older imaging systems may utilize non-optimal layouts of screen elements, such as the size, placement and / or type of selectable controls provided to the user. Similarly, the placement, size setting and / or manipulation of the image display area may be non-optimal, considering current display or viewing technology or conventions. In addition, current interface technology may use dynamic previews or other real-time interactive elements that do not exist in older interfaces.

[0026] In view of the foregoing, it may be desirable to update the medical imaging graphical user interface to improve aesthetics, accessibility, visibility, or clarity, or otherwise make a given imaging system graphical user interface conform to modern standards or design practices. However, these updates may prove to be technically challenging. For example, an updated graphical user interface may be required to integrate with existing medical imaging functionality, applications, modalities, device control functions, etc., while avoiding disruption to existing systems and clinicians' workflows. An updated graphical user interface may also need to work seamlessly with a variety of imaging devices, display devices, and communication protocols. In addition, an updated graphical user interface may need to be thoroughly tested for performance issues so that it meets the standards of clinicians, regulatory agencies, etc. Therefore, updating a medical imaging graphical user interface using traditional methods may be complex and may prove resource demanding in terms of money, time, and personnel.

[0027] Provided herein are techniques for updating a graphical user interface (such as a graphical user interface utilized by a medical imaging system) while maintaining underlying application processes using interactive hijacking and masking techniques. As used herein, when referring to a user interface (such as a graphical user interface), the term "updated" may be understood to mean a version that is different in appearance, presentation, or aesthetics, providing functionality that may be the same as or mapped to pre-existing functionality of an existing or previous interface (which may be a graphical user interface or other interface). As discussed herein, such an updated interface may be presented or displayed in place of an existing or previous interface. However, the existing or previous interface continues to execute in the background or in some other manner that is not visible or displayed to the user, such that interaction with the functionality of the existing or previous interface is performed via the updated graphical user interface.

[0028] The disclosed technology may include overlaying an updated user interface on a target user interface (e.g., an existing user interface) so that the updated user interface seamlessly accesses the target function of a target application (e.g., an existing application). The updated user interface may access the target function through hijacking technology. As used herein, the term "hijacking" may be understood to mean a software component that processes intercepted function calls, software events, etc. The hijacking technology described herein may include, for example, intercepting user input to the updated user interface by modifying an executable source or inserting an event hijacking at runtime of an operating system. The intercepted user input may be mapped to the target function via, for example, a server query, and then the target function may be executed. Finally, the output of the executed target function may affect the appearance changes in the updated user interface. Therefore, the updated user interface may access and execute existing functions of existing applications, and the existing functions may change in the updated user interface.

[0029] The updated user interface may include or be implemented as a multi-platform web page, and thus may be accessed across various mobile, desktop or web-based operating systems. Additionally or alternatively, the updated user interface may be locally contained and may include dedicated local storage media, processing capabilities, memory, etc. As mentioned herein, as an example, the updated user interface may be described as a mask, a new user interface, a web page, or a fat client. The communication between the updated user interface, the target user interface (i.e., the original user interface, or more generally, the user interface to be updated), the target application and the target function may be facilitated by a server (such as a web server). For example, a web server may respond to requests from client components (such as a web browser) and may manage the functionality of a web page, such as storing web pages, processing web pages, and providing web pages to users. For example, a server may receive input from an updated user interface, such as an indication of user input (e.g., mouse input) and / or a request for button feedback. In response to receiving the input, the server may map the input to the corresponding target function of the target application via hijacking. Therefore, hijacking may simulate the initiation of the corresponding target function, such as the selection of a visual element on the target user interface (e.g., button presses).

[0030] When executing the corresponding target function, the target application may send the output of the corresponding target function (e.g., button feedback) to the server. The server may then cause the updated user interface to reflect the output, such as by updating the visual elements of the multi-platform web page of the updated user interface. Therefore, the corresponding target function may affect changes in the updated user interface via the server. However, the changes in the updated user interface may present aesthetics different from those presented by the target (e.g., original) user interface in response to the output, may change different visual elements, etc. The above process may be described herein as the updated user interface "obscuring" the target user interface (e.g., as a "mask" of the target user interface), which may not be displayed to the user (i.e., not displayed) in some embodiments.

[0031] Taking into account the foregoing and referring to Figure 1 and Figure 2 , a CT imaging system 10 is shown by way of example. As can be appreciated, with respect to Figure 1 and Figure 2The disclosed and described CT imaging system 10 provides only a real-world example of one type of device that may be present in a healthcare environment and that may generate complex data (e.g., volumetric image data) of a living subject that may be processed and displayed for a clinician by a graphical user interface. Similarly, such a graphical user interface as described herein may be used to parameterize and operate such imaging systems of similar devices, such as to perform scans or other device-specific tasks. As may be appreciated, in a hospital or clinical environment, there may be a variety of types of such electronic devices (e.g., imaging devices, patient monitors, diagnostic equipment, therapeutic equipment, etc.), and they may each generate data or outputs that may be suitable for processing and display via a graphical user interface and / or that may be operated or configured by such a graphical user interface. With this in mind, Figure 1 and Figure 2 One such system is described to illustrate a real-world example of a possible system operating using such a graphical user interface in a specialized environment (such as one in which the system and interface are not frequently updated or replaced) and the complexities that may be involved in such a system and its output, which distinguishes such medical type systems from systems that might be found in an office or other non-medical environment.

[0032] With this in mind, and turning to the drawings, a CT imaging system 10 (e.g., a CT scanner) includes a gantry 12 having a housing 13 (e.g., a gantry housing) and rotating and stationary components. By way of example, the gantry 12 has an X-ray source 14 that projects a beam of X-rays 16 toward an X-ray detector assembly or X-ray detector array 15 (e.g., having a plurality of detector modules) on an opposite side of the gantry 12. The X-ray detector assembly 15 is coupled to a data acquisition system (DAS) 33.

[0033] The multiple detector modules of the X-ray detector assembly 15 detect the projected X-rays that pass through the patient or subject 22, and the DAS 33 converts the data into digital signals for subsequent processing. During scanning to acquire X-ray projection data, the gantry 12 and components mounted thereon rotate about a center of rotation 24 (e.g., an isocenter) to allow volumetric imaging. The rotation of the gantry 12 and the operation of the X-ray source 14 are controlled by a control mechanism 26. The control mechanism 26 includes an X-ray controller 28 that provides power and timing signals to the X-ray source 14 and a gantry motor controller 30 that controls the rotational speed and position of the gantry 12.

[0034] The computer 42 (separate from or as part of the CT imaging system 10) may perform a data correction unit and image reconstruction. For example, the computer 42 may receive sampled and digitized X-ray data from the DAS 33 and perform high-speed image reconstruction. The reconstructed image may be stored in the mass storage device 50 and / or displayed to the clinician via a graphical user interface. With respect to the presently described techniques, an updated graphical user interface may be implemented on or by the computer 42 and displayed to allow a user to operate the CT imaging system 10 and / or to examine, annotate, and manipulate images generated using the CT imaging system 10 (or similar system or device).

[0035] As can be appreciated, the imaging system 10 or similar imaging systems can be complex in nature and can benefit from ease of use that allows clinicians to control medical imaging devices through buttons, sliders, and other visual elements that are straightforward to interpret and easy to manipulate or otherwise use. These benefits can be provided by an updated medical imaging graphical user interface that allows clinicians to interface with the imaging system 10 in a more streamlined or more efficient manner than would be possible using a previous or existing version of the interface. Additionally, as described herein, updates to the medical imaging graphical user interface may be desired to improve aesthetics, accessibility, visibility, or clarity, or otherwise bring a given imaging system graphical user interface into compliance with modern standards or design practices. However, the complexity of imaging systems, such as the imaging system 10, may present challenges in updating a graphical user interface associated with the imaging system 10.

[0036] Considering the foregoing, Figure 3 1 is a block diagram of a system 10 including an updated user interface 19 that masks a target user interface 18, which in some embodiments may not be displayed (i.e., not displayed) even when executed. The target user interface 18 may be included as part of or used in conjunction with a target application 17, and the target application 17 may be managed by a target platform / operating system (OS) 11. In addition, the updated user interface 19 may accept input from a user (e.g., a clinician) via an input device 20.

[0037] The target platform / OS 11 may include an operating system associated with, for example, a medical imaging system. The target platform / OS 11 may include a proprietary operating system developed to manage medical imaging applications, or may include a commercially available (e.g., general-purpose) operating system that runs medical imaging applications. In any case, the target platform / OS 11 may manage multiple software or hardware applications including a target application 17. As used herein, managing a target application 17 may include handling storage, memory, processing, input / output (I / O), and other tasks of the target application 17. In addition to the target user interface 18, the target application 17 may include other core functions, such as controlling medical imaging equipment, processing or displaying medical images, accessing medical records, etc.

[0038] As described herein, the updated user interface 19 may include or be implemented as a web page, and the web page may be compatible across multiple platforms (such as various web browsers or operating systems). In addition, in some embodiments, the visual elements of the web page may be rearranged or reorganized so that they can be appropriately displayed on multiple devices. For example, the visual elements of the web page may be repositioned based on screen size, resolution, and other suitable attributes so that the updated user interface 19 is compatible with various mobile devices, monitors, or other displays. The web page may include or be a part of a hypertext markup language (HTML) file, and the HTML file may define the structure and content characteristics of the web page and / or the updated user interface 19. For example, the visual elements of the updated user interface 19 may include HTML elements that define the characteristics of the visual elements. In addition, the web page may include or be a part of a cascading style sheet (CSS) file, which may define the visual effects of the web page and / or the updated user interface, such as layout, color, etc. In any case, at least a portion of the processing functions associated with the web page may be completed by the server, and the web page may reflect instructions (e.g., display instructions from the server). Therefore, the web page may be described as a "thin client" in this article.

[0039] In other embodiments, the updated user interface 19 may include a fat client, and the fat client may be installed on a client device, such as a workstation (e.g., a laptop or desktop computer) or a console of a medical imaging system. As used herein, the term "fat client" may be understood to mean a device that can process, store, and manage data independently of a server or alternatively in conjunction with communication with a server. The fat client may perform most or all of the processing functions associated with updating the user interface 19 and may be managed by, for example, an operating system associated with the medical imaging system. Therefore, communication with a server for interface updates may not be required, and a server may not be required to facilitate communication between the updated user interface 19, the target user interface 18, the target application 17, and the target platform / OS 11.

[0040] Figure 4 is an illustration of an example updated user interface 19 for use with a medical imaging device. The updated user interface 19 may include components such as aesthetics 21 (e.g., logos, slogans, etc.), medical imaging device information 23, patient diagnosis information 29, buttons 25, scrolling windows 27, calibration information 32, and medical images 31. As discussed herein, components of the updated user interface 19 may provide advantages over previous implementations or original user interfaces (such as Figure 3 In some embodiments, the medical imaging device information 22 may provide additional information about the medical imaging device, the patient diagnosis information 29 may display more interpretable statistics associated with the patient, the button 25 may include a more intuitive, differently sized or actuated symbol, etc. These improvements may be implemented, for example, via a CSS file or HTML file associated with the updated user interface 19.

[0041] User input received at the updated user interface may be intercepted and mapped to a target function of a target application (e.g., target application 17). The user input may include, for example, an indication of a button press (e.g., from a mouse) at a cursor position corresponding to a selected button in buttons 25. The selected button may be mapped to a corresponding target function such that when the button is selected via user input, the target function is executed. In addition, execution of the target function may cause an update to the updated user interface 19, such as different information being displayed as part of the medical imaging device information 22. Additionally or alternatively, execution of the target function may cause the medical imaging device to perform a function, such as causing the X-ray machine to change the amount of radiation output, initiate a scan, terminate a scan, recalibrate the detector array, etc.

[0042] In addition, a selected button in the button 25 can be mapped to a plurality of (e.g., a group of) corresponding target functions, and a user input at the selected button can result in the execution of a plurality of corresponding target functions. A plurality of corresponding target functions can be associated with, for example, a clinical scenario, and executing a plurality of corresponding target functions via an updated graphical user interface can be useful for educational or demonstration purposes in a clinical environment. For example, a clinician may want to present advanced X-ray post-processing functions to an audience (e.g., a group of interns). Using a target user interface, a clinician may have to dock with the target user interface multiple times to perform the target functions necessary for preparing for post-processing. For example, an inspection may need to be initiated for an X-ray system, a clinical protocol may need to be selected, a frame rate and a dose level may need to be selected, etc., and each target function may be mapped to a separate button on the target user interface. However, on an updated user interface, these plurality of corresponding target functions may be mapped to a specific button (e.g., a "scenario button") that accesses the necessary target functions when a user input is made. Therefore, a clinician may provide user input at a specific button to perform all necessary functions, thereby eliminating the need for complex docking with an X-ray system to prepare for a demonstration.

[0043] Figure 5 is a block diagram of a system 10 in which an updated user interface 19 accesses a target function 37 via a backend masking process 35 and a hijack 39. As described herein, the updated user interface 19 can be part of or used in conjunction with a web page, and the updated user interface 19 can be accessed via a thick client or web browser 38. The target application 17 may include a target function 37 and a target user interface 18. The target application 17 may be a previously implemented application (e.g., a legacy application) associated with a medical imaging device, and the target user interface may be a previously implemented or original user interface (e.g., a legacy user interface) for displaying data of the medical imaging device and / or controlling the medical imaging device. The target user interface 18 may, for example, be used to perform the target function 37. Additionally or alternatively, the target function may result in changes to components of the target user interface 18, such as graphs, data, and digital information of a medical information device or a patient.

[0044] The updated user interface 19 and / or the thick client or web browser 38 may cause (e.g., trigger) a backend masking process 35 that provides access to the target functionality 37 and / or the target user interface 18 via a hijack 39. In some examples, the backend masking process 35 and / or the hijack 39 may be managed or facilitated by one or more dynamic link libraries (DLLs) capable of storing a collection of one or more functions. Thus, the DLL may store the target functionality 37 to be accessed by the updated user interface 19 and / or the web browser 38. In other examples, similar or corresponding methods are contemplated, such as debugger assistive technology. The backend masking process 35 may include an input event (e.g., a click event, a click wake event, or a touch wake event) that causes a query to be sent to the server when user input is received at the updated user interface 19. The input event may correspond to a click, a tap, a touch, or an input from, for example, Figure 3 39 can intercept the user input at the updated user interface 19 and cause the execution of the corresponding target function. In other embodiments, the hijacking can intercept the user input and cause the simulated user input at the target user interface 18. For example, the server and / or the hijacking 39 can map the user input at the updated user interface 19 to the corresponding HTML element of the target user interface 18. Therefore, the target application 17 can execute one or more target functions associated with the corresponding HTML element.

[0045] Figure 6 3 is a block diagram illustrating communication paths between components of system 10, including updated user interface 19, server 36, and target user interface 18. Updated user interface 19 may receive user input from input device 20, and the user input may correspond to a click event or other input event associated with updated button 32. The click event may cause server 36 to be queried. In response, server 36 may send a request via hijack 39 to activate a corresponding button 34 of target user interface 18, which may be part of target application 17. Activating a button may include a simulated press of the corresponding button 34, which causes the corresponding target function to be executed, as described herein.

[0046] As illustrated, the updated user interface 19 may include a web page and may be accessed by a user via, for example, a web browser 38. Also as illustrated, the web browser 38 and the target application 17 may share a target platform / OS 11. For example, the web browser 38 may be managed by the same operating system through which the target application 17 is managed. Additionally or alternatively, the updated user interface 19 may include or be accessed by a thick client, as described herein.

[0047] When the corresponding button 34 is activated and / or the corresponding target function is executed, the target button feedback may be routed to the updated user interface 19 via the server 36. For example, the target function may cause a change in a visual element of the target user interface 18 and / or the target application 19. The target button feedback may be sent to the server 36, and the server 36 may then map the button feedback to the reflected feedback in the updated user interface 19. The reflected feedback may then be sent to the updated user interface 19 and displayed at the updated user interface. For example, the reflected feedback may include a visual change in an HTML element of the updated user interface 19.

[0048] The updated button 34 may have an updated aesthetic, display characteristics, viewing pane, and / or control layout, etc., which may provide a more modern or more intuitive experience for the user. In addition, the corresponding button 34 and the updated button 32 may correspond to similar portions of the update user interface 19 and the target user interface 18, respectively, and the target button feedback and the reflected feedback may cause similar changes in the updated user interface 19 and the target user interface 18. The target button feedback and the reflected feedback may cause similar control functions of, for example, a medical imaging device. Therefore, users of the updated user interface 19 who are familiar with the target user interface 18 may maintain some familiarity when using the updated user interface 19. In addition, by maintaining the functionality of the corresponding button 34, the target user interface 18, and the target function of the target application 17, the need for specific implementation of new functions may be eliminated, and thus interface updates may be less expensive.

[0049] Although Figure 6 One web browser 38 and updated user interface 19 are shown, but in some embodiments, multiple web browsers 38 and multiple updated user interfaces 19 may access the target application 17 via the server 36 . Figure 7A system 10 is shown in which a plurality of web browsers 38 query a server 36. Each web browser 38 includes an updated user interface 19, and each web browser 38 may receive input from an input device 20. Each input may correspond to an input event (such as a click event), and each click event may cause the server 36 to be queried. In response, the server 36 may send a request to activate a corresponding button of a target user interface 18 via hijacking, which may be part of a target application 17, as described herein. Activating a button may include a simulated press of a corresponding button, which causes a corresponding target function to be executed, and the target function may cause a change in one or more of the updated user interfaces 19. Thus, the target application 17 and the target user interface 18 may include a target function that may be accessed by a plurality of web browsers 38, and the server may manage communications between each web browser 38 and the target application 17.

[0050] Each web browser 38 and server 36 can communicate using a suitable network communication technology, such as Hypertext Transfer Protocol (HTTP), Transmission Control Protocol (TCP), Internet Protocol (IP), Wi-Fi, Bluetooth, Near Field Communication (NFC), etc. Therefore, when a click event is triggered, for example, in response to an input from a user input device 20, each web browser 38 can send a query (e.g., an HTTP request). In some embodiments, the web browser 38 can process queries from each web browser 38 simultaneously. For example, the web browser 38 can maintain a thread for each web browser 38 to manage queries to the corresponding web browser. In another example, the web browser 38 can utilize asynchronous event-driven programming techniques so that queries from multiple web browsers 38 can be maintained on a single thread. In addition, the server 36 can implement a cache to store and retrieve items frequently requested by web browsers, such as icons, logos, or other visual elements.

[0051] Figure 8 1 is a flow chart of an example method 100 for implementing an updated graphical user interface using the techniques described herein, and is discussed with reference to the aforementioned figures. Although the method 100 is described as being performed by a web browser 38 and a target application 17, the method 100 may be performed by a thick client, a server 36, or other suitable components of the system 11. The method may begin in block 102, where the target platform / OS 11 attaches the target application 17 along with associated target functions and a target user interface 18 to the web browser 38. For example, attaching the target application 17 may include exchanging access permissions (such as read permissions, write permissions, etc.) necessary to perform subsequent steps between the web browser 38 and the target application 17.

[0052] In box 104, the target platform / OS11 may allocate memory associated with the web browser 38 to store functions, objects, etc. associated with accessing the target application 17. Specifically, in box 106, the path associated with the hijack 39 may be copied to the allocated memory. For example, the path may include various target functions to the target application 17, paths to visual elements of the target user interface 18, etc. The copied path may have an associated memory address that can be referenced when executing the hijack. For example, the memory address may be referenced in response to a click event and / or a query to the server 36, as described herein. Once the hijack is placed in memory, in box 108, the target platform / OS11 may prepare the web browser 38 to execute the hijack. Preparation may include modifying the executable source of the target platform / OS11 or injecting the target function into the runtime process of the web browser 38. Preparation may also include targeting the thread of the web browser 38, saving the current register of the web browser 38, changing the instruction pointer to the memory address of the copied path, and / or restoring the thread.

[0053] Fig. 9 2 is a flow chart of a method 200 for launching an updated user interface, hijacking to a target application in response to user input, and reflecting feedback on the updated user interface. In block 202, a web browser (e.g., web browser 38) launches an updated user interface (e.g., updated user interface 19). The web browser 38 may launch the updated user interface 19 in response to user input (e.g., from a user input device 20). For example, the updated user interface 19 may include a web page, and the web page may be launched in response to a user entering a URL associated with the web page at the web browser 38. When launching the updated user interface 19, hijacks may be placed in appropriate locations in the memory of the web browser 38, such as within an initialization loop instruction, within a click event instruction, etc. These hijacks may be inserted at the runtime of the target platform / OS 11, the web browser 38, or a thick client associated with the updated user interface 19.

[0054] In box 204, the web browser 38 receives input from, for example, a user via the user input device 20. The input may include a user selecting an HTML element (such as a button) of the updated user interface 19, and may cause an associated input event to be executed. In box 206, the execution of the input event may result in a reference to a location in a memory (e.g., a memory of the web browser 38 or the server 36) that includes an associated hijack path. In box 208, the hijack path may result in the execution of a corresponding target function of a target application (e.g., the target application 17). In box 210, the execution of the target function may result in reflected feedback at the updated user interface 19. The reflected feedback may include changes to visual elements of the updated user interface 19, control indications of medical devices, information updates, etc.

[0055] Considering the foregoing, Fig.10 An embodiment of a client-server architecture 300 for a specific implementation of an updated graphical user interface according to the presently described techniques is illustrated. For example, the client-server architecture 300 can be used to facilitate communications between a client device 302, a server 36, a computer 42, and an imaging system 10. In the depicted example, the client-server architecture 300 generally includes at least one computer 42 and at least one front-end computing system, represented here as a client device 302 that is directly or indirectly communicatively coupled via a suitable network 312 (e.g., a local area network (LAN), a wide area network (WAN), a virtual private network (VPN), the Internet). For example, the client device 302 can perform functions associated with a web browser 38 and / or an updated user interface 19, and the computer 42 can perform functions associated with a target application 17 and / or a target user interface 18, such as the target functions described herein. The computer 42 can also be communicatively coupled to the imaging system 10, and can store and analyze images 100 received from the imaging system 10. In some embodiments, the cloud server architecture 300 can include at least one server deployed on a LAN of a medical facility. For embodiments where the client-server architecture 300 is, in whole or in part, a cloud-based client-server architecture, the imaging system 10 or the client-server architecture 300 may include one or more rack-mounted servers deployed at a remote data center or locally at a medical facility. The client device 302 may include or may be coupled to a clinician's desktop or laptop computer, for example, deployed on a LAN at a medical facility or coupled to the computer 42 via a suitable network connection.

[0056] In the following description, it should be understood that certain functions may be implemented on the computer 42, the front-end computing system (e.g., the client device 302 and / or associated workstation), or both. Thus, certain routines and / or functionality may be described as potentially present on both the computer 42 and / or the client device 302 (or associated workstation). In practice, such routines or functionality will likely be implemented on only one of the front end or the back end, as determined based on specific implementation and business-specific decisions. However, for completeness, this discussion describes such functions as potentially being implemented on either the front end or the back end.

[0057] With this in mind, in the depicted example, the computer 42 includes at least one processor 320, at least one memory 336 (e.g., random access memory (RAM), read-only memory (ROM)), at least one networking device 352 (e.g., wireless networking card, Ethernet networking card), and at least one storage device 340 (e.g., non-transitory computer-readable media, such as, but not limited to, hard disk devices, solid-state disk devices, flash memory devices). The processor 320 may include one or more central processing units (CPUs), each having one or more processing cores configured to execute instructions and process data loaded from the storage device 340 into the memory 336. In the present context, the storage device 340 of the computer 42 may store an image 100 (e.g., a medical image or scan), a target application configured to perform one or more target functions associated with a target user interface 18 or control of the imaging system 10, a processor-executable GUI routine or framework 18 configured to present patient and imaging system data, and one or more processor-executable routines for analysis (e.g., multiplanar reconstruction) and presentation of the image 100.

[0058] The front-end computing system (represented here as a client device 302 or a workstation associated with such a client device) typically includes at least one processor 320, at least one memory 322 (e.g., random access memory (RAM), read-only memory (ROM)), at least one networking device 350 (e.g., wireless networking card, Ethernet networking card) and at least one storage device 326 (e.g., non-transitory computer-readable media, such as but not limited to hard disk devices, solid-state disk devices, flash memory devices), and a display 324 that can display graphical components and medical images. In addition, the client device 302 includes input / output (I / O) devices 352, such as a keyboard, mouse, touchpad, touch screen, speaker, display, etc., which enable the clinician to provide input to the client device 302 and receive output from the client device. In some embodiments, the client device 302 includes at least one graphics processing unit (GPU) that is generally configured to perform graphics processing to present images on the display of the client device 302. In the present context, the storage device 326 may store an image 100 (e.g., a medical image or scan), one or more processor-executable GUI routines or frameworks 19 configured to present patient and imaging system data, and one or more processor-executable routines for analysis (e.g., multiplanar reconstruction) and presentation of the image 100. The one or more processor-executable GUI routines or frameworks 19 may include routines for updating and displaying a web page facilitated by a web browser, and may query a network server (e.g., server 36) for updates to the web page.

[0059] The server 36, which may include one or more rack-mounted server devices located on-site at a medical facility, may include at least one networking device 356 (e.g., a wireless networking card, an Ethernet networking card) and at least one storage device 346 (e.g., a non-transitory computer-readable medium, such as but not limited to a hard disk device, a solid-state disk device, a flash memory device). In this context, the storage device 346 may store one or more processor-executable hijacks 39 that are configured to map the GUI routines of the updated user interface 19 to the GUI routines of the target user interface 18. The server 36 may also include a processor, a memory, or other components suitable for networking and executing the hijack 39.

[0060] In the depicted example, information is shown as being exchanged between the client device 302 and the computer 42 via the server 36. In practice, such data may include, but is not limited to, data associated with input events detected by the client device, such as button activation indications and server requests, feedback and / or instructions to change one or more elements of the updated graphical user interface 19, instructions associated with the medical imaging system, and the like. For example, in a first specific implementation, user input obtained by the client device 302 (e.g., via the I / O device 352) may be sent to the computer 42 via the server 36, where the user input may be mapped to one or more user interface routines (e.g., target functions) of the target user interface 18. In one such example, the user input received via the I / O device 352 is determined to correspond to a click event, and the client device 302 sends a button activation server request to the computer 42 via the server 36. The server 36 may map the button activation server request to one or more GUI routines (e.g., target functions) of the target graphical user interface 18. In response, the imaging system 304 may send the output of the one or more GUI routines as feedback / instructions 344 to the client device 302 via the server 36. In the depicted example, one or more of feedback or instructions may be sent back to the client device 308, although such data may not be transmitted in all cases.

[0061] In another specific implementation, the front end (illustrated here as the client device 302) may include a fat client architecture, and additional functionality may be performed on the front end. In this example, user input obtained via the I / O device 352 or other components may be determined to correspond to a click event, and the processor 320 may execute a mapping routine (e.g., hijacking) to map the click event to a GUI routine of the target user interface 18. In this example, the client device 302 may also locally store (e.g., in the storage device 326) a processor executable routine for locally maintaining and updating a graphical user interface (e.g., an updated graphical user interface 19). Therefore, the graphical user interface may be maintained without a web page or web browser, and may be updated without a query to a web server. In this example, portions of the communication functionality (such as the exchange of button activation / server request 342 and feedback / instruction 344 between the client device 302 and the computer 42) may be performed by the client device 302 alone or by the client device 302 in conjunction with the server 36.

[0062] As can be understood from the foregoing examples, various aspects of the technology currently described (such as user input acquisition, graphical user interface updates, image processing and analysis / visualization) can be performed at one location or can be distributed between systems (such as between a local system and a cloud-based system). Although the foregoing examples relate to certain possible specific implementation scenarios, it is understood that the present disclosure can perform and anticipate other actual specific implementations. In addition, it is understood that various aspects of the foregoing examples can be mixed to achieve a mixed specific implementation, such as there are multiple client devices 302 with different capabilities (e.g., different graphical user interface elements) and they can therefore each send different input event information, button activations, server requests, etc. For example, in some mixed scenarios, some client devices 302 can send certain button activations or server requests (e.g., associated with certain graphical user interface elements), while other client devices 302 can send other button activations or server requests. Therefore, this example should be understood in view of its intention to provide useful context and specific implementation scenarios, but should not be understood as an exhaustive list of all possible specific implementations or permutations.

[0063] This written description uses examples to disclose the invention, including the best mode, and also to enable any person skilled in the art to practice the invention, including making and using any devices or systems and performing any included methods. The patentable scope of the invention is defined by the claims, and may include other examples that occur to those skilled in the art. Such other examples are intended to fall within the scope of the claims if they have structural elements that do not differ from the literal language of the claims, or if they include equivalent structural elements with insignificant differences from the literal language of the claims.

Claims

1. A processor-implemented method, comprising: receiving user input from one or more user input devices, the user input corresponding to a selected element of a displayed graphical user interface having one or more software hijacks of functionality provided by a non-displayed interface; determining one or more corresponding target functions of the non-displayed interface based on the selected element; executing the one or more corresponding target functions via the one or more software hijackings; as well as One or more changes to the displayed graphical user interface are displayed based on the execution of the one or more corresponding target functions.

2. The processor-implemented method of claim 1 , wherein the displayed graphical user interface is stored on and displayed by a first computing device, the undisplayed interface is stored on a second computing device, and the one or more software hijacks are stored on a server. 3 . The processor-implemented method of claim 2 , wherein the server is communicatively coupled to the first computing device and the second computing device. 4 . The processor-implemented method of claim 2 , wherein the second computing device is configured to receive medical images from a medical imaging device. 5 . The processor-implemented method of claim 1 , wherein the displayed graphical user interface is configured to display an image acquired by a medical imaging device. 6 . The processor-implemented method of claim 1 , wherein the displayed graphical user interface comprises a layout, a window, a button, an icon, a menu, or a combination thereof.

7. The processor-implemented method of claim 1, wherein the selected element comprises a window, a button, an image, an icon, or a menu. 8 . The processor-implemented method of claim 1 , wherein the one or more corresponding target functions are determined based on a mapping between elements of the displayed graphical user interface and a plurality of target functions of the non-displayed interface. 9 . The processor-implemented method of claim 8 , wherein the element of the displayed graphical user interface includes the selected element, and wherein the plurality of target functions includes the one or more corresponding target functions.

10. A non-transitory computer readable medium comprising processor executable code which, when executed by a processor, causes the processor to: receiving user input from one or more user input devices, the user input corresponding to a selected element of a displayed graphical user interface having one or more software hijacks of functionality provided by a non-displayed interface; determining one or more corresponding target functions of the non-displayed interface based on the selected element; executing the one or more corresponding target functions via the one or more software hijackings; and One or more changes to the displayed graphical user interface are displayed based on the execution of the one or more corresponding target functions.

11. The non-transitory computer-readable medium of claim 10, wherein the selected element comprises a Hypertext Markup Language (HTML) element.

12. The non-transitory computer-readable medium of claim 10, wherein the displayed graphical user interface is part of a first software application, and wherein the undisplayed interface is part of a second software application.

13. The non-transitory computer-readable medium of claim 12, wherein the first software application comprises a web page.

14. The non-transitory computer readable medium of claim 13, wherein the web page is updated by a web browser.

15. The non-transitory computer readable medium of claim 12, wherein the first software application and the second software application are managed by a shared operating system.

16. A system, comprising: A first computing device, wherein the first computing device is configured to: Displaying a graphical user interface; receiving user input from one or more user input devices, the user input corresponding to a selected element of the displayed graphical user interface having one or more software hijacks of functionality provided by a non-displayed interface; sending an indication of the selected element; as well as A second computing device, wherein the second computing device is configured to: receiving said indication of said selected element; determining one or more corresponding target functions of the non-displayed interface based on the selected element; instructing a third computing device to execute the one or more corresponding target functions via the one or more software hijacks; receiving output associated with said execution of said one or more corresponding target functions; and The first computing device is instructed to change one or more elements of the displayed graphical user interface based on the output.

17. The system of claim 16, wherein the indication of the selected element comprises an input event.

18. The system of claim 16, wherein the second computing device comprises a server.

19. The system of claim 16, wherein the third computing device is configured to receive medical images from a medical imaging device.

20. The system of claim 19, wherein the output is associated with control of the medical imaging device, display of the medical image, or a combination thereof.

Images