Adaptive user interface for medical imaging systems

Abstract

The ultrasound imaging system may include a user interface that may be adapted based at least in part on usage data of one or more users. In some examples, the functionality of soft or hard control elements may be changed. In some examples, the layout or appearance of soft or hard control elements may be changed. In some examples, infrequently used control elements may be removed from the user interface. In some examples, the user interface may be adapted based on the anatomical feature being imaged. In some examples, the usage data may be analyzed by artificial intelligence / machine learning models that can provide output that may be used to adapt the user interface.

Description

[Technical Field]

[0001] TECHNICAL FIELD

[0001] This disclosure relates to adaptive user interfaces for medical imaging systems, such as ultrasound imaging systems, for example, user interfaces that automatically adapt based on previous use of the user interface. [Background technology]

[0002]

[0002] User interfaces (UIs), particularly graphical user interfaces, are an important aspect of the overall user experience for operators of medical imaging systems, such as ultrasound imaging systems. Typically, users operate medical imaging systems in a particular way, which may vary from user to user based on several factors, including, for example, personal preference (e.g., following or not following standard protocols, heavy time gain compensation (TGC) users), geography, user type (e.g., physician, sonographer), and application (e.g., abdominal, vascular, breast). However, current medical imaging systems on the market offer little or no support for user customization of the UI, and none feature UIs that adapt over time. Summary of the Invention

[0003] Systems and methods are disclosed that can overcome the limitations of current medical imaging system user interfaces by dynamically changing (e.g., adapting, adjusting) the presentation of hard and / or soft control elements based at least in part on an analysis of one or more users' previous button usage, keystrokes, and / or control element sequence patterns (collectively, usage data). In some applications, the flow of an ultrasound procedure can be simplified and more efficient for the user than with prior art fixed user interface (UI) systems.

[0004] As disclosed herein, a UI for a medical imaging system may include a dynamic button layout that allows a user to customize button positions, show / hide buttons, and customize which pages buttons appear on. As disclosed herein, one or more processors may analyze usage data, e.g., usage data stored in log files containing logs of past keystrokes and / or control element selection sequences, to determine the percentage of use of particular control elements (e.g., buttons) and / or the typical control element usage sequence. In some examples, the one or more processors may implement artificial intelligence, machine learning, and / or deep learning models, for example, trained using previously acquired log files, to analyze the usage data (e.g., keystroke and control sequence patterns entered by a user, or user types for a given procedure). Based on the analysis, the one or more processors may adjust the dynamic button layout of the UI based on the output of the trained model.

[0005]

[0005] According to at least one example of the present disclosure, a medical imaging system may include a user interface including a plurality of control elements, each of the plurality of control elements being operated by a user to modify the operation of the medical imaging system, a memory that stores usage data resulting from the operation of the plurality of control elements, and a processor in communication with the user interface and the memory, wherein the processor receives the usage data and, based on the usage data, determines a first control element of the plurality of control elements that is associated with a lower frequency of use than a second control element of the plurality of control elements, and adapts the user interface based on the frequency of use by reducing the visibility of the first control element, increasing the visibility of the second control element, or a combination thereof.

[0006]

[0006] According to at least one example of the present disclosure, a medical imaging system may include a user interface including a plurality of control elements operated by a user to modify operation of the medical imaging system, a memory that stores usage data resulting from operation of the plurality of control elements, and a processor in communication with the user interface and the memory, wherein the processor receives the usage data, receives an indication of a first selected control element of the plurality of control elements that is associated with a first function, determines a predicted next function based at least in part on the usage data and the first function, and, after operation of the first control element, adapts the user interface by changing the function of one control element of the plurality of control elements to the predicted next function, increasing visibility of the control element that performs the predicted next function relative to other control elements of the plurality of control elements, or a combination thereof. [Brief explanation of the drawings]

[0007] [Figure 1] FIG. 1 is a block diagram of an ultrasound system in accordance with the principles of the present disclosure. [Figure 2]

[0008] FIG. 2 is a block diagram illustrating an example processor in accordance with the principles of the present disclosure. [Figure 3]

[0009] FIG. 3 illustrates a portion of an ultrasound imaging system according to an example of the present disclosure. [Figure 4]

[0010] FIG. 4 illustrates soft control elements provided on a display according to an example of the present disclosure. [Figure 5A]

[0011] FIG. 5A illustrates soft control elements provided on a display according to an example of the present disclosure. [Figure 5B]

[0012] FIG. 5B illustrates soft control elements provided on a display according to an example of the present disclosure. [Figure 6]

[0013] FIG. 6 illustrates soft control elements provided on a display according to an example of the present disclosure. [Figure 7]

[0014] FIG. 7 illustrates soft controls provided on a display according to an example of the present disclosure. [Figure 8A]

[0015] FIG. 8A illustrates soft control elements provided on a display according to an example of the present disclosure. [Figure 8B]

[0016] FIG. 8B illustrates soft control elements provided on a display according to an example of the present disclosure. [Figure 9]

[0017] FIG. 9 shows an example ultrasound image on a display and example soft control elements provided on the display, according to an example of the present disclosure. [Figure 10]

[0018] FIG. 10 graphically illustrates an example of a statistical analysis of one or more log files, according to an example of the present disclosure. [Figure 11]

[0019] FIG. 11 graphically illustrates an example of a statistical analysis of one or more log files, according to an example of the present disclosure. [Figure 12]

[0020] FIG. 12 illustrates a neural network that may be used to analyze usage data according to examples of this disclosure. [Figure 13]

[0021] FIG. 13 illustrates a cell of a long-short-term memory model that may be used to analyze usage data according to examples of the present disclosure. [Figure 14]

[0022] FIG. 14 shows a block diagram of a process for training and deploying a neural network in accordance with the principles of the present disclosure. [Figure 15]

[0023] FIG. 15 shows a diagrammatic overview of how a user moves buttons within a page of a menu provided on a display according to an example of the present disclosure. [Figure 16]

[0024] FIG. 16 illustrates a diagrammatic overview of how a user moves buttons between pages of a menu on a display according to an example of the present disclosure. [Figure 17]

[0025] FIG. 17 illustrates a diagrammatic overview of how a user may swap the positions of buttons on a display according to an example of the present disclosure. [Figure 18]

[0026] FIG. 18 illustrates a diagrammatic overview of how a user may move a group of buttons on a display according to an example of the present disclosure. [Figure 19]

[0027] FIG. 19 illustrates a diagrammatic overview of how a user may change a rotary control element on a display to a list button according to an example of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0008]

[0028] The following description of specific embodiments is for illustrative purposes only and is not intended to limit the invention or its application or uses. In the following detailed description of embodiments of the present system and method, reference is made to the accompanying drawings, which form a part hereof, and which illustrate specific embodiments in which the described system and method may be practiced. These embodiments have been described in sufficient detail to enable those skilled in the art to practice the disclosed system and method, and it should be understood that other embodiments may be utilized, and that structural and logical changes may be made without departing from the spirit and scope of the present system. Moreover, for purposes of clarity, detailed descriptions of certain features, if deemed apparent to those skilled in the art, will be omitted so as not to obscure the description of the present system. Therefore, the following detailed description is not to be taken in a limiting sense, and the scope of the present system is defined only by the appended claims.

[0009]

[0029] Users of medical imaging systems have expressed frustration with the inability to customize the user interface (UI) of their medical imaging systems. While different users can operate medical imaging systems in widely different ways, each user typically follows the same or similar patterns each time they use the medical imaging system, especially for the same application (e.g., fetal scans or echocardiograms in ultrasound imaging). That is, for a particular application, the user typically uses the same set of control elements, performs the same tasks, and / or performs the same tasks in the same order each time. This is especially true when users follow an imaging technician-driven workflow, in which an imaging technician (e.g., a sonographer) performs imaging exams that are later read by a physician (e.g., a radiologist) for review. Exams based on this workflow are common in North America.

[0010]

[0030] Users often adjust or customize system settings to optimize workflow-based examinations. Such customization can improve the efficiency and quality of examinations for that particular user. However, customization can be time-consuming and / or require re-entry each time the user initializes a particular system. Therefore, the inventors have recognized that in addition to, or instead of, allowing users to perform their own UI customizations, a medical imaging system can be configured to "learn" the user's preferences and automatically adapt the UI to the user's preferences without the user having to manually perform the customizations. Thus, considerable time and effort can be saved and the quality of the examination can be improved.

[0011]

[0031] As disclosed herein, a medical imaging system may analyze and automatically adapt (e.g., adjust, change) the UI of the ultrasound imaging system based at least in part on usage data (e.g., keystrokes, button press patterns) collected from one or more users of the ultrasound imaging system. In some examples, the medical imaging system may dim infrequently used controls on the display. In some examples, the degree of dimming may increase over time until the control element is removed from the display. In some examples, infrequently used control elements may be moved lower on the display and / or to a second or subsequent page of a menu in the UI. In some examples, frequently used control elements may be highlighted (e.g., displayed brighter or in a different color than other control elements). In some examples, the medical imaging system may infer which control element the user will select next and highlight that control element on the display and / or control panel. In some examples, the medical imaging system may change the function of a soft control element (e.g., a button on a touchscreen) or a hard control element (e.g., a switch, dial, slider) based on an inference of the control function the user will use next. In some examples, this analysis and adaptation may be provided for each individual user of the medical imaging system. Thus, the medical imaging system may provide a customized, adaptive UI for each user without requiring the user to manipulate system settings. In some applications, automatically adapting the UI may reduce exam time, improve efficiency, and provide ergonomic benefits to the user.

[0012]

[0032] The examples disclosed herein are provided with reference to an ultrasound imaging system, however, this is for illustrative purposes only and the adaptive UI and its features disclosed herein may be applied to other medical imaging systems.

[0013]

[0033] FIG. 1 illustrates a block diagram of an ultrasound imaging system 100 constructed in accordance with the principles of the present disclosure. The ultrasound imaging system 100 according to the present disclosure may include a transducer array 114, which may be included in an ultrasound probe 112. The ultrasound probe 112 may be, for example, an external probe or an internal probe such as an intracardiac echocardiogram (ICE) probe or a transesophageal (TEE) probe. In other embodiments, the transducer array 114 may take the form of a flexible array configured to conform to the surface of an imaging object (e.g., a patient). The transducer array 114 is configured to transmit ultrasound signals (e.g., beams or waves) and receive echoes in response to the transmitted ultrasound signals. Various transducer arrays may be used, such as a linear array, a curved array, or a phased array. The transducer array 114 may include, for example, a two-dimensional array including multiple transducer elements that can be scanned in both the elevation and azimuth dimensions for 2D and / or 3D imaging (as shown). As is commonly known, the axial direction is the direction perpendicular to the plane of the array (for curved arrays, the axial direction fans out), the azimuth direction is generally defined by the vertical dimension of the array, and the elevation direction is the direction transverse to the azimuth direction.

[0014]

[0034] In some embodiments, the transducer array 114 may be coupled to a microbeamformer 116 that may control the transmission and reception of signals by the transducer elements in the array 114. The microbeamformer may be provided within the ultrasound probe 112. In some embodiments, the microbeamformer 116 may control the transmission and reception of signals by the active elements in the array 114 (e.g., a subset of the active elements of the array that define an active aperture at any time).

[0015]

[0035] In some embodiments, the microbeamformer 116 may be coupled, for example, by a probe cable or wirelessly, to a transmit / receive (T / R) switch 118 that switches between transmitting and receiving and protects the main beamformer 122 from high-energy transmit signals. In some embodiments, such as portable ultrasound systems, the T / R switch 118 and other elements in the system may be contained within the ultrasound probe 112 rather than within an ultrasound system base that may house image processing electronics. The ultrasound system base typically includes software and hardware components, including circuitry for signal processing and image data generation, and executable instructions for providing a user interface (e.g., processing circuitry 150 and user interface 124).

[0016]

[0036] The transmission of ultrasound signals from the transducer array 114, controlled by the microbeamformer 116, is managed by a transmit controller 120, which may be coupled to the T / R switch 218 and the beamformer 122. The transmit controller 120 may control the direction in which the beam is steered. The beam may be steered straight from the transducer array 114 (orthogonal to the array) or at several different angles for a wider field of view. The transmit controller 120 may be coupled to a user interface 124 to receive input from user manipulation of user control elements. The user interface 124 may include one or more input devices, such as a control panel 152, which may include one or more mechanical control elements (e.g., buttons, encoders, etc.), touch-sensitive control elements (e.g., trackpads or touchscreens, etc.), and / or other known input devices.

[0017]

[0037] In some embodiments, the partially beamformed signals generated by the microbeamformer 116 are sent to the main beamformer 122, which may combine the partially beamformed signals from each transducer element patch to generate a fully beamformed signal. In some embodiments, the microbeamformer 116 is omitted, and the transducer array 114 is under the control of the main beamformer 122, which performs all beamforming of the signals. In embodiments with or without the microbeamformer 116, the beamformed signals of the main beamformer 122 are coupled to processing circuitry 150, which may include one or more processors (e.g., signal processor 126, B-mode processor 128, Doppler processor 160, and one or more image generation and processing components 168) configured to generate ultrasound images from the beamformed signals (e.g., beamformed RF data).

[0018]

[0038] The signal processor 126 may be configured to process the received beamformed RF data in various ways, such as bandpass filtering, decimation, I and Q component separation, and harmonic signal separation. The signal processor 126 may also perform additional signal enhancements, such as speckle reduction, signal combining, and noise removal. The processed signals (also referred to as I and Q components or IQ signals) may be coupled to additional downstream signal processing circuitry for image generation. The I and Q signals may be coupled to multiple signal paths within the system, and each signal path may be associated with a specific signal processing component configuration suitable for generating different types of image data (e.g., B-mode image data, Doppler image data). For example, the system may include a B-mode signal path 158 that couples signals from the signal processor 126 to a B-mode processor 128 to generate B-mode image data.

[0019]

[0039] The B-mode processor may use amplitude detection for imaging structures within the body. The signals generated by the B-mode processor 128 may be coupled to the scan converter 130 and the multiplanar reformatter 132. The scan converter 130 arranges the echo signals into a desired image format from the spatial relationship in which they were reflected. For example, the scan converter 130 may arrange the echo signals into a two-dimensional (2D) sector format or a pyramidal or other shaped three-dimensional (3D) format. The multiplanar reformatter 132 may convert echoes received from multiple points in a common plane within a volumetric region of the body into an ultrasound image of that plane (e.g., a B-mode image), as described, for example, in U.S. Pat. No. 6,443,896 (Detmer). In some embodiments, the scan converter 130 and the multiplanar reformatter 132 may be implemented as one or more processors.

[0020]

[0040] The volume renderer 134 may generate an image of the 3D dataset as viewed from a given reference point (also called a projection or rendering), for example, as described in U.S. Patent No. 6,530,885 (Entrekin et al.). In some embodiments, the volume renderer 134 may be implemented as one or more processors. The volume renderer 134 may generate renderings, such as positive or negative renderings, by any known or future known techniques, such as surface rendering and maximum intensity rendering.

[0021]

[0041] In some embodiments, the system may include a Doppler signal path 162 that couples the output from the signal processor 126 to a Doppler processor 160. The Doppler processor 160 may be configured to estimate the Doppler shift and generate Doppler image data. The Doppler image data may include color data that is overlaid with B-mode (i.e., grayscale) image data for display. The Doppler processor 160 may be configured to remove undesired signals (i.e., noise or clutter associated with stationary tissue), for example, using a wall filter. The Doppler processor 160 may further be configured to estimate velocity and power according to known techniques. For example, the Doppler processor may include a Doppler estimator, such as an autocorrelator, in which velocity (Doppler frequency) estimation is based on the argument of a lag-1 autocorrelation function and Doppler power estimation is based on the magnitude of a lag-0 autocorrelation function. Motion may also be estimated by known phase-domain (e.g., parametric frequency estimators such as MUSIC or ESPRIT) or time-domain (e.g., cross-correlation) signal processing techniques. Instead of or in addition to the velocity estimator, other estimators related to the time or spatial distribution of velocity, such as an acceleration estimator or an estimator of the temporal and / or spatial velocity derivative, may be used. In some embodiments, the velocity and / or power estimates may undergo post-processing, such as additional thresholding to further reduce noise, segmentation, filling, and smoothing. The velocity and / or power estimates may then be mapped to a desired display color range according to a color map. The color data, also referred to as Doppler image data, may then be combined with the scan converter 230. In the scan converter 230, the Doppler image data may be converted to a desired image format and overlaid on a B-mode image of the tissue structure to form a color Doppler or power Doppler image. In some examples, the scan converter 130 may register the Doppler image with the B-mode image.

[0022]

[0042] Output from the scan converter 130, multiplanar reformatter 132, and / or volume renderer 134 may be coupled to the image processor 136 for further enhancement, buffering, and temporary storage before being displayed on the image display 138. The graphics processor 140 may generate graphic overlays to be displayed with the images. The graphic overlays may include standard identifying information, such as the patient's name, the date and time of the image, imaging parameters, etc. For these purposes, the graphics processor may be configured to receive input from the user interface 124, such as a typed patient name or other annotations. The user interface 124 may also be coupled to the multiplanar reformatter 132 for selection and control of the display of multiple MPR (multiplanar reconstructed) images.

[0023]

[0043] The ultrasound imaging system 100 may include a local memory 142. The local memory 142 may be implemented as any suitable non-transitory computer-readable medium (e.g., a flash drive or a disk drive). The local memory 142 may store data generated by the ultrasound imaging system 100, including ultrasound images, log files containing usage data, executable instructions, imaging parameters, training data sets, and / or any other information necessary for the operation of the ultrasound imaging system 100. To avoid obscuring FIG. 1 , not all connections are shown, but the local memory 142 may be accessible by additional components other than the scan converter 130, the multiplanar reformatter 132, and the image processor 136. For example, the graphics processor 140, the transmit controller 120, the signal processor 126, the user interface 124, etc. may be able to access the local memory 142.

[0024]

[0044] As described above, the ultrasound imaging system 100 may include a user interface 124. The user interface 124 may include a display 138 and a control panel 152. The display 138 may include a display device implemented using various known display technologies, such as LCD, LED, OLED, or plasma display technology. In some embodiments, the display 138 may include multiple displays. The control panel 152 may be configured to receive user input (e.g., a preset number of frames, a filter window length, an imaging mode). The control panel 152 may include one or more hard control elements (e.g., buttons, knobs, dials, encoders, a mouse, a trackball, etc.). Hard control elements may sometimes be referred to as mechanical control elements. In some embodiments, the control panel 152 may additionally or instead include soft control elements (e.g., GUI control elements such as buttons and sliders, or simply GUI control elements) provided on a touch-sensitive display. In some embodiments, the display 138 may be a touch-sensitive display that includes one or more soft control elements of the control panel 152.

[0025]

[0045] According to examples of the present disclosure, the ultrasound imaging system 100 may include a user interface (UI) adapter 170 that automatically adapts the appearance and / or functionality of the user interface 124 based at least in part on a user's use of the ultrasound imaging system 100. In some examples, the UI adapter 170 may be implemented by one or more processors and / or application-specific integrated circuits. The UI adapter 170 may collect usage data from the user interface 124. Examples of usage data include, but are not limited to, keystrokes, button presses, other hard control element operations (e.g., selections) (e.g., turning a dial, flipping a switch), screen touches, other soft control element operations, menu selection and navigation, and voice commands. In some examples, additional usage data may be received, such as, for example, the geographic location of the ultrasound machine, the type of ultrasound probe used, a unique user identifier, the type of exam, and / or the object being imaged by the ultrasound imaging system 100. In some examples, some additional data may be provided by the user via the user interface 124, the image processor 136, and / or may be pre-programmed and stored within the ultrasound imaging system 100 (e.g., in the local memory 142).

[0026]

[0046] The UI adapter 170 may capture and analyze usage data live. That is, the UI adapter 170 may receive and analyze usage data while a user is interacting with the ultrasound imaging system 100 via the user interface 124. In these examples, the UI adapter 170 may automatically adapt the user interface 124 based at least in part on the usage data while the user is interacting with the user interface 124. However, in some examples, the UI adapter 170 may automatically adapt the user interface 124 when the user is not interacting with the user interface 124 (e.g., when a workflow is paused or an exam is completed). Instead of or in addition to live analysis, the UI adapter 170 may capture and store the usage data (e.g., as a log file in the local memory 142) and later analyze the stored usage data. In some examples where usage data is later analyzed, the UI adapter 170 may automatically adapt the user interface 124, but these adaptations may not be provided to the user until the next time the user interacts with the ultrasound imaging system 100 (e.g., when the user initiates the next step in a workflow, when the user next logs into the ultrasound imaging system 100). Additional details of example adaptations of the user interface 124 that the UI adapter 170 may perform are described with reference to FIGS.

[0027]

[0047] In some examples, UI adapter 170 may include and / or implement any one or more machine learning models, deep learning models, artificial intelligence algorithms, and / or neural networks that can analyze usage data and adapt user interface 124. In some examples, UI adapter 170 may include a long short term (LSTM) model, a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), an autoencoder neural network, or the like to adapt control panel 152 and / or display 138. The models and / or neural networks may be implemented with hardware components (e.g., neurons represented by physical components) and / or software components (e.g., neurons and pathways implemented in a software application). Models and / or neural networks implemented in accordance with the present disclosure may use various topologies and learning algorithms to train the models and / or neural networks to generate desired outputs. For example, a software-based neural network may be implemented using a processor configured to execute instructions (e.g., a single or multi-core CPU, a single GPU or GPU cluster, or multiple processors arranged for parallel processing). The instructions may be stored in a computer-readable medium, and when executed, the processor executes trained algorithms to adapt the user interface 124 (e.g., determining the most and / or least used control elements, predicting the next control element to be subsequently selected by the user, changing the appearance of control elements shown on the display 138, changing the function of physical control elements on the control panel 152). In some embodiments, the UI adapter 170 may implement the model and / or neural network in combination with other data processing methods (e.g., statistical analysis).

[0028]

[0048] In various embodiments, the model and / or neural network may be trained using any of a variety of now-known or later-developed learning techniques to obtain a model and / or neural network (e.g., a trained algorithm, transfer function, or hardware-based node system) configured to analyze input data in the form of screen touches, keystrokes, control element manipulations, usage log files, other user input data, ultrasound images, measurements, and / or statistics. In some embodiments, the model and / or neural network may be trained statically; that is, the model and / or neural network may be trained using a data set and deployed on the UI adapter 170. In some embodiments, the model and / or neural network may be trained dynamically. In these embodiments, the model and / or neural network may be trained using an initial data set and deployed on the ultrasound system 100. However, the model and / or neural network may continue to be trained and modified based on inputs obtained by the UI adapter 170 after the model and / or neural network has been deployed on the UI adapter 170.

[0029]

[0049] 1, UI adapter 170 need not be physically located within user interface 124 or in close proximity to user interface 124. For example, UI adapter 170 may be co-located with processing circuit 150.

[0030]

[0050] In some embodiments, the various components shown in FIG. 1 may be combined. For example, in some examples, a single processor may implement multiple components of processing circuit 150 (e.g., image processor 136, graphics processor 140) and UI adapter 170. In some embodiments, the various components shown in FIG. 1 may be implemented as separate components. For example, signal processor 126 may be implemented as a separate signal processor for each imaging mode (e.g., B-mode, Doppler, SWE). In some embodiments, one or more of the various processors shown in FIG. 1 may be implemented by a general-purpose processor and / or microprocessor configured to perform specific tasks. In some embodiments, one or more of the various processors may be implemented as application-specific circuitry. In some embodiments, one or more of the various processors (e.g., image processor 136) may be implemented using one or more graphics processing units (GPUs).

[0031]

[0051] Figure 2 is a block diagram illustrating an example processor 200 in accordance with the principles of the present disclosure. Processor 200 may be used to implement one or more of the processors and / or controllers described herein, such as image processor 136, graphics processor 140, and / or UI adapter 170 shown in Figure 1, and / or any other processor or controller shown in Figure 1. Processor 200 may be any suitable processor type, including, but not limited to, a microprocessor, a microcontroller, a digital signal processor (DSP), a field programmable gate array (FPGA) programmed to form a processor, a graphics processing unit (GPU), an application specific circuit (ASIC) designed to form a processor, or a combination thereof.

[0032]

[0052] Processor 200 may include one or more cores 202. Core 202 may include one or more arithmetic logic units (ALUs) 204. In some embodiments, core 202 may include a floating point unit (FPLU) 206 and / or a digital signal processing unit (DSPU) 208 in addition to or instead of ALU 204.

[0033]

[0053] The processor 200 may include one or more registers 212 communicatively coupled to the core 202. The registers 212 may be implemented using dedicated logic gate circuits (e.g., flip-flops) and / or any memory technology. In some embodiments, the registers 212 may be implemented using static memory. The registers may provide data, instructions, and addresses to the core 202.

[0034]

[0054] In some embodiments, processor 200 may include one or more levels of cache memory 210 communicatively coupled to cores 202. Cache memory 210 may provide computer-readable instructions to cores 202 for execution. Cache memory 210 may provide data for processing by cores 202. In some embodiments, computer-readable instructions may be provided to cache memory 210 by local memory, e.g., local memory attached to external bus 216. Cache memory 210 may be implemented using any suitable cache memory type, such as, for example, metal-oxide-semiconductor (MOS) memory, e.g., static random access memory (SRAM), dynamic random access memory (DRAM), and / or any other suitable memory technology.

[0035]

[0055] Processor 200 may include a controller 214 that can control input to processor 200 from other processors and / or components included in the system (e.g., control panel 152 and scan converter 130 shown in FIG. 1 ) and / or output from processor 200 to other processors and / or components included in the system (e.g., display 138 and volume renderer 134 shown in FIG. 1 ). Controller 214 may control data paths within ALU 204, FPLU 206, and / or DSPU 208. Controller 214 may be implemented as one or more state machines, data paths, and / or dedicated control logic. Gates in controller 214 may be implemented as standalone gates, FPGAs, ASICs, or other suitable technologies.

[0036]

[0056] Registers 212 and cache memory 210 may communicate with controller 214 and core 202 via internal connections 220A, 220B, 220C, and 220D. The internal connections may be implemented as buses, multiplexers, crossbar switches, and / or other suitable connection technologies.

[0037]

[0057] Inputs and outputs of processor 200 may be provided via bus 216, which may include one or more conductive lines. Bus 216 may be communicatively coupled to one or more components of processor 200, such as controller 214, cache memory 210, and / or registers 212. Bus 216 may be coupled to one or more components of the system, such as display 138 and control panel 152, described above.

[0038]

[0058] The bus 216 may be coupled to one or more external memories. The external memory may include a read-only memory (ROM) 232. The ROM 232 may be masked ROM, an electronically programmable read-only memory (EPROM), or any other suitable technology. The external memory may include a random access memory (RAM) 233. The RAM 233 may be static RAM, battery-backed static RAM, dynamic RAM (DRAM), or other suitable technology. The external memory may include an electrically erasable programmable read-only memory (EEPROM) 235. The external memory may include flash memory 234. The external memory may include a magnetic storage device such as a disk 236. In some embodiments, the external memory is included in a system such as the ultrasound imaging system 100 shown in FIG. 1 and may be, for example, the local memory 142.

[0039]

[0059] Next, a more detailed description of an example adaptation of the UI of an ultrasound imaging system based on usage data from one or more users is provided, according to an example of the present disclosure.

[0040]

[0060] 3 illustrates a portion of an ultrasound imaging system according to an example of the present disclosure. Ultrasound imaging system 300 may be included in and / or used to implement ultrasound imaging system 100. Ultrasound imaging system 300 may include a display 338 and a control panel 352, which may be included as part of a user interface 324 of ultrasound imaging system 300. In some examples, display 338 may be used to implement display 138, and / or in some examples, control panel 352 may be used to implement control panel 152.

[0041]

[0061] The control panel 352 may include one or more hard control elements that can be manipulated by a user to control the ultrasound imaging system 300. In the example shown in FIG. 3 , the control panel 352 includes a button 302, a trackball 304, a knob (e.g., a dial) 306, and a slider 308. In some examples, the control panel 352 may include fewer, additional, and / or different hard control elements, such as a keyboard, a trackpad, and / or a rocker switch. In some examples, the control panel 352 may include a flat-panel touchscreen 310. The touchscreen 310 may provide soft control elements that are manipulated by a user to operate the ultrasound imaging system 300. Examples of soft control elements include, but are not limited to, buttons, sliders, and gesture control elements (e.g., a two-point contact "pinch" motion to zoom out, a single-point contact "drag" to draw a line or move a cursor). In some examples, the display 338 may be a touchscreen that provides the soft control elements. In other examples, touchscreen 310 may be omitted, and display 338 may be a touchscreen that provides soft control elements. In some examples, display 338 may not be a touchscreen, but rather may display soft control elements that can be selected by a user by manipulating one or more hard control elements on control panel 352. For example, a user may use trackball 304 to manipulate a cursor on display 338 and select a soft control element on display 338 by clicking button 302 when the cursor is over the desired button. In these examples, touchscreen 310 may optionally be omitted. In some examples, additional hard control elements not shown in FIG. 3 may be provided around display 338. In some examples, control panel 352 may include a microphone for accepting voice commands or input from a user.Although not shown in FIG. 3, if the ultrasound imaging system 300 is included in a cart-based ultrasound imaging system, the user interface 324 may further include a foot pedal that can be operated by a user to provide input to the ultrasound imaging system 300.

[0042]

[0062] As described herein, in some examples, the control panel 352 may include hard control elements 312 with variable functionality. That is, the functions performed by the hard control elements 312 are not fixed. In some examples, the functions of the hard control elements 312 are changed by commands executed by a processor of the ultrasound imaging system 300, such as the UI adapter 170. The processor may change the function of the hard control elements 312 based at least in part on usage data received by the ultrasound imaging system 300 from a user. Based on an analysis of the usage data, the ultrasound imaging system 300 may predict the next function the user will select. Based on this prediction, the processor may assign the predicted next function to the hard control elements 312. Optionally, in some examples, the analysis and prediction of the usage data may be performed by a processor different from the processor that adapts (e.g., changes) the function of the hard control elements 312. In some examples, the hard control elements 312 may have an initial function assigned based on the exam type and / or default settings programmed into the ultrasound imaging system 300. In other examples, the hard control element 312 may have no function prior to the first input from the user (e.g., stop). Although the hard control element 312 is shown as a button in Figure 3, in other examples, the hard control element 312 may be a dial, a switch, a slider, and / or any other suitable hard control element (e.g., a trackball).

[0043]

[0063] FIG. 4 illustrates soft control elements provided on a display according to an example of the present disclosure. The soft control elements 400 may be provided on a touchscreen of an ultrasound imaging system, such as the touchscreen 310 shown in FIG. 3. The soft control elements 400 may additionally or alternatively be provided on a non-touch display, such as the display 338 and / or the display 138, and a user may interact with the soft control elements 400 by manipulating hard control elements on a control panel, such as the control panel 352 and / or the control panel 152. For example, a user may use the trackball 304 to move a cursor over one or more of the soft control elements 400 on the display 338 and press the button 302 to operate the soft control elements 400 and operate the ultrasound imaging system 300. Although the soft control elements 400 shown in FIG. 4 are all buttons, the soft control elements 400 may be any combination of soft control elements (e.g., buttons, sliders), and any number of soft control elements 400 may be provided on the display.

[0044]

[0064] As described with reference to FIG. 3 , the control panel may include hard control elements (e.g., hard control element 312) with variable functions. Similarly, as described herein, one or more of the soft control elements 400 may have variable functions. As shown in panel 401, soft control element 402 may initially be assigned a first function FUNC1 (e.g., freeze, acquire). The first function FUNC1 may be based at least in part on the specific user logged in, the exam type selected, or a default function stored in the ultrasound imaging system (e.g., ultrasound imaging system 100 and / or ultrasound imaging system 300). Alternatively, in some examples, soft control element 402 may be disabled in panel 401.

[0045]

[0065] A user can provide input to the ultrasound imaging system to select a control, such as by touching or otherwise manipulating one of the soft control elements 400 and / or by manipulating a hard control element on a control panel (not shown in FIG. 4 ). In some examples, the user may touch the soft control element 402. At least in part in response to the user's input, the function of the soft control element 402 may change to a second function FUNC2 (e.g., acquire, annotate), as shown in panel 403. The second function FUNC2 may be assigned based on a prediction of which function the user will select next. In some examples, the prediction may be based at least in part on an analysis of usage data. The usage data may include user input provided in panel 401 and / or may include past usage data (e.g., previous user input during the exam, user input from past exams). In some examples, the analysis and prediction may be performed by a processor of the ultrasound imaging system, such as the UI adapter 170.

[0046]

[0066] A user can provide a second input to the ultrasound imaging system, for example, by touching or otherwise manipulating one of the soft control elements 400 and / or by manipulating a hard control element on a control panel. In some examples, the user may touch the soft control element 402, for example, when the processor correctly predicts the next function to be selected by the user. At least in part, in response to the user's input, the function of the soft control element 402 may be changed by the processor to a third function FUNC3 (e.g., annotate, caliper), as shown in panel 405. Similar to the above, the third function FUNC3 may be assigned based on an analysis of usage data. For example, the function assigned as the third function FUNC3 may differ depending on whether the user used the second function FUNC2 (e.g., the processor correctly predicted) or selected another function (e.g., the processor incorrectly predicted).

[0047]

[0067] A user can provide a third input to the ultrasound imaging system, for example, by touching or otherwise manipulating one of the soft control elements 400 and / or by manipulating a hard control element on a control panel. In some examples, the user may touch soft control element 402. At least in part in response to the user's input, the function of soft control element 402 may be changed by the processor to a fourth function, FUNC4 (e.g., update, depth change), as shown in panel 407. As above, the fourth function, FUNC4, may be assigned based on an analysis of usage data. For example, the function assigned as fourth function, FUNC4, may differ depending on whether the processor provided a correct prediction in panels 403 and 405. While changes in the function of soft control element 402 for three user inputs are shown, the function of soft control element 402 may be changed for any number of user inputs. Furthermore, in some examples, the provided user input may be stored for future analysis performed by the processor to predict the next function the user will desire during a subsequent examination.

[0048]

[0068] By changing the function of one or more hard control elements (e.g., hard control element 312) and / or one or more soft control elements (e.g., soft control element 402), a user can continue to use the hard or soft control elements for various functions during an examination. In some applications, this may reduce the need for a user to search for a control element for a desired function on a user interface (e.g., user interface 124, user interface 324). Reducing the need for searching may save time and increase the efficiency of the examination. In some applications, using a single hard control element for multiple functions may improve the ergonomics of an ultrasound imaging system (e.g., ultrasound imaging system 100, ultrasound imaging system 300).

[0049]

[0069] FIG. 5A illustrates soft control elements provided on a display according to an example of the present disclosure. The soft control elements 500 may be provided on a touchscreen of an ultrasound imaging system, such as touchscreen 310 shown in FIG. 3. The soft control elements 500 may additionally or alternatively be provided on a non-touch display, such as display 338 and / or display 138, and a user may interact with the soft control elements 500 by manipulating hard control elements on a control panel, such as control panel 352 and / or control panel 152. Although the soft control elements 500 shown in FIG. 5A are all buttons, the soft control elements 500 may be any combination of soft control elements, and any number of soft control elements 500 may be provided on a display.

[0050]

[0070] In the example shown in FIG. 5A , a processor (e.g., UI adapter 170) of the ultrasound imaging system may analyze usage data to determine which soft control elements 500 are most frequently used by the user. Based on the analysis, the processor may adjust the appearance of less frequently used soft control elements 500. In some examples, such as the example shown in panel 501, all of the soft control elements 500 may initially have the same appearance. After a period of user use (e.g., one or more user inputs, one or more exams), the appearance of at least one of the soft control elements 500 may be changed by the processor. For example, as shown in panel 503, a less frequently used soft control element 502 may have a lighter appearance (e.g., dimmer and more transparent than the other control elements) compared to the remaining soft control elements 500. In some examples, dimming can be achieved by reducing the display backlight on the soft control element 502. In some examples, the degree of difference in appearance may become more noticeable after further use by the user of the ultrasound system (e.g., additional user inputs, additional exams). For example, it may become increasingly pale (eg, dimmer, more transparent) over time.

[0051]

[0071] Optionally, in some examples, as shown in FIG. 5B , one or more infrequently used soft control elements 502 shown in panel 505 may be eventually removed from the display by the processor, as shown in panel 507. The removal of the soft control elements 502 may be based at least in part on an analysis of usage data and may occur, for example, when the usage data indicates that the user rarely or never selects the soft control element 502 (e.g., when the frequency of use is below a predetermined threshold). The time at which the soft control elements 502 are dimmed and / or removed may vary based on user preferences, the frequency of the user's use of the ultrasound imaging system, and / or preset settings of the ultrasound imaging system. While a blank space 504 is shown where the soft control elements 502 have been removed, in other examples, the processor may replace the removed soft control elements 502 with other soft control elements, for example, soft control elements that usage data indicates the user selects more frequently.

[0052]

[0072] Changing the appearance of infrequently used soft control elements, for example by making them thinner, can help direct a user's attention to the most frequently used soft control elements, thereby reducing the time a user spends searching for a desired control element. Removing unused soft control elements reduces clutter on the UI display and makes desired controls easier to find. However, for some users and applications, simply changing the appearance of infrequently used soft control elements may be preferable because it does not change the UI layout and allows the infrequently used controls to continue to be available.

[0053]

[0073] FIG. 6 illustrates soft control elements provided on a display according to an example of the present disclosure. The soft control elements 600 may be provided on a touchscreen of an ultrasound imaging system, such as touchscreen 310 shown in FIG. 3. The soft control elements 600 may additionally or alternatively be provided on a non-touch display, such as display 338 and / or display 138, and a user may interact with the soft control elements 600 by manipulating hard control elements on a control panel, such as control panel 352 and / or control panel 152. Although the soft control elements 600 shown in FIG. 6 are all buttons, the soft control elements 600 may be any combination of soft control elements, and any number of soft control elements 600 may be provided on a display.

[0054]

[0074] In the example shown in FIG. 6 , a processor (e.g., UI adapter 170) of the ultrasound imaging system may analyze usage data to determine which soft control elements 600 are most frequently used by the user. Based on the analysis, the processor may adjust the appearance of the frequently used soft control elements 600. In some examples, such as the example shown in panel 601, all of the soft control elements 600 may initially have the same appearance. After a period of user use (e.g., one or more user inputs, one or more tests), the appearance of at least one of the soft control elements 600 may be changed by the processor. For example, as shown in panel 603, a frequently used soft control element 602 may have a highlighted appearance (e.g., brighter or a different color than the other control elements) compared to the remaining soft control elements 600. In some examples, the degree of difference in appearance may become more noticeable after further use by the user of the ultrasound system (e.g., additional user inputs, additional tests). For example, the highlighting may increase (e.g., become brighter) over time.

[0055]

[0075] By changing the appearance of frequently used soft control elements, for example by highlighting them, the user's attention can be more easily directed to the most frequently used soft control elements, thereby reducing the time the user spends searching for the desired control element.

[0056]

[0076] FIG. 7 illustrates soft control elements provided on a display according to an example of the present disclosure. Soft control elements 700 may be provided on a touchscreen of an ultrasound imaging system, such as touchscreen 310 shown in FIG. 3. Soft control elements 700 may additionally or alternatively be provided on a non-touch display, such as display 338 and / or display 138, and a user may interact with soft control elements 700 by manipulating hard control elements on a control panel, such as control panel 352 and / or control panel 152. Although the soft control elements 700 shown in FIG. 7 are all buttons, soft control elements 700 may be any combination of soft control elements (e.g., buttons, sliders), and any number of soft control elements 700 may be provided on a display.

[0057]

[0077] 6, one or more soft control elements 700 may be highlighted (e.g., brighter, a different color). In some examples, a soft control element 700 may be highlighted based on a prediction by the ultrasound imaging system's processor (e.g., UI adapter 170) of which soft control element 700 the user will select. The prediction may be based at least in part on user usage data from previous user inputs during the exam and / or user inputs from past exams.

[0058]

[0078] As shown in panel 701, soft control element 700a may be highlighted first. The first soft control element 700 to be highlighted may be based at least in part on the particular user logged in, the exam type selected, or default functions stored in the ultrasound imaging system (e.g., ultrasound imaging system 100 and / or ultrasound imaging system 300). Alternatively, in some examples, no soft control elements 700 may be highlighted in panel 701.

[0059]

[0079] A user can provide input to the ultrasound imaging system by, for example, touching or otherwise manipulating one of the soft control elements 700 and / or manipulating a hard control element on a control panel (not shown in FIG. 7 ). In some examples, the user may touch soft control element 700a. As shown in panel 703, a different soft control element, such as soft control element 700d, may be highlighted at least in part in response to the user's input. Soft control element 700d may be highlighted based on a prediction of which function the user will select next. In some examples, the prediction may be based at least in part on an analysis of usage data. The usage data may include user input provided in panel 701 and may also include historical usage data (e.g., previous user input during the exam, user input from past exams). In some examples, the processor may change the appearance of soft control element 700 by executing one or more commands.

[0060]

[0080] A user can provide a second input to the ultrasound imaging system, for example, by touching or otherwise manipulating one of the soft control elements 700 and / or by manipulating a hard control element on a control panel. In some examples, the user may touch soft control element 700d, for example, when the processor correctly predicts the next function the user desires. At least in part in response to the user's input, the highlighted soft control element may be changed by the processor, for example, to soft control element 700c as shown in panel 705. Similar to the above, soft control element 700c may be highlighted based on an analysis of usage data. For example, the soft control element highlighted in panel 705 may differ depending on whether the user used soft control element 700d (e.g., the processor predicted correctly) or selected a different soft control element (e.g., the processor predicted incorrectly).

[0061]

[0081] A user can provide a third input to the ultrasound imaging system, for example, by touching or otherwise manipulating one of the soft control elements 700 and / or by manipulating a hard control element on a control panel. In some examples, the user may touch soft control element 700c. At least in part in response to the user's input, the highlighted soft control element may be changed by the processor, for example, to soft control element 700e as shown in panel 707. As above, soft control element 700e may be highlighted based on an analysis of usage data. For example, the soft control element highlighted in panel 707 may differ depending on whether the processor provided a correct prediction in panels 703 and 705. While a change in the highlighting of soft control element 700 for three user inputs is shown, the highlighting of soft control element 700 may be changed for any number of user inputs. Furthermore, in some examples, the provided user input may be stored for future analysis by the processor to predict the next function the user desires during a subsequent examination.

[0062]

[0082] By highlighting the soft control element that the user is most likely to use next, the user can find the desired soft control element more quickly. Furthermore, in protocol-heavy areas, highlighting the soft control element that is most likely to be used next can prevent the user from accidentally skipping steps in the protocol.

[0063]

[0083] While the examples shown in Figures 5-7 have been described with reference to soft control elements, depending on the application, these examples may also be applied to hard control elements. For example, in an ultrasound imaging system with backlit hard control elements and / or hard control elements adjacent to light sources in a control panel, the backlight and / or other light source may be set to a higher intensity and / or a different color to highlight desired hard control elements. Conversely, the backlight and / or other light source may be dimmed or turned off to "dim" less frequently used hard control elements. In other words, the illumination of hard control elements may be adapted based on usage data.

[0064]

[0084] 8A illustrates soft control elements provided on a display according to an example of the present disclosure. Soft control elements 800 may be provided on a touchscreen of an ultrasound imaging system, such as touchscreen 310 shown in FIG. 3. Soft control elements 800 may additionally or alternatively be provided on a non-touch display, such as display 338 and / or display 138, and a user may interact with soft control elements 800 by manipulating hard control elements on a control panel, such as control panel 352 and / or control panel 152. Although soft control elements 800 shown in FIG. 8A are all buttons, soft control elements 800 may be any combination of soft control elements, and any number of soft control elements 800 may be provided on a display.

[0065]

[0085] In the example shown in FIG. 8A , a processor (e.g., UI adapter 170) of the ultrasound imaging system may analyze usage data to determine which soft control elements 800 are being used most frequently by the user. Based on the analysis, the processor may adjust the placement of the soft control elements 800. In some examples, such as the example shown in panel 801, the soft control elements 800 may have an initial placement. After a period of user use (e.g., one or more user inputs, one or more tests), the placement of the soft control elements 800 may be changed by the processor. For example, as shown in panel 803, the soft control elements determined to be used most frequently (e.g., 800g, 800e, and 800f) may be moved to the top of the display, and the soft control elements determined to be used least frequently (e.g., 800a, 800h, 800c) may be moved to the bottom of the display.

[0066]

[0086] In some examples, soft control element 800 may be part of a menu that includes multiple pages, as shown by panels 809 and 811 in FIG. 8B . A user can navigate between pages of the menu by “swiping” or by manipulating hard control elements on a control panel. Multiple pages may be used, for example, when there are more soft control elements than can be simultaneously displayed on the display. In addition to rearranging soft control element 800 on the display, rearranging soft control element 800 may include changing which page of the menu on which soft control element 800 appears, as shown by panels 813 and 815. In some examples, less frequently used control elements (e.g., 800e, 800f, 800g) may be moved from page 1 to page 2 of the menu, and more frequently used control elements (e.g., 800p, 800q, 800r) may be moved from page 2 to page 1. In the example shown in FIG. 8B, the locations of soft control elements 800e, 800f, and 800g are swapped with the locations of soft control elements 800p, 800q, and 800r, although adjusting the placement of soft control elements 800 on a page of a menu or between pages of a menu is not limited to this example.

[0067]

[0087] By moving frequently used control elements to the top of the display and / or to the first page of a menu, the frequently used control elements are more visible and easier to find for the user. In some applications, this may reduce the time a user spends navigating through pages of a menu to find a desired control element. However, some users may dislike and / or find the automatic rearrangement of soft control elements confusing. Therefore, in some examples, the ultrasound imaging system may allow the user to provide a user input to override this setting.

[0068]

[0088] 8A is described with reference to soft control elements, it can also be applied to hard control elements of an ultrasound imaging system. In an example where an ultrasound imaging system includes multiple reconfigurable hard control elements, more frequently used functions may be assigned to the hard control elements most easily accessible to the user, and less frequently used functions may be assigned to more remote locations on the control panel.

[0069]

[0089] In some examples, the ultrasound imaging system can automatically adapt its user interface based on input provided by the user as well as the identity of the object being imaged. In some examples, a processor of the ultrasound imaging system, such as image processor 136, may identify the anatomical structure currently being scanned by an ultrasound probe, such as ultrasound probe 112. In some examples, the processor may implement an artificial intelligence / machine learning model trained to identify anatomical features within ultrasound images. Examples of techniques for identifying anatomical features within ultrasound images can be found in PCT Application PCT / EP2019 / 084534, filed December 11, 2019, entitled “SYSTEMS AND METHODS FOR FRAME INDEXING AND IMAGE REVIEW.” The ultrasound imaging system may adapt its user interface based on the identified anatomical feature, for example, by displaying soft controls for functions most commonly used in imaging the identified anatomical feature.

[0070]

[0090] 9 shows an example of an ultrasound image on a display and example of soft control elements provided on the display, according to an example of the present disclosure. Displays 900 and 904 may be included in an ultrasound imaging system, such as ultrasound imaging system 100 and / or ultrasound imaging system 300. In some examples, display 900 may be included within or used to implement display 138 and / or display 238. In some examples, display 904 may be included within or used to implement display 138 and / or display 338. In some examples, display 904 may be included within or used to implement touchscreen 310. In some examples, both displays 902 and 904 may be touchscreens.

[0071]

[0091] Display 900 may present an ultrasound image acquired by an ultrasound probe of an ultrasound imaging system, such as ultrasound probe 112. Display 904 may provide soft control elements manipulated by a user to operate the ultrasound imaging system. However, in other examples (not shown in FIG. 9 ), both ultrasound images and soft control elements may be provided on the same display. As noted above, the soft control elements provided on display 904 may depend, at least in part, on the anatomical feature being imaged by the ultrasound probe. In the left side of the example shown in FIG. 9 , display 900 displays an image of a kidney 902. A processor of the ultrasound imaging system, such as image processor 136, may analyze the image and determine that a kidney is being imaged. This determination may be used to determine which soft control elements are provided on display 904. A processor, such as UI adapter 170, may execute commands to modify the soft control elements on display 904 based on the determination. In some examples, the same processor may be used to determine which anatomical feature is being imaged and to adapt the user interface. Continuing with the same example, when the ultrasound imaging system recognizes that a kidney is being imaged, the ultrasound imaging system may provide buttons 906 and sliders 908. In some examples, buttons 906 and sliders 908 may perform functions most commonly used during a kidney exam. In some examples, the processor may further analyze usage data to determine the soft control elements to be provided on display 904. Thus, buttons 906 and sliders 908 may perform functions most commonly used by a particular user during a kidney exam.

[0072]

[0092] On the right side of FIG. 9 , a display 900 displays an image of a heart 910. A processor of the ultrasound imaging system may analyze the image and determine that a heart is being imaged. This determination may be used to adapt soft control elements provided on the display 904. The same processor or another processor may execute commands to change the soft control elements on the display 904 based on the determination. Once the ultrasound imaging system recognizes that a heart is being imaged, the ultrasound imaging system may provide a button 912. In some examples, the button 912 may perform a function most commonly used during an echocardiogram. In some examples, the processor may further analyze usage data to determine the soft control elements provided on the display 904. Thus, the button 912 may perform a function most commonly used by a particular user during an echocardiogram.

[0073]

[0093] While the example of Figure 9 shows two distinct organs, the kidney and the heart, an ultrasound imaging system may be trained to recognize different parts of the same organ or object. For example, it may be trained to recognize different parts of the heart (e.g., left atrium, mitral valve) or different parts of a fetus (e.g., spine, heart, head). The UI may be adapted based on these different parts, as well as completely different organs.

[0074]

[0094] Automatic detection of the anatomical features being imaged and dynamic adjustment of the user interface allows users to more efficiently access the desired controls. Additionally, certain exams, such as fetal scans, may require different tools for different parts of the exam, which could result in an inadequately adapted UI if based solely on exam type.

[0075]

[0095] While Figures 3-9 are described as separate examples, an ultrasound imaging system may implement two or more and / or a combination of the UI adaptation examples described herein. For example, the feature of removing rarely used control elements described with reference to Figure 5B may be combined with the rearrangement of control elements described with reference to Figures 8A-8B to condense a two-page menu into a single page over time. In another example, the example of Figure 3 may be combined with the example of Figure 4 to change the functionality of both hard and soft control elements in the user interface based on a predicted next desired control. In another example, the anatomical feature to be imaged may be determined as described with reference to Figure 9, and the anatomical feature may be used to predict the next control element to be selected by the user as described with reference to Figures 3 and 4. In a further example, similar to the example shown in Figure 9, the set of control elements provided to the user on the display may change dynamically but need not be based on the anatomical feature currently being imaged. For example, the processor may predict the next stage of a user's workflow based on previously selected control elements and provide control elements to be used in that stage (e.g., after anatomical measurements have been taken on the kidney, control elements for Doppler analysis may be provided).

[0076]

[0096] Additionally, other adaptations of the UI that do not directly involve the function, appearance, and / or placement of hard and / or soft control elements may be performed based on the usage data. For example, the processor may adjust default values ​​of the ultrasound imaging system to create custom presets based at least in part on the usage data. Examples of default values ​​that may be changed may include, but are not limited to, imaging depth, 2D thickness, chroma mapping settings, dynamic range, and gray map settings.

[0077]

[0097] According to examples of the present disclosure, an ultrasound imaging system may apply one or more techniques to analyze usage data to provide automatic UI adaptation of the ultrasound imaging system, such as the examples described with reference to Figures 3-9. In some examples, the analysis and / or UI adaptation may be performed by one or more processors (e.g., UI adapter 170) of the ultrasound imaging system.

[0078]

[0098] As disclosed herein, the ultrasound imaging system may receive and store usage data in a computer-readable medium, such as local memory 142. Examples of usage data include, but are not limited to, keystrokes, button presses, other hard control element operations (e.g., turning a dial, flipping a switch), screen touches, other soft control element operations (e.g., swiping, pinching), menu selection and navigation, and voice commands. In some examples, additional usage data may be received, such as the geographic location of the ultrasound system, the type of ultrasound probe used (e.g., type, make, model), a unique user identifier, the type of exam, and / or the object currently being imaged by the ultrasound imaging system. In some examples, the usage data may be provided by a user via a user interface, such as user interface 124, by a processor, such as image processor 136, by the ultrasound probe (e.g., ultrasound probe 112), and / or may be pre-programmed and stored within the ultrasound imaging system (e.g., local memory 142).

[0079]

[0099] In some examples, some or all of the usage data may be written and stored in a computer-readable file, such as a log file, for later retrieval and analysis. In some examples, the log file may store a record of some or all of the user interactions with the ultrasound imaging system. The log file may include time and / or sequence data so that the time and / or sequence of various interactions the user had with the ultrasound imaging system can be determined. The time data may include a timestamp associated with each interaction (e.g., each keystroke, each button press). In some examples, the log file may store a list of interactions in the order in which they occurred so that the sequence of interactions can be determined even if timestamps are not stored in the log file. In some examples, the log file may indicate a particular user associated with an interaction recorded in the log file. For example, if a user logs into the ultrasound imaging system using a unique identifier (e.g., username, password), the unique identifier can be stored in the log file. The log file may be a text file, a spreadsheet, a database, and / or other suitable file or data structure that can be analyzed by one or more processors. In some examples, one or more processors (e.g., UI adapter 170) of the ultrasound imaging system collect usage data and write the usage data to one or more log files, which may be stored on a computer-readable medium. In some examples, the log files and / or other usage data may be received by the imaging system from one or more other imaging systems. The log files and / or other usage data may be stored in local memory. The log files and / or other usage data may be received by any suitable method, including wireless (e.g., Bluetooth, WiFi) and wired (e.g., Ethernet cable, USB device) methods. Thus, usage data from one or more users and one or more imaging systems may be used to adapt the UI of the imaging system.

[0080]

[0100] In some examples, usage data (e.g., usage data stored in one or more log files) may be analyzed using statistical methods. FIG. 10 graphically illustrates an example of a statistical analysis of one or more log files according to an example of the present disclosure. A processor 1000 of an ultrasound imaging system (e.g., ultrasound imaging system 100, ultrasound imaging system 300) may receive one or more log files 1002 to be analyzed. In some examples, processor 1000 may be implemented by UI adapter 170. Processor 1000 may analyze the usage data in log files 1002 to calculate various statistics related to a user interface (e.g., user interface 124, user interface 324) of the ultrasound imaging system and provide one or more outputs 1004. In the example illustrated in FIG. 10 , processor 1000 may determine the total number of times one or more control elements (e.g., button A, button B, button C) were selected (e.g., pressed on a control panel and / or touchscreen) by one or more users, and the percentage likelihood that each of the one or more control elements was selected. In some examples, the percentage of likelihood may be based on the total number of times a particular control element is selected divided by the total number of all control element selections.

[0081]

[0101] In some examples, the output 1004 of the processor 1000 may be used to adapt a user interface (e.g., user interface 124, user interface 324) of the ultrasound imaging system. The user interface may be adapted by the processor 1000 and / or another processor of the ultrasound imaging system. For example, the user interface may be adapted such that control elements that are less likely to be selected are dimmed and / or removed from the display of the user interface, as described with reference to FIGS. 5A-5B. In another example, the user interface may be adapted such that control elements that are more likely to be selected are highlighted on the display of the user interface, as described with reference to FIG. 6. In another example, control elements may be arranged on the display and / or across multiple menus based at least in part on the likelihood of being selected, as described with reference to FIGS. 8A-8B.

[0082]

[0102] FIG. 11 graphically illustrates an example of statistical analysis of one or more log files according to another example of the present disclosure. A processor 1100 of an ultrasound imaging system (e.g., ultrasound imaging system 100, ultrasound imaging system 300) may receive one or more log files 1102 to be analyzed. In some examples, the processor 1100 may be implemented by the UI adapter 170. The processor 1100 may analyze the usage data in the log files 1102 to calculate various statistics related to the ultrasound imaging system's user interface (e.g., user interface 124, user interface 324) and provide one or more outputs 1104, 1106. As shown in FIG. 11 , the processor 1100 may analyze the log files to determine one or more control element selection sequences. The processor 1100 may use a moving window to search for the sequence, search for a specific control element selection that indicates the beginning of a sequence (e.g., “start,” “pause,” “type of exam”), and / or use other methods (e.g., a sequence ends when the time interval between control element selections exceeds a maximum duration). For one or more control element selections that begin a sequence, processor 1100 may calculate a percentage of the likelihood of the next control element being selected. For example, as shown in output 1104, if button A is pressed at the beginning of the sequence, processor 1100 calculates the probability (e.g., a percentage) that one or more other control elements (e.g., buttons B, C, etc.) will be selected next in the sequence. As shown in output 1104, processor 1100 may further calculate the probability that one or more other control elements (e.g., button D, button E, etc.) will be selected after one or more of the other control elements selected after button A. The probability calculations can continue for any desired sequence length.

[0083]

[0103] Based on output 1104, processor 1100 can calculate the most likely sequence of control element selections by the user. As shown in output 1106, it can be determined that button B is most likely to be selected by the user after the user selects button A, and button C is most likely to be selected by the user after the user selects button B.

[0084]

[0104] In some examples, the output 1106 of the processor 1100 may be used to adapt a user interface (e.g., user interface 124, user interface 324) of the ultrasound imaging system. The user interface may be adapted by the processor 1100 and / or another processor of the ultrasound imaging system. For example, the user interface may be adapted such that the function of a hard or soft control element may be changed to the function most likely to be the desired one, as described with reference to Figures 3 and 4. In another example, the user interface may be adapted such that the control element whose function is most likely to be the desired one is highlighted on the display, as described with reference to Figure 7.

[0085]

[0105] Analysis of the log file, including the example statistical analysis described with reference to Figures 10 and 11, may be performed as the usage data is received and recorded in the log file (e.g., live capture), and / or the analysis may be performed at a later time (e.g., upon a pause in the workflow, at the end of an examination, or when the user logs off).

[0086]

[0106] While statistical analysis of log files has been described, in some examples, one or more processors (e.g., UI adapter 170) of the ultrasound imaging system may implement one or more trained artificial intelligence, machine learning, and / or deep learning models (collectively referred to as AI models) to analyze usage data in log files or other formats (e.g., live capture before saving to a log file). Examples of models that may be used to analyze usage data include, but are not limited to, decision trees, convolutional neural networks, and long-short-term memory (LSTM) networks. In some examples, using one or more AI models may enable faster and / or more accurate analysis of usage data and / or faster adaptation of the ultrasound imaging system's user interface in response to the usage data. More accurate analysis of usage data may include, but is not limited to, more accurate prediction of the next control element to be selected in a sequence, more accurate prediction of the control element a particular user is most likely to use during a particular exam type, and / or more accurate determination of the anatomical feature being imaged.

[0087]

[0107] FIG. 12 illustrates a neural network that may be used to analyze usage data according to examples of the present disclosure. In some examples, the neural network 1200 may be implemented by one or more processors (e.g., UI adapter 170, image processor 136) of an ultrasound imaging system (e.g., ultrasound imaging system 100, ultrasound imaging system 300). In some examples, the neural network 1200 may be a convolutional network having one-dimensional and / or multidimensional layers. The neural network 1200 may include one or more input nodes 1202. In some examples, the input nodes 1202 may be organized into layers of the neural network 1200. The input nodes 1202 may be coupled to one or more layers 1208 of a hidden unit 1206 by weights 1204. In some examples, the hidden unit 1206 may perform an operation on one or more inputs from the input nodes 1202 based at least in part on the associated weights 1204. In some examples, hidden unit 1206 may be coupled to one or more layers 1214 of hidden unit 1212 by weights 1210. Hidden unit 1212 may perform an operation on one or more outputs from hidden unit 1206 based at least in part on weights 1210. The output of hidden unit 1212 may be provided to output node 1216 to provide an output (e.g., an inference) of neural network 1200. While only one output node 1216 is shown in FIG. 12, in some examples, a neural network may have multiple output nodes 1216. In some examples, a confidence may be associated with the output. The confidence may be a value greater than or equal to 0 and less than or equal to 1, where a confidence of 0 indicates that neural network 1200 is not confident that the output is correct and a confidence of 1 indicates that neural network 1200 is 100% confident that the output is correct.

[0088]

[0108] In some examples, inputs to the neural network 1200 provided at one or more input nodes 1202 may include log files, live capture usage data, and / or images acquired by an ultrasound probe. In some examples, the output provided at output node 1216 may include a prediction of the next control element to be selected in a sequence, a prediction of control elements likely to be used by a particular user, control elements likely to be used during a particular exam type, and / or control elements likely to be used when a particular anatomical feature is being imaged. In some examples, the output provided at output node 1216 may include a determination of the anatomical image currently being imaged by an ultrasound probe (e.g., ultrasound probe 112) of an ultrasound imaging system.

[0089]

[0109] The output of the neural network 1200 may be used by the ultrasound imaging system to adapt (e.g., adjust) the user interface (e.g., user interface 124, user interface 324) of the ultrasound imaging system. In some examples, the neural network 1200 may be implemented by one or more processors (e.g., UI adapter 170, image processor 136) of the ultrasound imaging system. In some examples, the one or more processors (e.g., UI adapter 170) of the ultrasound imaging system may receive an inference of which control elements are most frequently used (e.g., manipulated, selected) by the user. Based on the inference, the processor may dim and / or remove infrequently used control elements (e.g., as described with reference to FIGS. 5A-5B) or highlight frequently used control elements (e.g., as described with reference to FIG. 6). In some examples, the processor may move control elements that are more likely to be used to the top of the display and / or to the first page of a multi-page menu, as described with reference to FIGS. 8A-8B.

[0090]

[0110] In some examples, the processor may receive multiple outputs from neural network 1200 and / or multiple neural networks that may be used to adapt the user interface of the ultrasound imaging system. For example, the processor may receive an output indicating the anatomical feature currently being imaged by an ultrasound probe (e.g., ultrasound probe 112) of the ultrasound imaging system. The processor may also receive an output indicating the control elements that a user most often uses when a particular anatomical feature is being imaged. Based on these outputs, the processor can execute commands to provide the most frequently used control elements on the display, as described with reference to FIG. 9 .

[0091]

[0111] FIG. 13 illustrates a cell of a long-short-term memory (LSTM) model that may be used to analyze usage data according to examples of the present disclosure. In some examples, the LSTM model may be implemented by one or more processors (e.g., UI adapter 170, image processor 136) of an ultrasound imaging system (e.g., ultrasound imaging system 100, ultrasound imaging system 300). LSTM models are a type of recurrent neural network that can learn long-term dependencies. Thus, LSTM models may be suitable for analyzing and predicting sequences, such as the sequence of user selections of various control elements of an ultrasound machine's user interface. LSTM models typically include multiple connected cells. The number of cells may be based, at least in part, on the length of the sequence analyzed by the LSTM. For simplicity, only a single cell 1300 is shown in FIG. 13.

[0092]

[0112] The variable C across the top of cell 1300 is the state of the cell. t-1The state of the cell 1300 may be provided as an input to the cell 1300. Cell 1300 may selectively add or remove data to / from the cell's state. The addition or removal of data may be controlled by three "gates," each containing a separate neural network layer. The modified or unmodified state of cell 1300 may be passed by cell 1300 to C t can be provided to the next LSTM cell as

[0093]

[0113] The variable h across the bottom of cell 1300 is the hidden state vector of the LSTM model. t-1 may be provided as input to cell 1300. The hidden state vector h t-1 is the current input x to the LSTM model provided in cell 1300. t The hidden state vector can be modified by cell 1300C t The modified hidden state vector of cell 1300 can be modified based on the state of output h t The output h t can be provided as a hidden state vector to the next LSTM cell and / or as the output of the LSTM model.

[0094]

[0114] Turning now to the inner workings of cell 1300, the first gate (e.g., forget gate) for controlling the state of cell C includes a first layer 1302. In some examples, this first layer is a sigmoid layer. The sigmoid layer stores the hidden state vector h t-1 and the current input x t The first layer 1302 may receive a concatenation of the output f , which includes weights indicating which data from the previous cell state the cell 1300 should “forget” and which data from the previous cell state the cell 1300 should “remember.” t The previous cell state C t-1 is processed by point operation 1304 as follows: t and removes data that the first layer 1302 determined should be forgotten.

[0095]

[0115] The second gate (e.g., input gate) includes a second layer 1306 and a third layer 1310. Both the second layer 1306 and the third layer 1310 input the hidden state vector h t-1 and the current input x t In some examples, the second layer 1306 is a sigmoid function. The second layer 1306 outputs i which contains weights that indicate what data should be added to the cell state C. t The third layer 1310 may include a tanh function in some examples. t-1 and x t A vector containing all possible data that can be added to the cell state from

number

[0096]

[0116] The third gate (e.g., output gate) includes a fourth layer 1314. In some examples, the fourth layer 1314 is a sigmoid function. The fourth layer 1314 is configured to generate a hidden state vector h t-1 and the current input x t and the hidden state vector h t As the cell state C t Output o containing weights indicating which data should be provided t Cell state C t The data is converted into a vector by the tanh function in point processing 1316, and then the data is converted into a vector by point processing 1318. t is multiplied with the hidden state vector / output vector h t In some cases, the output vector h t may be accompanied by a confidence value, similar to the output of a convolutional neural network as described with reference to FIG. 12.

[0097]

[0117] As shown in Figure 13, cell 1300 is an "intermediate" cell, i.e., it receives input C from the previous cell in the LSTM model. t-1 and h t-1 and outputs C to the next cell of the LSTM. t and h t If cell 1300 is the first cell in the LSTM, then the input x t If cell 1300 is the last cell in the LSTM, it receives only the output h t and C t is not provided to another cell.

[0098]

[0118] In some examples where a processor (e.g., UI adapter 170) in an ultrasound imaging system implements an LSTM model, the current input x t may contain data related to the control elements selected by the user and / or other usage data. t-1 may contain data related to previous predictions of control element selections by the user. t-1 may contain data related to previous choices made by the user. In some examples, the output of the LSTM model, h t may be used by the processor and / or another processor in the ultrasound imaging system to adapt a user interface (e.g., user interface 124, user interface 324) of the ultrasound imaging system. For example, h t includes a prediction of the control element to be next selected by the user, the processor may use the prediction to modify the function of a hard or soft control element, as described with reference to Figures 3 and 4. In another example, the processor may use the prediction to highlight a soft control element on the display, as described with reference to Figure 7.

[0099]

[0119] As described herein, the AI / machine learning model (e.g., neural network 1200 including cell 1300 and LSTM) can provide a confidence associated with one or more outputs. In some examples, the processor (e.g., UI adapter 170) may adapt the UI of the ultrasound imaging system only if the confidence associated with the output is above a threshold (e.g., 50% or above, 70% or above, 90% or above, etc.). In some examples, the processor may not adapt the UI if the confidence is below a threshold. In some examples, this may mean not dimming, highlighting, removing, toggling, and / or rearranging control elements on the display. In some examples, this may mean not changing the functionality of hard or soft control elements (e.g., maintaining existing functionality).

[0100]

[0120] Although convolutional neural networks and LSTM models are described herein, these AI / machine learning models are provided as examples only, and the principles of the present disclosure are not limited to these particular models.

[0101]

[0121] FIG. 14 shows a block diagram of a process for training and deploying a model in accordance with the principles of the present disclosure. The process shown in FIG. 14 may be used to train a model (e.g., an artificial intelligence algorithm, a neural network) included in an ultrasound system, such as a model implemented by a processor (e.g., UI adapter 170) of the ultrasound system. Phase 1 on the left side of FIG. 14 illustrates training the model. To train the model, a training set containing multiple instances of an input array and an output classification may be presented to the model's training algorithm (e.g., Krizhevsky, A., Sutskever, I., and Hinton, G.E., "ImageNet Classification with Deep Convolutional Neural Networks," NIPS 2012, or a derivative thereof). Training may include selecting a starting algorithm and / or network architecture 1412 and preparing training data 1414. The starting architecture 1412 may be a blank architecture (e.g., an architecture with a defined arrangement of layers and nodes but without pre-trained weights, a defined algorithm with or without a set number of regression coefficients), or it may be a partially trained model (e.g., an Inception network) that can later be further tuned for analysis of ultrasound data. The starting architecture 1412 (e.g., blank weights) and training data 1414 are provided to a training engine 1410 to train the model. After a sufficient number of iterations (e.g., when the model behaves consistently within an acceptable error), the model 320 is considered trained and ready for deployment. This is shown in Phase 2 in the center of FIG. 14. On the right side of FIG. 14, or Phase 3, the trained model 1420 is applied (via the inference engine 1430) to analyze new data 1432, which is data that was not presented to the model during initial training (Phase 1). For example, the new data 1432 may include unknown data, such as live keystrokes captured from a control panel during a patient scan (e.g., during an echocardiogram).The trained model 1420, implemented via the engine 1430, is used to analyze unknown data according to the training of the model 1420 and provides output 1434 (e.g., least frequently used button on the display, likely next input, anatomical feature being imaged, confidence level). The output 1434 may then be used by the system for subsequent processes 1440 (e.g., dimming a button on the display, changing the function of a hard control element, highlighting a button on the display).

[0102]

[0122] In examples where the trained model 1420 is used as a model implemented or embodied by a processor of an ultrasound system (e.g., UI adapter 170), the starting architecture may be that of a convolutional neural network, a deep convolutional neural network, or a long-short-term memory model, in some examples, and these networks or models may be trained to identify least frequently used or most frequently used control elements, to predict the control element most likely to be selected next, and / or to identify the anatomical feature being imaged. The training data 1414 may include multiple (hundreds, often thousands, or even more) annotated / labeled log files, images, and / or other recorded usage data. It will be understood that the training data need not include complete images or log files generated by the imaging system (e.g., a log file representing all user inputs during an examination, an image representing the entire field of view of the ultrasound probe), but may include patches or portions of log files or images. In various examples, the trained model may be implemented, at least in part, as a computer-readable medium including executable instructions executed by one or more processors of an ultrasound system, such as the UI adapter 170.

[0103]

[0123] As described herein, the ultrasound imaging system may automatically and / or dynamically change the user interface of the ultrasound imaging system based at least in part on usage data from one or more users. However, in some examples, the ultrasound imaging system may allow the user to adjust the user interface. Allowing the user to adjust the user interface may be done in addition to or instead of automatically and / or dynamically changing the user interface by the ultrasound imaging system (e.g., by one or more processors such as UI adapter 170).

[0104]

[0124] 15-19 illustrate examples of how a user of an ultrasound imaging system (e.g., ultrasound imaging system 100, ultrasound imaging system 300) may adjust the ultrasound imaging system's user interface (e.g., user interface 124, user interface 324). In some examples, a user may provide user input to adjust the user interface. The input may be provided via a control panel (e.g., control panel 152, control panel 352), which may or may not include a touchscreen (e.g., touchscreen 310). In examples that include a touchscreen, a user may provide input by pressing, tapping, dragging, and / or other gestures. In examples that do not include a touchscreen, a user may provide input via one or more hard control elements (e.g., buttons, dials, sliders, switches, trackballs, mice, etc.). In response to the user input, one or more processors (e.g., UI adapter 170, graphics processor 140) may adapt the user interface. The examples provided with reference to Figures 15-19 are for illustrative purposes only, and the principles of the present disclosure are not limited to these particular ways in which a user may adapt the user interface of an ultrasound imaging system.

[0105]

[0125] FIG. 15 graphically illustrates an overview of how a user moves a button within a page of a menu provided on a display in accordance with an example of the present disclosure. In some examples, the menu may be provided on display 138, display 338, and / or touch screen 310. As shown in panel 1501, user 1502 may press and hold button 1504. In some examples, user 1502 may press and hold a finger on a touch screen (e.g., touch screen 310) on which button 1504 is displayed. In some examples, user 1502 may move a cursor over button 1504 and press and hold a button on a control panel (e.g., control panel 152, control panel 352). After a delay, button 1504 may “pop” out of its original position and “snap” to the position of user 1502’s finger and / or cursor. As shown in panel 1503, user 1502 may drag button 1504 to a new position, as indicated by line 1506, by dragging a finger on a touchscreen or by holding down a button on a control panel and moving a cursor. Button 1504 may follow the finger and / or cursor of user 1502. When user 1502 lifts their finger from the touchscreen or releases the button on the control panel, button 1504 may "snap" to the new position, as shown in panel 1505.

[0106]

[0126] FIG. 16 graphically illustrates an overview of how a user moves a button between pages of a menu on a display according to examples of the present disclosure. In some examples, the menu may be provided on display 138, display 338, and / or touch screen 310. As shown in panel 1601, user 1602 may press and hold button 1604. In some examples, user 1602 may press and hold a finger on a touch screen (e.g., touch screen 310) on which button 1604 is displayed. In some examples, user 1602 may move a cursor over button 1604 and press and hold a button on a control panel (e.g., control panel 152, control panel 352). After a delay, button 1604 may “pop” out of its original position and “snap” to the position of user 1602’s finger and / or cursor. For example, as discussed herein with reference to FIG. 8B , menu 1600 may have multiple pages, as indicated by dot 1608. In some examples, such as FIG. 16 , a colored dot may indicate the current page of menu 1600 being displayed. User 1602 may drag button 1604 to the edge of the menu, as indicated by line 1606, by dragging a finger on the touchscreen or by holding down a button on a control panel and moving a cursor. When user 1602 reaches near the edge of the screen, as shown in panel 1603, menu 1600 may automatically move (e.g., display the next page) to the next page of the menu (page 2 in this example), as shown in panel 1605. When user 1602 removes their finger from the touchscreen or releases the button on the control panel, button 1604 may “snap” to a new position on page 2.

[0107]

[0127] FIG. 17 diagrammatically illustrates an overview of how a user may swap the position of a button on a display in accordance with an example of the present disclosure. In some examples, a menu may be provided on display 138, display 338, and / or touchscreen 310. As shown in panel 1701, user 1702 may press and hold button 1704. In some examples, user 1702 may press and hold a finger on a touchscreen (e.g., touchscreen 310) on which button 1704 is displayed. In some examples, user 1702 may move a cursor over button 1704 and press and hold a button on a control panel (e.g., control panel 152, control panel 352). After a delay, button 1704 may “pop” out of its original position and “snap” to the position of user 1702’s finger and / or cursor. User 1702 may drag button 1704 to a desired position over another button 1706 by dragging a finger on the touchscreen or by holding down a button on a control panel and moving the cursor as shown in panel 1703. As shown in panel 1705, when user 1702 removes his finger from the touchscreen or releases a button on the control panel, button 1704 may "snap" into position with button 1706, which may switch to the original position of button 1704.

[0108]

[0128] FIG. 18 graphically illustrates an overview of how a user may move a group of buttons on a display according to examples of the present disclosure. In some examples, a menu may be provided on display 138, display 338, and / or touch screen 310. As shown in FIG. 18 , in some examples, one or more buttons 1806 may be organized into groups, such as groups 1804 and 1808. As shown in panel 1801, user 1802 may press and hold the heading for group 1804. In some examples, user 1802 may press and hold a finger on a touch screen (e.g., touch screen 310) on which group 1804 is displayed. In some examples, user 1802 may move a cursor over the heading for group 1804 and press and hold a button on a control panel (e.g., control panel 152, control panel 352). After a delay, group 1804 may “pop out” from its original position and “snap” to the position of user 1802’s finger and / or cursor. User 1802 may drag group 1804 to a new position by dragging a finger on a touchscreen or by holding down a button on a control panel and moving a cursor. Group 1804 may follow user 1802's finger and / or cursor. When user 1802 lifts their finger from the touchscreen or releases the button on the control panel, group 1804 may "snap" to the new position, as shown in panel 1803. If a group, such as group 1808, was already in the desired position, group 1808 may move to the original position of group 1804.

[0109]

[0129] FIG. 19 graphically illustrates an overview of how a user changes a rotary control element on a display to a list button according to an example of the present disclosure. In some examples, a menu may be provided on display 138, display 338, and / or touch screen 310. In some examples, some buttons, such as button 1904, may be rotary control elements, and other buttons, such as button 1906, may be list buttons. As shown in panel 1901, user 1902 may press and hold button 1904. In some examples, user 1902 may press and hold a finger on a touch screen (e.g., touch screen 310) on which button 1904 is displayed. In some examples, user 1902 may move a cursor over button 1904 and press and hold the button on a control panel (e.g., control panel 152, control panel 352). After a delay, button 1904 may “pop out” from its original position and “snap” to the position of user 1902’s finger and / or cursor. As shown in panel 1903, user 1902 may drag button 1904 to a new position in the list by dragging a finger on a touchscreen or by holding down a button on a control panel and moving a cursor. Button 1904 may follow user 1902's finger and / or cursor. When user 1902 removes their finger from the touchscreen or releases a button on the control panel, button 1904 may "snap" to the new position. In some examples, if another button, such as button 1906, is in button 1904's desired position, button 1906 may replace button 1904 as the rotation control element. In other examples, button 1906 can be shifted up or down in the list to allow button 1904 to become a list button.

[0110]

[0130] As disclosed herein, an ultrasound imaging system may include a user interface that can be customized by a user. Additionally or alternatively, the ultrasound imaging system may automatically adapt the user interface based on usage data of one or more users. The ultrasound imaging system disclosed herein can provide an adaptive UI that is customized for each user. In some applications, automatically adapting the UI can reduce examination time, improve efficiency, and provide ergonomic benefits to the user.

[0111]

[0131] In various embodiments in which the components, systems, and / or methods are implemented using a computer-based system or a programmable device, such as programmable logic, the systems and methods may be implemented using any of a variety of known or later-developed programming languages, such as "C," "C++," "C#," "Java," and "Python." Accordingly, various storage media, such as magnetic computer disks, optical disks, and electronic memory, may be provided containing information capable of instructing a device, such as a computer, to perform the systems and / or methods. When an appropriate device accesses the information and programs contained on the storage media, the storage media provides the information and programs to the device, enabling the device to perform the functions of the systems and / or methods described herein. For example, if a computer is provided with a computer disk containing appropriate material, such as source files, object files, or executable files, the computer can receive the information, configure itself appropriately, and perform the functions of the various systems and methods outlined in the figures and flowcharts above to achieve its various functions. That is, the computer can receive various portions of information from the disk regarding the various elements of the systems and / or methods, and can perform and coordinate the functions of the individual systems and / or methods.

[0112]

[0132] In connection with this disclosure, it should be noted that the various methods and devices described herein can be implemented as hardware, software, and firmware. Furthermore, the various methods and parameters are merely illustrative and not limiting. In connection with this disclosure, those skilled in the art will be able to implement the present teachings while determining their own techniques and necessary equipment as they affect the technology within the scope of the present invention. The functionality of one or more of the processors described herein may be incorporated into fewer or a single processing unit (e.g., a CPU) or may be implemented using an application-specific integrated circuit (ASIC) or general-purpose processing circuit programmed to perform the functions described herein in response to executable instructions.

[0113]

[0133] While the present system has been described with specific reference to an ultrasound imaging system, it is contemplated that the present system may be extended to other medical imaging systems in which one or more images are systematically acquired. The present system may be applied to acquire and / or record image information related to the kidneys, testes, breasts, ovaries, uterus, thyroid, liver, lungs, musculoskeletal system, spleen, heart, arteries, and vasculature, as well as other imaging applications related to ultrasound-guided interventions. Furthermore, the present system may include one or more programs that enable it to be used with conventional imaging systems and provide the features and advantages of the present system. Additional advantages and features of the present disclosure may be apparent to those skilled in the art upon studying this disclosure or may be experienced by those utilizing the novel systems and methods of the present disclosure. Another advantage of the present system and method is that conventional medical imaging systems can be easily upgraded to incorporate the features and advantages of the present system, device, and method.

[0114]

[0134] It will be appreciated that in accordance with the systems, devices, and methods of this invention, any one of the examples, embodiments, or processes described herein may be combined with one or more other examples, embodiments, and / or processes, or may be separated and / or performed among different devices or portions of devices.

[0115]

[0135] Finally, the foregoing discussion is merely illustrative of the inventive system, and should not be construed as limiting the appended claims to any particular embodiment or group of embodiments. Thus, while the inventive system has been described in particular detail with reference to exemplary embodiments, it should also be understood that various modifications and alternative embodiments can be devised by those skilled in the art without departing from the broad intended spirit and scope of the inventive system as set forth in the following claims. Accordingly, the specification and drawings should be considered in an illustrative sense, and not as a limitation on the appended claims.

Claims

1. 1. A medical imaging system, comprising: a user interface including a plurality of control elements manipulated by a user to modify the operation of the medical imaging system; a memory for storing usage data resulting from said operation of said plurality of control elements; a processor in communication with the user interface and the memory, the processor comprising: receiving said usage data; receiving an indication of a first selected control element of the plurality of control elements that is associated with a first function; determining a predicted next function based at least in part on the usage data and the first function; A medical imaging system that adapts the user interface by changing a function of one of the plurality of control elements to the predicted next function after operation of a first control element.

2. The medical imaging system of claim 1 , wherein the processor changes the function of the first control element to the predicted next function after operation of the first control element.

3. 2. The medical imaging system of claim 1, wherein the user interface includes a control panel, and the processor changes the function of one of a plurality of hard control elements provided on the control panel to the predicted next function.

4. The medical imaging system of claim 1 , wherein the processor implements an artificial intelligence model to analyze the usage data and determine one or more control element selection sequences.

5. The medical imaging system of claim 4 , wherein the artificial intelligence model comprises a long-short-term memory model.

6. 5. The medical imaging system of claim 4, wherein the artificial intelligence model further outputs a confidence level associated with the predicted next function, and the processor adapts the user interface only if the confidence level is greater than or equal to a threshold.

7. The processor adapts the user interface by combining changing the function of one of the plurality of control elements to the predicted next function with increasing visibility of the control element that performs the predicted next function relative to other control elements of the plurality of control elements; 2. The medical imaging system of claim 1, wherein increasing the visibility of a control element that performs the predicted next function relative to other controls of the plurality of control elements includes at least one of increasing brightness or changing color of the control element that performs the predicted next function.

8. The medical imaging system of claim 1 , wherein the usage data further comprises a make and model of an ultrasound probe, a user identifier, a geographic location of the ultrasound imaging system, or a combination thereof.

9. the medical imaging system further comprises an ultrasound probe for acquiring ultrasound signals for generating an ultrasound image; The medical imaging system of claim 1 , wherein the processor further determines anatomical features included in the ultrasound image and determines the predicted next function based at least in part on the anatomical features.

10. 10. The medical imaging system of claim 9, wherein the processor implements a convolutional neural network to analyze the ultrasound images and determine the anatomical features contained in the ultrasound images.

11. 10. The medical imaging system of claim 9, further comprising a second display for displaying the ultrasound image.

Images