Medical imaging systems, devices, and related methods of use

Abstract

A portable computing device of a medical imaging system connectable to a controller includes a medical device connector for connecting a medical device including an imaging device to the computing device, at least one memory for storing instructions, and one or more processors including an image processor. Execution of the instructions by the processor causes the computing device to perform operations, including determining a connection status between the computing device and the controller, and operating the computing device in a first operating mode or a second operating mode based on the connection status, and receiving an image signal from the imaging device. When the computing device is operating in the first operating mode, the image signal is processed by the image processor to generate image data. When the computing device is operating in the second operating mode, the image signal is provided to the controller for processing.

Description

[Technical Field]

[0001] The present disclosure relates generally to medical imaging systems, devices, and related methods of use. More specifically, aspects of the present disclosure relate to medical imaging systems that include a portable computing device detachable and operable independently from a controller for performing image processing to accommodate implementation across a variety of clinical settings. [Background technology]

[0002] Medical procedures performed to image a patient's body lumen using an endoscopic imaging system are often performed in one of two clinical settings. In the first clinical setting, the procedure may be performed in a dedicated area or room, such as an endoscopy suite. In the second clinical setting, the procedure may be performed at the bedside (e.g., if the procedure is urgent or if patient isolation may be required). The first and second clinical settings typically have different needs regarding the portability and / or size constraints of the endoscopic imaging system, as well as the image processing capabilities of the endoscopic imaging system. As a result, two separate types of endoscopic imaging systems have traditionally been used in each setting.

[0003] For example, a mobile endoscopic imaging system including a portable computing device (e.g., a tablet) connected to an endoscope may be used for bedside procedures. This system has less advanced image processing capabilities, but is highly portable, consumes less space, and has a quicker setup time. Alternatively, for procedures performed in dedicated areas, a dedicated controller configured to perform advanced image processing may be used and may be connected to an external display device and / or other system interface for outputting image data. Summary of the Invention

[0004] A portable computing device of the medical imaging system connectable to the controller may include a medical device connector for connecting the medical device to the portable computing device. The medical device may include an imaging device. The portable computing device may also include at least one memory storing instructions and one or more processors, including an image processor, where execution of the instructions by the one or more processors may cause the portable computing device to perform operations. The operations may include determining a connection status between the portable computing device and the controller and operating the portable computing device in a first operating mode or a second operating mode based on the connection status. The operations may also include receiving an image signal from the imaging device. When the portable computing device is operating in the first operating mode, the image signal may be processed by the image processor to generate image data. When the portable computing device is operating in the second operating mode, the image signal may be provided to the controller for processing to generate image data.

[0005] In any of the example portable computing devices disclosed herein, to determine the connection status, a physical connection status between the portable computing device and the controller may be determined, and in response to determining that a physical connection exists between the portable computing device and the controller based on the physical connection status, a communicative connection status between the portable computing device and the controller may be determined. If the physical connection status indicates that a physical connection exists between the portable computing device and the controller, the portable computing device may be operated in a first operating mode. To determine the communicative connection status, a negotiation process between the portable computing device and the controller may be initiated to determine whether a communicative connection threshold is met. In response to determining that the communicative connection threshold is not met, the portable computing device may be operated in the first operating mode. Otherwise, in response to determining that the communicative connection threshold is met, the portable computing device may be operated in a second operating mode.

[0006] In some aspects, the connection status between the portable computing device and the controller may be determined periodically at predefined intervals. In other aspects, the connection status between the portable computing device and the controller may be determined in response to detecting a trigger event. At least one trigger event may include receiving an image signal from an imaging device.

[0007] In a further aspect, when the portable computing device is operating in a first mode of operation, the generated image data may be displayed on a display of the portable computing device, and / or the generated image data may be provided for display on an external display device connected to the portable computing device. When the portable computing device is operating in the first mode of operation, the portable computing device may provide a first user interface on the display of the portable computing device. The first user interface may include a subset of imaging-related function controls corresponding to a subset of operations capable of being performed by the portable computing device. When the portable computing device is operating in a second mode of operation, the portable computing device may provide a second user interface on the display of the portable computing device. The second user interface may include a subset of imaging-related function controls corresponding to operations capable of being performed by the controller.

[0008] In an additional aspect, when the portable computing device is operating in the second mode of operation and an image signal is provided to the controller for processing to generate image data, the image signal may be converted to a standard protocol and the standard protocol may be sent to the controller for processing. The image processor of the controller may be configured to perform more advanced image processing than the image processor of the portable computing device. Additionally, when the portable computing device is operating in the second mode of operation, the generated image data may be received from the controller for display on one or more of the display of the portable computing device or an external display device connected to the portable computing device.

[0009] In some aspects, the portable computing device may also include a connector receptacle configured to receive a connector of the controller to connect the portable computing device to the controller. In other aspects, the portable computing device may be configured to receive one or more connector cables that connect the portable computing device to the controller. The one or more connector cables may connect the portable computing device to the controller via a mount configured to receive the portable computing device.

[0010] A system for medical image processing may include a controller having a first image processor and a portable computing device connectable to the controller. The portable computing device may include a medical device connector for connecting the medical device to the portable computing device. The medical device may include an imaging device. The portable computing device may also include at least one memory storing instructions and one or more processors, including a second image processor, where execution of the instructions by the one or more processors may cause the portable computing device to perform operations. The operations may include determining a connection status between the portable computing device and the controller and operating the portable computing device in a first operating mode or a second operating mode. The portable computing device may be operated in the first operating mode based on a negative connection status and in the second operating mode based on a positive connection status. The operations may also include receiving an image signal from the imaging device. When the portable computing device is operating in the first operating mode, the image signal may be processed by the second image processor to generate image data. When the portable computing device is operating in the second mode of operation, the image signal may be provided to the controller for processing by the first image processor to generate image data.

[0011] In any of the systems disclosed herein, to provide the image signal to the controller for processing, the image signal may be converted to a standard protocol, and the standard protocol may be sent to the controller for processing to generate image data by the first image processor. The first image processor of the controller may be configured to perform more advanced image processing than the second image processor of the portable computing device. In some aspects, the controller may be a docking station having a connector, and the portable computing device further includes a connector receptacle for receiving the connector to connect the portable computing device to the controller. In other aspects, a mount for the controller and the portable computing device may be attached to a mobile stand, and one or more connector cables connect the portable computing device to the controller.

[0012] A method for medical image processing may include determining a connection status between a portable computing device having a first image processor and a controller having a second image processor, wherein an imaging device of the medical device may be connected to the portable computing device. The method may also include operating the portable computing device in a first operating mode or a second operating mode based on the connection status. The portable computing device may be operated in the first operating mode based on the negative connection status and in the second operating mode based on the positive connection status. The method may further include receiving an image signal from the imaging device. When the portable computing device is operating in the first operating mode, the image signal may be processed by the first image processor to generate image data. When the portable computing device is operating in the second operating mode, the image signal may be converted to a standard protocol, and the standard protocol may be transmitted to the controller for processing by the second image processor to generate image data.

[0013] In any of the example methods disclosed herein, determining the connection status may include determining a physical connection status between the portable computing device and the controller, and determining a communicative connection status between the portable computing device and the controller in response to determining that a physical connection exists between the portable computing device and the controller based on the physical connection status.

[0014] It will be understood that both the foregoing general description and the following Detailed Description are exemplary and explanatory only and are not limiting of the invention, as claimed. As used herein, the terms "comprises," "comprising," "including," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or device that comprises a list of elements does not comprise only those elements, but may include other elements not expressly listed or inherent to such process, method, article, or device. The term "exemplary" is used in the sense of "example," not "ideal." The term "distal" refers to a direction away from the operator / toward the treatment site, and the term "proximal" refers to a direction toward the operator. The term "approximately" or similar terms (e.g., "substantially") include values ​​of ±10% of the stated value.

[0015] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments of the present disclosure and, together with the description, serve to explain the principles of the disclosure. [Brief explanation of the drawings]

[0016] [Figure 1] 1 illustrates an exemplary medical imaging system. [Figure 2A] 2 illustrates a first configuration of a portable computing device and controller of the medical imaging system of FIG. 1. [Figure 2B]2 illustrates a first configuration of a portable computing device and controller of the medical imaging system of FIG. 1. [Figure 3A] 2 illustrates a second configuration of the portable computing device and controller of the medical imaging system of FIG. 1. [Figure 3B] 2 illustrates a second configuration of the portable computing device and controller of the medical imaging system of FIG. 1. [Figure 4] 1 illustrates an exemplary process for processing an image signal received from an imaging device. [Figure 5] 1 illustrates an exemplary process for determining a connection status between a portable computing device and a controller. [Figure 6] 1 illustrates an exemplary process for operation mode dependent image processing. [Figure 7] 1 illustrates an example of a computing device. DETAILED DESCRIPTION OF THE INVENTION

[0017] As briefly discussed in the Background section, endoscopic procedures are often performed in one of two clinical settings. As a result, two types of conventional endoscopic imaging systems have emerged that are commonly implemented to accommodate each clinical setting. That is, each commonly implemented system type may have advantages or benefits for the corresponding clinical setting.

[0018] As an example, in a first clinical setting where endoscopic procedures are performed in a dedicated area or room of a medical facility, such as an endoscopy suite, a first conventional endoscopic imaging system may be utilized. In the first conventional endoscopic imaging system, the endoscope may be connected to a dedicated controller. One or more external display devices may also be installed at various locations within the dedicated area or room and connected to the dedicated controller. The dedicated controller may be configured to perform advanced image processing techniques to process image signals received from the endoscope to generate image data for display, for example, via the external display device. The dedicated controller may require a significant amount of power during runtime to perform the advanced image processing techniques. Therefore, the dedicated controller may be relatively fixed or stationary, such that the dedicated controller may be connected to the facility's electrical supply via a wired connection.

[0019] As another example, in a second clinical setting where an endoscopic procedure is performed at a bedside, a second conventional endoscopic imaging system that is more portable than the first conventional endoscopic imaging system may be utilized. For example, the endoscope may be connected to a portable computing device, such as a tablet, capable of performing image processing to generate and / or display image data based on image signals received from the endoscope. Due to the portability and corresponding battery constraints of the portable computing device, the image processing may be less sophisticated than the image processing performed by a dedicated controller in the first conventional type of endoscopic imaging system. However, in a bedside clinical setting, portability may be important given the limited space available for medical personnel and any medical equipment operated by the medical personnel during the procedure (e.g., the benefits of portability may outweigh less sophisticated image processing). In some examples, bedside procedures may be performed when emergency procedures and / or patient isolation are deemed necessary (e.g., to prevent the transmission of infectious pathogens).

[0020] Aspects of the present disclosure relate to a medical imaging system that combines the advantages of both of the conventional systems described above into a single system that can be utilized across different clinical settings. An exemplary medical imaging system may include both a portable computing device and a controller, each having its own image processing capabilities. A medical device, such as an endoscope having an imaging device (e.g., at the distal end of the insertion section of the endoscope), may be connected to the portable computing device. The portable computing device may be connectable to and disconnectable from the controller and may operate in one of two modes based on the connection status.

[0021] For example, when the portable computing device is disconnected from the controller, the portable computing device may operate independently from the controller in a first mode of operation. In the first mode of operation, the portable computing device may be configured to process image signals received from the imaging device to generate and display image data via the portable computing device (e.g., the portable computing device functions as an image processing system similar to a second conventional imaging system). In the first mode of operation, in addition to displaying image data, the portable computing device may also be configured to display a first user interface including a subset of imaging-related function controls for operations performable by the portable computing device.

[0022] When the portable computing device is connected to the controller, the portable computing device may operate in a second operating mode. In the second operating mode, the portable computing device may send image signals from the imaging device to the controller for processing to generate image data (e.g., the controller functions as an image processing system similar to the first conventional imaging system). In addition, the portable computing device may provide a user interface for the controller (e.g., display a second user interface) to enable user interaction with the imaging system. The second user interface may include a more extensive set of image processing-related function controls than the first user interface based on the controller's ability to perform additional imaging-related operations compared to the portable computing device. In some examples, the second user interface may also optionally display image data generated by the controller. The image processing performed by the controller may be more advanced than the imaging processing capable of being performed by the portable computing device, but the imaging system may be less portable overall. The controller may also provide connection to accessory devices, including additional memory for storing image data, wired network connectivity, and / or external devices for triggering image capture by the imaging device.

[0023] Accordingly, aspects disclosed herein include a single medical imaging system that may be easily interchangeable to leverage the benefits of advanced image processing versus increased portability depending on a given clinical setting by connecting or disconnecting a portable computing device to or from the controller. Various configurations of the medical imaging system may be implemented. In one exemplary configuration, the controller may be a relatively fixed docking station to which the portable computing device 102 is docked and undocked to connect to or disconnect from the controller. In another exemplary form providing further portability, the controller and mount may each be attached to a mobile stand. The portable computing device may be docked and undocked to or from the mount to connect to or disconnect from the controller, and the mobile stand may be moved from one area to another within a medical facility.

[0024] 1 illustrates an exemplary medical imaging system 100. The medical imaging system 100 may include two independent image processing systems, such as a portable computing device 102 and a controller 104. The portable computing device 102 may be connectable to the controller 104 to form a combined image processing system 106.

[0025] The portable computing device 102 may be any computing device capable of connecting to and disconnecting from the controller 104. When disconnected from the controller 104, the portable computing device 102 may be highly portable and may be capable of operating independently as an image processing system. While the portable computing device 102 is illustrated and described in examples herein as a tablet, the portable computing device 102 is not limited to tablets. In other examples, the portable computing device 102 may be a laptop computer, a smart cellular phone, a personal digital assistant (PDA) device, or the like. The controller 104 may be a dedicated image processing unit, a high-power processing unit, a base station, a docking station, an auxiliary computing device, or the like capable of operating as an image processing system. In particular, the controller 104 may be configured to implement advanced image processing techniques, such as artificial intelligence (AI) or machine learning-based techniques, which are described in more detail elsewhere herein.

[0026] 2A and 2B illustrate a first configuration of the portable computing device 102 and the controller 104 in which the controller 104 can be a docking station to which the portable computing device 102 docks (e.g., to form a composite image processing system 106). FIGS. 3A and 3B illustrate a second configuration of the portable computing device 102 and the controller 104. In the second configuration, the controller 104 can be attached to a mobile stand, which can also include a mount to which the portable computing device 102 can dock. Docking the portable computing device 102 to the mount can facilitate a connection between the portable computing device 102 and the controller 104 (e.g., to form a composite image processing system 106). The first and second configurations are non-limiting and non-exhaustive configurations of the portable computing device 102 and the controller 104 of the medical imaging system 100. Various other configurations can be implemented.

[0027] The medical imaging system 100 may also include a medical device 108. The medical device 108 may be used to perform diagnostic and / or interventional procedures on a patient. The medical device 108 may be an endoscope or other type of scope, such as a bronchoscope, ureteroscope, duodenoscope, gastroscope, endoscopic ultrasonography (EUS) scope, colonoscope, laparoscope, arthroscope, cystoscope, suction scope, sheath, or catheter, among other examples. The medical device 108 includes an imaging device 110 located at a distal end (e.g., distal tip) of the medical device 108. The imaging device 110 may be configured to capture image signals as the distal end of the medical device 108 is inserted into and guided through a patient's body lumen to a target site during a diagnostic and / or interventional medical procedure. The imaging device 110 may include one or more cameras, one or more image sensors, an endoscopic viewing element, or an optical assembly including one or more image sensors and one or more lenses, among other similar devices. The medical device 108 may also include one or more illumination devices (not shown). The illumination devices (e.g., one or more LEDs, fiber optics, and / or other illuminators) may be configured to illuminate an area of ​​the patient's body (e.g., a target area) during a procedure to facilitate imaging by the imaging device 110. In some examples, the imaging device 110 and the illumination devices may form an imaging system.

[0028] The medical device 108 may be connected to the portable computing device 102 such that image signals captured by the imaging device 110 are received by the portable computing device 102. For example, a connector plug extending from the proximal end of the medical device 108 may connect the medical device 108 to the portable computing device 102. The connector plug may accommodate one or more wires or cables connected to the imaging device 110 to enable image signals captured by the imaging device 110 to be delivered to the portable computing device 102. Depending on the operational mode of the portable computing device 102, the portable computing device 102 may operate as an image processing system that processes the image signals to generate image data, or may operate as a translator that facilitates image processing performed by the controller 104 to generate image data (i.e., the controller 104 operates as an image processing system). Communication between the portable computing device 102 and the controller 104 may be agnostic to the imaging device 110.

[0029] The operational mode of the portable computing device 102 may be based on the connection status between the portable computing device 102 and the controller 104. When the portable computing device 102 and the controller 104 are not connected (e.g., there is a no connection status indicated by a lack of physical and / or communicative connection), the portable computing device 102 may operate in a first operational mode as an image processing system to generate and display image data. User interaction with the displayed image data may be enabled via the display 210 ( FIG. 2B ) of the portable computing device 102. For example, the portable computing device 102 may display a first user interface via the display 210 ( FIG. 2B ) of the portable computing device 102. The first user interface may include a subset of imaging-related function controls selectable by a user for operations capable of being performed by the portable computing device 102. The first operational mode may be the default operational mode of the portable computing device 102.

[0030] In some examples, the portable computing device 102 and the controller 104 may be intentionally disconnected based on a clinical setting that requires the portability enabled by the portable computing device 102 when operated independently of the controller 104, for example. One such clinical setting may be a bedside procedure where available space is limited. In other examples, the portable computing device 102 and the controller 104 may be unintentionally disconnected. For example, the portable computing device 102 may be improperly physically connected to the controller 104 (e.g., a complete physical connection is not achieved), a communicative connection is not established within a threshold period (e.g., a timeout has occurred), and / or the communicative connection between the portable computing device 102 and the controller 104 may be suboptimal (e.g., due to a hardware issue or error in the controller 104).

[0031] Alternatively, when the portable computing device 102 and the controller 104 are connected (e.g., there is a positive connection indicated by a physical and / or communicative connection) to form a combined imaging system 106, the controller 104 may operate as an imaging system to generate image data, while the portable computing device 102 may operate in a second operating mode as a translator. In some examples, the combined imaging system 106 may be utilized in a clinical setting that provides, for example, a dedicated area or room with sufficient space to accommodate the controller 104 and the portable computing device 102 connected thereto. One such clinical setting may be a medical procedure performed in an endoscopy suite. As described in more detail below, image data generated by the controller 104 when the portable computing device 102 is operated in the second operating mode may be displayed via one or more external displays located in the dedicated area and / or may optionally be displayed by the portable computing device 102.

[0032] When operating in the second operating mode, the portable computing device 102 may provide a user interface for the controller 104 to enable user interaction with the controller 104. The user interface for the controller 104 may be a second user interface displayed via the display 210 ( FIG. 2B ) of the portable computing device 102. The second user interface may include a more extensive (e.g., full or complete) set of imaging-related function controls compared to the first user interface based on the capability of the controller 104 to perform additional imaging-related operations than the portable computing device 102. As one example, the second user interface may include control elements for enabling a user to activate the imaging device 110 to capture an image signal from image data generated by the controller 104 and / or record and save a still image and / or a series of still images (e.g., a video). As another example, the second user interface may include control elements for manipulating (e.g., zooming in, zooming out, rotating, cropping, increasing brightness, decreasing brightness, annotating, etc.) the image data generated by the controller 104. As a further example, the second user interface may include a control element for powering the controller on and off. In some examples, the second user interface may also optionally display image data generated by the controller 104.

[0033] In other aspects, the medical device 108 may be optionally connectable to the controller 104. For example, the aforementioned connector plug extending from the proximal end of the medical device 108 may connect the medical device 108 to the controller 104. That is, the connector plug of the medical device 108 may be interchangeably connected to either the portable computing device 102 or the controller 104. Thus, when the portable computing device 102 is not connected to (e.g., detached from) the controller 104, the medical device 108 may alternatively be connected to the controller 104 via the connector plug. In such a case, the controller 104 may be configured to receive and process image signals from the imaging device 110 to generate image data.

[0034] The medical imaging system 100 may also include one or more external devices 112. At least one of the external devices 112 may be a display device (e.g., a monitor, a computing device screen, a touchscreen display device, etc.) connectable to the controller 104 and configured to display image data generated by the controller 104. The external devices 112 may include one or more additional display devices connectable to the controller 104. Optionally, the external devices 112 may also include one or more display devices connectable to the portable computing device 102 and configured to display image data generated by the portable computing device 102 and / or the controller 104. The particular device or devices (e.g., the portable computing device 102 and / or the external devices 112) on which the image data is displayed may be based on the operational mode of the portable computing device 102 and / or user-defined settings, as described in more detail below.

[0035] In some examples, the external device 112 may further include one or more third-party processing systems, such as an AI processing system, connectable to the controller 104 and / or the portable computing device 102. An exemplary AI processing system may be configured to receive as input image data generated by the controller 104 and may be configured to process the image (and optionally other input data) to generate augmented image data. In one example, the augmented image data may visually emphasize or highlight an area of ​​interest, such as a lesion, a polyp, or the like. In another example, the augmented image data may include an overlay representing the predicted position or trajectory of an anatomical structure of interest that is currently occluded or obscured by other anatomical structures in the image data. In a further example, the augmented image data may include an overlay representing the predicted position and / or trajectory of a tool being delivered via the medical device 108.

[0036] One or more components of medical imaging system 100, such as portable computing device 102, controller 104, medical device 108, and / or external device 112, may be network-enabled and may communicate with each other via a wired or wireless network, such as network 120. Network 120 may be an electronic network. Network 120 may include one or more wired and / or wireless networks, such as a wide area network ("WAN"), a local area network ("LAN"), a personal area network ("PAN"), a cellular network (e.g., a 3G network, a 4G network, a 5G network, etc.), etc. In one non-limiting illustrated example, the components of medical imaging system 100 may communicate and / or connect to network 120 via a universal serial bus (USB) or other similar local, low-latency connection or direct wireless protocol.

[0037] In some embodiments, network 120 includes the Internet, and information and data provided between various systems occurs online. "Online" may mean connecting to or accessing source data or information from a location remote from other devices or networks coupled to the Internet. Alternatively, "online" may refer to connecting to or accessing an electronic network (wired or wireless) via a mobile communications network or device. The Internet is a worldwide system of computer networks, a network of networks in which parties at one computer or other device connected to the network can obtain information from any other computer and communicate with parties at other computers or devices. Components of medical imaging system 100 may be connected via network 120 using one or more standard communications protocols such that the components can send and receive communications to each other via network 120.

[0038] In some examples, when one or more components of medical imaging system 100 are connectable to network 120, medical imaging system 100 may also include one or more server-side systems 130. Server-side system 130 may include one or more remote image processing systems configured to perform at least a portion of image processing (e.g., to conserve local resources of portable computing device 102 and / or controller 104 when a network connection is available). Additionally or alternatively, server-side system 130 may include a data storage system for storing image data generated by portable computing device 102 and / or controller 104 and / or augmented image data generated by a third-party processing system (e.g., one or more of external devices 112). In some examples, at least one of the data storage systems may include a picture archiving and communication system (PACS) that stores image data and / or augmented image data along with other types of imaging data from various imaging modalities (e.g., ultrasound, magnetic resonance, nuclear medicine imaging, positron emission tomography, computed tomography, mammograms, digital radiography, histopathology, etc.). Additionally, the server-side system 130 may include an endoscopy report writer system configured to facilitate generation of a report based on the image data and / or the enhanced data.

[0039] 1 as separate components, it should be understood that components or portions of components of medical imaging system 100 may, in some embodiments, be integrated with or incorporated within one or more other components. In some embodiments, the operation or aspects of one or more of the components described above may be distributed among one or more other components. Any suitable arrangement and / or integration of the various systems and devices of medical imaging system 100 may be used.

[0040] Specific examples included throughout this disclosure implement the medical imaging system 100 during medical procedures performed in particular clinical settings (e.g., dedicated areas or rooms versus bedside). However, it should be understood that the techniques according to the present disclosure may be adapted to any clinical setting to take advantage of the advanced image processing of the combined image processing system 106 or the portability of the portable computing device 102 disconnected from the controller 104, to accommodate the particular clinical setting. It should also be understood that the above examples are illustrative only. The techniques and technologies of the present disclosure may be adapted to any suitable activity.

[0041] 2A shows a first configuration 200 of a portable computing device 102 and a controller 104. The controller 104 may be or otherwise include a docking station, and the portable computing device 102 may dock to the controller 104 (e.g., to form a combined image processing system 106). In some examples, the controller 104 may be a relatively fixed docking station located at a given location, for example, in a dedicated area or room of a medical facility where a procedure utilizing a medical device 108 ( FIG. 1 ) may be performed. As described above, the medical device 108 may be connected to the portable computing device 102 to perform the procedure. The portable computing device 102 may be detached or undocked from the controller 104 and operated independently, for example, when increased portability is required for the procedure. The docking and / or undocking (e.g., connection state) of the portable computing device 102 to and from the controller 104 may be detectable by the portable computing device 102 and may cause the portable computing device 102 to operate in either a first mode of operation or a second mode of operation. Image processing of signals received by the imaging device 110 of the medical device 108 during a procedure may depend on the mode of operation of the portable computing device 102, as described below with respect to FIG. 2B .

[0042] 2B shows a block diagram of each of the components of the portable computing device 102 and the controller 104 in the first configuration 200. As shown in FIG. 2B, the portable computing device 102 may include, among other components, one or more processors 202, memory 206, a battery 208, a display 210, one or more medical device connectors 212, and / or connector receptacles 218. At least one of the processors 202 may include an image processor 204 such that the portable computing device 102 has independent image processing capabilities. The portable computing device 102 may also optionally include one or more external video outputs 214 for connecting to external devices 112 (FIG. 1) and / or a communication interface 216 for providing connectivity to the network 120 (FIG. 1).

[0043] The controller 104 may include one or more processors 220, memory 224, a power connector 226, an external video output 228, and / or a portable computing device connector 234. At least one of the processors 220 may include an image processor 222. The controller 104 may optionally include one or more medical device connectors 230 for connecting to one or more medical devices, including, for example, the medical device 108 ( FIG. 1 ), when the medical device 108 is not otherwise connected to the portable computing device 102. In some examples, when the controller 104 includes multiple medical device connectors 230, multiple medical devices (e.g., of the same type or different types) may be connected at a given time to perform procedures in addition to or instead of the medical device 108. For example, an external imaging device (e.g., an X-ray device, a magnetic resonance imaging device, etc.) may be connected to the controller 104 via the medical device connectors 230 in addition to or instead of the medical device 108. The controller 104 may also optionally include a communications interface 232 for providing connectivity to the network 120 (FIG. 1).

[0044] Referring initially to the portable computing device 102, the memory 206 may store instructions that are executed by the processor 202 to cause the portable computing device 102 to perform corresponding operations. The battery 208 may provide power to the processor 202 and / or one or more other components of the portable computing device. The battery 208 may be rechargeable. In some examples, the battery 208 may be automatically recharged when the portable computing device 102 is docked to the controller 104 (e.g., by a power supply accessed via a power connector 226 of the controller 104, as described below). Additionally or alternatively, the battery 208 may be recharged by connecting a battery charging cable from the portable computing device 102 to an external power source (e.g., an electrical socket providing access to a power supply) and / or via wireless charging methods.

[0045] A medical device 108 may be connected to the portable computing device 102 via a medical device connector 212. In some examples, if the portable computing device 102 includes multiple medical device connectors 212, multiple medical devices (e.g., of the same type or different types) may be connected at a given time to perform a procedure in addition to or instead of the medical device 108. For example, an external imaging device (e.g., an X-ray device, a magnetic resonance imaging device, etc.) may be connected to the portable computing device 202 via the medical device connector 212 in addition to or instead of the medical device 108.

[0046] The image processor 204 of the portable computing device 102 may be configured to process image signals to generate image data. In some examples, the image processor 204 may be a basic field-programmable gate array (FPGA), a basic digital signal processing (DSP) processor, a graphics processing unit (GPU), etc., with runtime power requirements that can be met by the battery 208. That is, the image processor 204 may be capable of performing basic image processing operations. The memory 206 may store instructions to cause the image processor 204 to receive and process image signals from the imaging device 110 of the medical device 108 to generate image data when a first set of one or more conditions is met.

[0047] The first set of conditions may include the medical device 108 being connected to the portable computing device 102 via one of the medical device connectors 212 and the portable computing device 102 operating in a first mode of operation. As described with reference to FIG. 1 and in further detail below, the portable computing device 102 may operate in the first mode of operation when the portable computing device 102 is undocked from the controller 104 (e.g., lacks a physical connection to the controller 104) and / or when docked but nevertheless lacks a communicative connection to the controller 104.

[0048] One or more components of the portable computing device 102 may generate or be generated based on instructions / information stored in the memory 206, instructions / information received from other components of the medical imaging system 100, etc. For example, the portable computing device 102 may generate a first user interface and / or a second user interface based on a first or second operating mode of the portable computing device 102, respectively. Furthermore, one or more components of the portable computing device 102 may display a user interface via the display 210 of the portable computing device 102 and / or via one or more other displays (e.g., external device 112). The user interface may include generated image data (e.g., still or moving images), text, input text boxes, selection controls, etc. A given display may include a touch screen or a display with other input devices or systems (e.g., a mouse, a keyboard, etc.) for an operator to control functions of the portable computing device 102.

[0049] For example, display 210 may be a touch display, allowing an operator to interact with image data displayed thereon via touch input (e.g., by the operator's finger) or stylus input. In some examples, other input devices, such as a keyboard or mouse, may be connected to portable computing device 102 to facilitate operator interaction. Additionally or alternatively, if portable computing device 102 includes an optional external video output, image data may be provided for display on any external device 112 connected to portable computing device 102 via external video output 214. In a further example, if portable computing device 102 includes an optional communication interface 216 providing network connectivity, image data may be provided via network 120 to one or more other components of medical imaging system 100, including server-side system 130, among other examples, for further processing and / or storage.

[0050] Additionally, the memory 206 may store instructions that cause the processor 202 to instead convert image signals received from the imaging device 110 to the standard protocol and transmit the standard protocol to the controller 104 for processing when a second set of one or more conditions is met. The second set of conditions may include a medical device 108 being connected to the portable computing device 102 via one of the medical device connectors 212 and the portable computing device 102 operating in a second mode of operation. As described with reference to FIG. 1 and in further detail below, the portable computing device 102 may operate in the second mode of operation when the portable computing device 102 is docked (e.g., both physically and communicatively) to the controller 104 to form a combined image processing system 106. For example, the standard protocol may be transmitted from the portable computing device 102 to the controller 104 via a communication channel that is established when the portable computing device 102 is docked to the controller 104, as described in more detail below.

[0051] Referring now to controller 104, memory 224 may store instructions that are executed by processor 220 to cause controller 104 to perform corresponding operations. Power connector 226 may couple controller 104 to an electrical source for providing power to processor 220 and / or one or more other components of controller 104. For example, power connector 226 may connect to an electrical socket that accesses a medical facility's electrical power supply.

[0052] The image processor 222 of the controller 104 may be configured to process the image signals to generate image data. In some examples, the image processor 222 may be an advanced field programmable gate array (FPGA), an advanced digital signal processing (DSP) processor, or the like. Thus, the image processor 222 of the controller 104 may be configured to perform advanced image processing (e.g., more complex image processing than the image processor 204 of the portable computing device 102). Exemplary advanced image processing techniques may include AI or machine learning-based techniques that require more runtime power. These advanced image processing techniques may be supported by a direct connection of the controller 104 to the healthcare facility's electrical supply via the power connector 226.

[0053] The memory 224 may store instructions for causing the image processor 222 to receive and process image signals from the imaging device 110, converted to a standard protocol by the portable computing device 102, to generate image data when a first set of one or more conditions are met. The first set of conditions may include the medical device 108 being connected to the portable computing device 102 via one of the medical device connectors 212 and the portable computing device 102 operating in a second mode of operation, as described above. The generated image data may be provided for display to one or more of the external devices 112 ( FIG. 1 ) connected to the controller 104 via corresponding external video outputs 228. In some examples, based on user-defined settings, the generated image data may additionally or alternatively be provided to the portable computing device 102 via a connection made via the connector receptacle 218 and the portable computing device connector 234, described in detail below. The image data may then be displayed on the portable computing device 102 (e.g., on the display 210) and / or on any external device 112 connected to the portable computing device 102 via the external video output 214, if the portable computing device 102 includes an optional external video output 214.

[0054] One or more components of the controller 104 may generate or cause to be generated one or more user interfaces based on instructions / information stored in the memory 224, instructions / information received from other components of the medical imaging system 100, etc. For example, in some cases, the controller 104 may cause the portable computing device 102 to generate a second interface when the portable computing device 102 is operating in a second mode of operation. Additionally, one or more components of the controller 104 may cause a user interface to be displayed via the display 210 of the portable computing device 102 and / or via one or more other displays (e.g., external device 112). The user interface may include generated image data (e.g., still or moving images), text, input text boxes, selection controls, etc. A given display may include a touch screen or a display with other input devices or systems (e.g., a mouse, a keyboard, etc.) for an operator to control functions of the portable computing device 102 and / or the controller 104.

[0055] Additionally, if the controller 104 includes optional medical device connectors 230 through which a medical device 108 is connectable to the controller 104, the memory 224 may also store instructions for causing the image processor 222 to receive and process image signals from the imaging device 110 to generate image data when a second set of one or more conditions is met. The second set of conditions may include the medical device 108 being connected to the controller 104 via one of the medical device connectors 230 (e.g., instead of the portable computing device 102, which may or may not be docked to the controller 104). Under such conditions, the generated image data may be provided to one or more external devices 112 connected to the controller 104 via corresponding external video outputs 228 for display. Additionally or alternatively, the generated image data may be provided to the portable computing device 102 when the portable computing device 102 is docked to the controller 104 and providing a user interface (e.g., a second user interface) for the controller 104. The image data may then be displayed on the portable computing device (e.g., on display 210) and / or on any external device 112 connected to portable computing device 102 via external video output 214, if the portable computing device includes an optional external video output 214.

[0056] In a further example, if either the first or second set of conditions are met and the controller 104 includes an optional communication interface 232 that connectably provides a network, the controller 104 may provide the image data to one or more other components of the medical imaging system 100 via the network 120. For example, the controller 104 may provide the image data via the network 120 to the server-side system 130 for further processing and / or storage.

[0057] Docking or connection between the portable computing device 102 and the controller 104 in the first configuration 200 may be based on a connection between a connector receptacle 218 of the portable computing device 102 and a portable computing device connector 234 of the controller 104. For example, the connector receptacle 218 may be configured to receive the portable computing device connector 234. In some examples, the portable computing device connector 234 may be recessed within a docking bay formed in the housing of the controller 104. The docking bay may help support the portable computing device 102 in a readable, touchable, and / or otherwise ergonomic position for an operator. The portable computing device connector 234 may include one or more optical fibers that support high-resolution data transfer (e.g., high-resolution image data) between the portable computing device 102 and the controller 104. The portable computing device connector 234 may also include one or more wires or cables for transmitting power so that power may be provided from the controller 104 to recharge the battery 208 of the portable computing device 102 when the portable computing device 102 is docked to the controller 104 and the controller 104 is connected to an electrical source via the power connector 226. However, as mentioned above, the portable computing device 102 may also be rechargeable independent of docking to the controller 104. In some examples, the portable computing device connector 234 may further include additional wires or cables for transmitting other types of data, such as patient information.

[0058] 2B illustrates the portable computing device 102 as including a connector receptacle 218 and the controller as including a portable computing device connector 234. However, in other examples, the controller 104 may include a connector receptacle configured to receive a controller connector of the portable computing device 102.

[0059] 3A shows a second configuration 300 of portable computing device 102 and controller 104. In second configuration 300, controller 104 and mount 310 may each be attached to or otherwise coupled to a mobile stand 302. Portable computing device 102 may dock to mount 310 to connect portable computing device 102 to controller 104 (e.g., to form a combined image processing system 106). FIG. 3B shows a block diagram of the components included in second configuration 300.

[0060] 3A and 3B simultaneously, mobile stand 302 may include a base 304 configured to support a pole 306. Base 304 may include wheels 308 or other similar transportation mechanism to enable mobile stand 302 to be easily moved from one location to another within a healthcare facility (e.g., between designated areas or rooms, from a designated area or room to a bedside, etc.).

[0061] The controller 104 may be mounted to a base 304 and / or a pole 306 to allow the controller 104 to be moved as the mobile stand 302 is moved (e.g., as opposed to a relatively fixed controller 104 as in the first configuration 200). The controller 104 in the second configuration 300 may include similar components to the controller 104 in the first configuration 200. For example, the controller 104 may include a processor 220 including an image processor 222, a memory 224, a power connector 226, an external video output 228, an optional medical device connector 230, and an optional communication interface 232, each of which are described in detail with reference to FIG. 2B .

[0062] In the second configuration 300, the controller 104 may also include a battery 301. The battery 301 may be a more powerful battery compared to the battery 208, for example, to support the higher runtime power requirements of the image processor 222 (e.g., an advanced image processor). For example, the battery 301 may be an uninterruptible power supply. Thus, the controller 104 and its processor 220 may remain powered by the battery 301 even when the power connector 226 is disconnected from the medical facility's electrical power supply, for example, to move the mobile stand 302 from one location to another within the facility. Similarly, the controller 104 and its processor 220 may remain powered by the battery 301 when, depending on the clinical setting, for example, the mobile stand 302 is not located near an external electrical source. As shown in FIG. 3B , the battery 301 is shown as an integral part of the controller 104. However, in other examples, the battery 301 may be a separate, connectable, standalone component from the controller 104.

[0063] A mount 310, to which the portable computing device 102 may be docked or undocked, may be configured to attach the portable computing device 102 to the pole 306 or to otherwise provide structural support for the portable computing device 102. The height and / or angle of the mount 310 relative to the pole 306 may be adjustable to improve ergonomics for the operator.

[0064] The portable computing device 102 in the second configuration 300 may include similar components as the portable computing device 102 in the first configuration 200. For example, the portable computing device 102 may include a processor 202 including an image processor 204, a memory 206, a battery 208, a display 210, a medical device connector 212, an optional external video output 214, and an optional communication interface 216, each of which are described in detail with reference to FIG.

[0065] One or more connector cables 312 may connect the controller 104 to the portable computing device 102. For example, a first end of the connector cable 312 may be connected to the controller 104. In some examples, the first end of the connector cable 312 may be fixed to the controller 104. In other examples, the first end of the connector cable may be detachable from the controller 104. A second end of the connector cable 312 may be connected to and detached from the portable device 102, correspondingly connecting and disconnecting the portable device 102 from the controller 104.

[0066] In some aspects, a connector cable 312 may connect the controller 104 to the portable computing device 102 via the mount 310, e.g., to enable one-step docking and connection. For example, a second end of the connector cable may be received through an opening in the mount 310 and secured to and / or within the housing of the mount 310 using one or more securing mechanisms (e.g., screws, plates, and / or pogo pins) such that the second end of the connector cable 312 may interface with and connect to the portable computing device 102 when the portable computing device 102 is docked to the mount 310. As a result, when the portable computing device 102 is docked to the mount 310 by an operator, a connection may be simultaneously established between the portable computing device 102 and the controller 104 (e.g., to form a composite image processing system 106). Thereafter, e.g., when increased portability is desired, the portable computing device 102 may be removed or undocked from the mount 310 and thus detached from the second end of the connector cable 312 to disconnect the portable computing device 102 from the controller 104. The manner in which the second end of the connector cable 312 is secured to and / or within the housing of the mount 310 may facilitate quick disconnection when the portable computing device 102 is removed or undocked from the mount 310.

[0067] The connection or disconnection (e.g., connection state) of the portable computing device 102 to and from the controller 104 upon docking or undocking of the portable computing device 102 to and from the mount 310 may be detectable by the portable computing device 102. The detected connection state may cause the portable computing device 102 to operate in either a first mode of operation or a second mode of operation. In some examples, the mount 310 may include a mechanical feature (e.g., a switch), an electrical feature (e.g., an electrical contact), and / or an electronic detection feature that may facilitate detectability of the connection state by the portable computing device 102.

[0068] In other aspects, the connector cable 312 may connect the controller 104 directly to the portable computing device 102, independent of the mount 310. That is, docking and undocking of the portable computing device 102 to and from the mount 310 does not affect the connection state between the portable computing device 102 and the controller 104. Thus, even when undocked from the mount 310, the portable computing device 102 may still be connected to the controller 104 via the connector cable 312. As a result, an operator may be given some additional mobility (e.g., the length of the connector cable 312) for handling the portable computing device 102 when undocked, while still taking advantage of the benefits of connection to the controller 104, for example, to perform more advanced image processing. In such aspects, the connection or disconnection (e.g., connection state) of the portable computing device 102 to and from the controller 104 due to the connection or removal of the connector cable 312 to and from the portable computing device 102 may be detectable by the portable computing device 102. The detected connection state can cause the portable computing device 102 to operate in either a first mode of operation or a second mode of operation.

[0069] Optionally, rather than the second end of connector cable 312 connecting to the portable computing device through or independent of mount 310, the second end of connector cable 312 may connect to mount 310 itself. When portable computing device 110 is docked to mount 310, mount 310 may provide a connection between portable computing device 102 and controller 104. When portable computing device 102 is undocked, the connection may be interrupted.

[0070] The connector cable 312 may include similar features and / or functionality as the portable computing device connector 234 in the first configuration 200. For example, the connector cable 312 may include one or more optical fibers supporting high-resolution data transfer (e.g., high-resolution image data) between the portable computing device 102 and the controller 104. The connector cable 312 may also include one or more wires or cables for power transmission. For example, when the portable computing device 102 is docked to the controller 104 and the controller 104 is connected to an electrical source via the power connector 226, power may be supplied from the controller 104 to the portable computing device 102 via the power transmission cable to recharge the battery 208. Alternatively, the controller 104 may supply power from the battery 301 when the controller 104 is not connected to an electrical source via the power connector 226. In other examples, the portable computing device 102 may be charged by other means (e.g., via wireless charging). Additionally, the connector cable 312 may include one or more wires or cables for transmitting standard image or video data (also referred to as a standard video cable). A standard video cable may enable the use of standardized video display data for communication between portable computing device 102 and controller 104 when connected to form composite image processing system 106. Such use may also minimize the number of types of video output signals that need to be generated by controller 104 to output generated image data to portable computing device 102 for display (e.g., on display 210) and to other external display devices connected to controller 104 (e.g., from external device 112).

[0071] In some examples, the mount 310 may be configured to recharge the battery 208 of the portable computing device 102 independently of the controller 104. That is, the mount 310 may be configured to recharge the battery 208 when the portable computing device 102 is docked to the mount 310, even when the controller 104 is powered off and / or not connected to an electrical source. In one example, if the battery 301 is a standalone component separate from the controller 104, the mount 310 may be connected to the battery 301 and may receive a power supply from the battery 301 for use in recharging the battery 208 of the portable computing device 102. In another example, the mount 310 itself may include a battery (not shown) that provides a power supply for recharging the battery 208 of the portable computing device.

[0072] Before or during a medical procedure, a connector plug extending from the proximal end of the medical device 108 may be connected to the portable computing device 102 via one of the medical device connectors 212. The distal end of the medical device 108 may be inserted into a patient's body lumen and guided therethrough to a target site during a medical procedure. An imaging device 110 located at the distal end of the medical device 108 may be configured to capture image signals as the medical device 108 is of interest to and guided therethrough to a target site. The portable computing device 102 may receive image signals from the imaging device 110.

[0073] 2B , in the second configuration 300, the operations performed by the portable computing device 102 and / or the controller 104 to process the image signal may depend on the operational mode of the portable computing device 102. For example, when the portable computing device 102 is not docked to the mount 310 and / or otherwise connected to the controller 104 via the connector cable 312, the portable computing device 102 may operate in the first operational mode. As a result, in the first operational mode, the portable computing device 102 may be configured to process the image signal to generate and display image data, as described in detail above with reference to FIG. 2B .

[0074] Alternatively, when the portable computing device 102 is docked to the mount 310 and / or otherwise connected to the controller 104 via the connector cable 312, the portable computing device 102 may operate in a second mode of operation. As a result, in the second mode of operation, the portable computing device 102 may be configured to convert image signals to a standard protocol and transmit the standard protocol to the controller 104 for processing. The image processor 222 of the controller may then be configured to receive and process the standard protocol to generate and provide image data for display, as described in detail above with reference to FIG. 2B .

[0075] In the second mode of operation, the portable computing device 102 may also be configured to provide a user interface for the controller 104 (e.g., generate and display a second user interface) to enable user interaction via the display 210. Optionally, one or more peripheral devices 314, such as a keyboard 314A, a mouse 314B, or other similar input device, may be indirectly connected to the portable computing device 102 via the mount 310 (or optionally connected directly to the portable computing device 102 via a wired or wireless connection, such as Bluetooth or other similar connection, via the network 120) to further facilitate user interaction via the second user interface. In other examples, the peripheral devices 314 may be connected to the controller 104 instead of the portable computing device 102 and / or the mount 310. The peripheral devices 314 may be supported by a tray 316 or other similar support structure attached to the pole 306 of the mobile stand 302. The height and / or angle of the tray 316 relative to the pole 306 may be adjustable to improve operator ergonomics.

[0076] In other examples, the controller 104 may include any medical device connectors 230, and before or during a medical procedure, a connector plug extending from the proximal end of the medical device 108 may be connected to the controller 104 via one of the medical device connectors 230. In such examples, the image processor 222 may receive and process image signals from the imaging device 110 (e.g., directly from the medical device 108, without going through the portable computing device 102) to generate and provide image data for display, as described in detail above with reference to FIG.

[0077] 4 shows an example process 400 for processing image signals received from imaging device 110. In some examples, the steps of process 400 may be performed by portable computing device 102 and / or controller 104.

[0078] At step 402, a connection status between the portable computing device 102 and the controller 104 may be determined. As described in more detail with reference to FIG. 5, the determination may be a two-step process that evaluates the physical connection between the portable computing device 102 and the controller 104 and, if a physical connection exists, evaluates the communicative connection between the portable computing device 102 and the controller 104. The connection status may be determined to be a negative connection status when there is no physical connection and / or when there is a physical connection but there is no (or suboptimal) communicative connection. The connection status may be determined to be a positive connection status when there is both a physical connection and a sufficient communicative connection (e.g., a communicative connection that meets a communicative connection threshold). The connection status may be determined periodically and / or in response to detection of a trigger event, as described in more detail below with reference to FIG. 5.

[0079] In step 404, the portable computing device 102 may be operated in a first operational mode or a second operational mode based on the connection status determined in step 402. For example, if the connection status determined in step 402 is a negative connection status, the portable computing device 102 may be operated in the first operational mode. Alternatively, if the connection status determined in step 402 is a positive connection status, the portable computing device 102 may be operated in the second operational mode.

[0080] The first mode of operation may be a default mode of operation in which the portable computing device 102 operates independently of the controller 104, functions as an image processing system that performs image processing operations and displays image data on the display 210 (and / or optionally, other display devices connected to the portable computing device 102) via a first user interface. In the second mode of operation, the portable computing device 102 may operate in conjunction with the controller 104 to form a composite image processing system 106. For example, when in the second mode of operation, the portable computing device 102 may provide a user interface for the controller 104 (e.g., display a second user interface on the display 210) and function as an image signal translator to facilitate image processing operations performed by the controller 104, as described in more detail below with reference to FIG. 6 .

[0081] In step 406, an image signal may be received from the imaging device 110. As described in detail above with reference to Figure 1, the imaging device 110 may be an imaging device (e.g., an endoscopic camera) of the medical device 108. Furthermore, the imaging device 110 may be configured to capture multiple images of the body lumen as the medical device 108 is inserted into and guided through the body lumen to a target site during a diagnostic and / or interventional procedure.

[0082] At step 408, based on the first or second operating mode of the portable computing device 102, one or more of the portable computing device 102 or the controller 104 may process the image signal to generate image data. For example, as described in more detail with reference to FIG. 6, when the portable computing device 102 is operating in the first operating mode, the image processor 204 of the portable computing device 102 may perform image processing of the image signal to generate image data. Alternatively, when the portable computing device 102 is operating in the second operating mode, the portable computing device 102 may function as an image signal translator to first process (e.g., convert) the image signal to a standard protocol, and the image processor 222 of the controller may receive the standard protocol and perform image processing to generate image data. The image processor 222 may be configured to implement more advanced image processing techniques than the image processor 204, such as AI or machine learning-based image processing techniques.

[0083] Additionally, in some examples, one or more of the external devices 112 connected to the portable computing device 102 and / or the controller 104 may include one or more third-party processing systems, such as an AI processing system. The third-party processing systems may be configured to receive image data generated by the portable computing device 102 and / or the controller 104, respectively, as input for generating the augmented image data. The augmented image data may then be returned to the portable computing device 102 and / or the controller 104, respectively, for display.

[0084] At step 410, the image data may be output for display on the display 210 of the portable computing device 102 or an external display device (e.g., one of the external devices 112). The device on which the image data is displayed may be based on the operating mode of the portable computing device 102 and / or user-defined settings.

[0085] For example, when the portable computing device 102 is operating in a first mode of operation, image data generated by the portable computing device 102 may be displayed on the display 210 of the portable computing device 102 and / or on one or more display devices from an external device 112 connected to the portable computing device 102 via the external video output 214. In one example, a user-defined setting may direct that the image data be displayed on the display 210 unless an external display device is detected as connected to the portable computing device 102. If an external display device is detected as connected to the portable computing device 102, the image data is directed to the external display device for display (e.g., so long as the characteristics and / or features of the external display device meet minimum display requirements). In another example, a user-defined setting may direct that the image data be displayed on the display 210 and that the display of the image data be replicated on any external display devices detected as connected to the portable computing device 102. If more than one external display device is detected as connected to the portable computing device 102, one or more hierarchical rules may optionally be defined in the user settings to direct which one or more of the external display devices the image data is provided to for display. In some examples, the rules may be based on the characteristics or features of the external display device to provide an optimal display of the image data. Additionally or alternatively, the rules may be based on the type of data being displayed and / or the number of different types of data being displayed (e.g., still images and / or moving images).

[0086] Alternatively, when the portable computing device 102 is operating in the second mode of operation and depending on the user-defined settings, the image data may be displayed on one or more display devices from the external device 112 connected to the controller 104 via the external video output 228, the display 210 of the portable computing device 102, and / or one or more display devices from the external device 112 connected to the portable computing device 102 via the external video output 214. In one example, the user-defined settings may direct the image data to be displayed on the external display device connected to the controller 104 unless such a connection is detected, at which point the image data is displayed on the display 210 of the portable computing device 102 and / or the external display device, if any, connected to the portable computing device 102. As another example, the user-defined settings may direct the display of the image data to be replicated on the external display device connected to the controller 104 and the display 210 of the portable computing device 102 and / or the external display device, if any, connected to the portable computing device 102. As previously mentioned, one or more hierarchical rules may optionally be defined in the user settings to determine which one or more of the available display devices to provide the image data to for display.

[0087] Accordingly, particular embodiments may be implemented to process image signals received from imaging device 110. The process 400 described above is provided by way of example only and may include additional, fewer, different, or differently arranged steps than those shown in FIG.

[0088] 5 shows an example process 500 for determining a connection status between the portable computing device 102 and the controller 104. In some examples, the process 500 may be performed by the portable computing device 102. The process 500 may be used to implement at least a portion of step 402 of the process 400 described above with reference to FIG.

[0089] In step 502, a physical connection status between the portable computing device 102 and the controller 104 may be determined. In some examples, the manner in which the physical connection status is determined may depend on the configuration of the portable computing device 102 and the controller 104 (e.g., the first configuration 200 or the second configuration 300).

[0090] When in the first configuration 200, as described with reference to FIGS. 2A and 2B, a positive physical connection state may be determined in response to the connector receptacle 218 of the portable computing device 102 receiving the portable computing device connector 234 of the controller 104. Otherwise, a negative physical connection state may be determined. When in the second configuration 300, as described with reference to FIGS. 3A and 3B, a positive physical connection state may be determined when the second end of the connector cable 312 is connected to the portable computing device 102 (the first end of the connector cable 312 is connected to the controller 104) via and / or independently of the mount 310. Alternatively, when in the second configuration 300, the second end of the connector cable 312 is instead connected to the mount 310, a positive physical connection state may be determined in response to the portable computing device 102 being docked to the mount 310. Otherwise, a negative physical connection state may be determined.

[0091] At step 504, a determination may be made whether there is a physical connection between the portable computing device 102 and the controller 104 based on the determined physical connection status. If the physical connection status is a negative physical connection status, there may not be a physical connection, and process 500 may proceed to step 506. At step 506, the portable computing device 102 may be operated in a first operating mode. Alternatively, if the physical connection status is a positive physical connection status, there may be a physical connection, and process 500 may proceed to step 508.

[0092] For other aspects, if the connection between the portable computing device 102 and the controller 104 is wireless (e.g., as opposed to wired via the connector receptacle 218 and the portable computing device connector 234 and / or via the connector cable 312), then the process 500 may include alternative steps for steps 502 and 504. For example, instead of a physical connection status, a proximity status between the portable computing device 102 and the controller 104 may be determined. If the portable computing device 102 and the controller 104 are within a threshold range of proximity that allows communication, then the process may proceed to step 508. Alternatively, in a wireless context, the process 500 may begin with step 508.

[0093] At step 508, the communicative connection status between the portable computing device 102 and the controller 104 may be determined. To determine the communicative connection status, a negotiation process (e.g., a handshake) between the portable computing device 102 and the controller 104 may be initiated by the portable computing device 102. As part of the negotiation process, the portable computing device 102 and the controller 104 may exchange signals with each other to establish a communications link or channel. The signals may be exchanged via the portable computing device connector 234 when the portable computing device 102 and the controller 104 are connectable in the first configuration 200, or via the connector cable 312 when the portable computing device 102 and the controller 104 are connectable in the second configuration 300. The exchanged signals may indicate to each other that the portable computing device 102 and the controller 104 are each powered on and operational. The portable computing device 102 and the controller 104 may also agree on a standard protocol (e.g., to which the portable computing device 102 converts image signals) and / or other related parameters for data transfer over the communications link through signals exchanged during the negotiation process. Exemplary parameters may include information transfer rates, coding alphabets, parity, interrupt procedures, and / or other protocol or hardware capabilities. In some examples, at least some of the parameters should meet agreed-upon and / or predefined minimum criteria in order to satisfy a communicative connection threshold between the portable computing device 102 and the controller 104. Meeting the communicative connection threshold may help ensure that the image signals (e.g., converted to a standard protocol by the portable computing device 102) can be effectively transmitted to the controller 104 for processing by the image processor 222.For example, to meet a communicative connection threshold, certain signals (e.g., synchronization signals) exchanged between the portable computing device 102 and the controller 104 may be detected at expected times, meet minimum signal levels, etc.

[0094] In some examples, if a problem or error occurs during the negotiation process, the negotiation process may be restarted or retried. In other examples, the negotiation process may include a predefined timeout period. If a communicative connection cannot be established within the predefined timeout period, the negotiation process may fail and the communicative connection threshold may not be met. Even if a communication link or channel can be established, the communicative connection threshold may not be met if communication over the link or channel is suboptimal for transmitting data. In some examples, suboptimal communication may be due to hardware issues with the controller 104.

[0095] At step 510, a determination may be made whether a communicative connection threshold between the portable computing device 102 and the controller 104 is met based on the communicative connection state determined at step 508 (e.g., based on a negotiation process). If the communicative connection threshold is not met, process 500 may proceed to step 512. At step 512, the portable computing device 102 may be operated in a first mode of operation. If the communicative connection threshold is met, process 500 may proceed to step 514. At step 514, the portable computing device 102 may be operated in a second mode of operation.

[0096] In some examples, process 500 may be repeated periodically, such as at predefined intervals. As one non-limiting example, process 500 may be repeated approximately every 0.25 seconds. In other examples, process 500 may be repeated in response to portable computing device 102 detecting a trigger event. One exemplary trigger event may include receiving an image signal from imaging device 110.

[0097] Depending on the operational mode of the portable computing device 102 when the image signal is received from the imaging device 110, different operations may be performed by the portable computing device 102. The image processing operations performed by the portable computing device 102 when operating in a first operational mode and the conversion operations performed by the portable computing device 102 when operating in a second operational mode are described in detail below with reference to FIG.

[0098] Accordingly, particular embodiments may be implemented for determining the connection status between portable computing device 102 and controller 104. The process 500 described above is provided merely as an example and may include additional, fewer, different, or differently arranged steps relative to those shown in FIG.

[0099] 6 shows an example process 600 for operational mode dependent image processing. In some examples, process 600 may be performed by portable computing device 102. Process 600 may be used to implement at least a portion of steps 406, 408, and / or 410 of process 400 described with reference to FIG.

[0100] In step 602, the portable computing device 102 may receive an image signal from the imaging device 110. In step 604, the process 600 may include determining whether the portable computing device 102 is operating in a first mode of operation or a second mode of operation, for example, as described above with respect to process 500 described with reference to Figure 5. Based on whether the portable computing device 102 is operating in the first mode of operation or the second mode of operation, the process 600 may either proceed to steps 606-608 when operating in the first mode of operation, or proceed to steps 610-612 and optionally step 614 when operating in the second mode of operation.

[0101] When the portable computing device 102 is operating in the first mode of operation, the image signal may be processed by the image processor 204 of the portable computing device 102 to generate image data, at step 606. The image data may then be provided for display, at 608, on the display 210 of the portable computing device 102 and / or on an external display (e.g., one of the external devices 112) connected to the portable computing device 102 via the external video output 214. In some examples, user settings may define which device (or devices) displays the image data.

[0102] When the portable computing device 102 is operating in the second mode of operation, the portable computing device 102 may convert the image signal to a standard protocol in step 610. The standard protocol may then be transmitted to the controller 104 to be processed into image data in step 612. For example, the image processor 222 of the controller 104 may process the standard protocol to generate the image data. In some examples, the image processor 222 may implement advanced image processing techniques, including AI and / or machine learning-based image processing techniques, to generate the image data.

[0103] Once the image data is generated, the controller 104 may provide the image data to one or more external devices 112 connected to the controller 104 via the external video output 228. In some examples, the external device 112 may include an external display device (e.g., a monitor) configured to display the image data. In other examples, the external device 112 may include a third-party processing system, such as an AI processing system, configured to receive the image data as input, e.g., to generate the augmented image data. The controller may receive the augmented image data from the third-party processing system and provide the augmented image data for display.

[0104] Additionally or alternatively, controller 104 may optionally provide the image data (or augmented image data, if applicable) to portable computing device 102 for display. In such an example, in optional step 614, portable computing device 102 may receive the image data from controller 104 for display on display 210 of portable computing device 102 and / or on an external display (e.g., one of external devices 112) connected to portable computing device 102 via external video output 214.

[0105] Accordingly, particular embodiments may be implemented for operational mode dependent image processing. The process 600 described above is provided merely as an example and may include additional, fewer, different, or differently arranged steps relative to those shown in FIG.

[0106] FIG. 7 illustrates an example of a computer 700. FIG. 7 is a simplified functional block diagram of a computer 700 that may be configured as a device for performing the processes, steps, or operations shown in or described with respect to FIGS. 1-6 and according to exemplary embodiments of the present disclosure. For example, the computer 700 may be configured as one of the portable computing device 102, the controller 104, the external device 112, the server-side system 130, and / or another device or component according to exemplary embodiments of the present disclosure. In various embodiments, any of the systems herein may be or include a computer 700 that includes, for example, a data communication interface 720 for packet data communication. The computer 700 may communicate with one or more other computers using, for example, an electronic network 725 (e.g., via the data communication interface 720). The electronic network 725 may include a wired or wireless network similar to the network 120 shown in FIG. 1.

[0107] The computer 700 may also include a central processing unit ("CPU") in the form of one or more processors 702 for executing program instructions 724. The program instructions 724 may include instructions for executing one or more applications or programs related to determining a connection state, operating the portable computing device 102 in a first or second mode of operation, and performing corresponding basic image processing and / or conversion operations based on the mode of operation (e.g., if the computer 700 is the portable computing device 102). The program instructions 724 may include instructions for performing image processing operations, including conducting image processing operations such as AI or machine learning-based image processing (e.g., if the computer 700 is one of the controllers 104 or server-side systems).

[0108] The computer 700 may include an internal communication bus 708. While the computer 700 may receive programming and data via network communications, the computer 700 may also include a drive unit 706 (e.g., a read-only memory (ROM), a hard disk drive (HDD), a solid-state disk drive (SDD), etc.) that may store data on a computer-readable medium 722 (e.g., a non-transitory computer-readable medium). The computer 700 may also have a memory 704 (e.g., a random-access memory (RAM)) that stores instructions 724 for performing the techniques presented herein. However, it should be noted that in some aspects, the instructions 724 may be stored temporarily or permanently within other modules of the computer 700 (e.g., the processor 702 and / or the computer-readable medium 722). The computer 700 may also include a user input and output device 712 and / or a display 710 for connecting to input and / or output devices such as a keyboard, mouse, touchscreen, monitor, display, etc. Various system functions may be implemented in a distributed manner across multiple similar platforms to distribute the processing load, or the system may be implemented by appropriate programming of one computer hardware platform.

[0109] Program aspects of the present technology may be considered to be "products" or "articles of manufacture," typically in the form of executable code and / or associated data carried on or embodied in some type of machine-readable medium. "Storage"-type media includes any or all of the tangible memory of a computer, processor, or the like, or their associated modules, which may provide non-transitory storage for software programming at any time, such as various semiconductor memories, tape drives, disk drives, etc. All or some portions of the software may sometimes be communicated via the Internet or various other telecommunications networks. Such communication may, for example, enable loading of software from one computer or processor to another. Thus, other types of media that may carry software elements include optical, electrical, and electromagnetic waves, such as those used across physical interfaces between local devices, through wired and optical landline networks, and over various airlinks. Physical elements that carry such waves, such as wired or wireless links, optical links, etc., may also be considered software-bearing media. As used herein, unless limited to non-transitory tangible "storage" media, terms such as computer or machine "readable medium" refer to any medium that participates in providing instructions to a processor for execution.

[0110] While the principles of the present disclosure have been described herein with reference to illustrative embodiments for particular applications, it should be understood that the disclosure is not limited thereto. Those skilled in the art and with access to the teachings provided herein will recognize that additional modifications, adaptations, and equivalent substitutions are all within the scope of the embodiments described herein. Accordingly, the present invention should not be deemed limited by the foregoing description.

Claims

1. 1. A portable computing device of a medical imaging system connectable to a controller, comprising: a medical device connector for connecting a medical device to the portable computing device, the medical device including an imaging device; and at least one memory for storing instructions; one or more processors including an image processor; Equipped with Execution of the instructions by the one or more processors causes the portable computing device to perform operations, the operations including: determining a connection status between the portable computing device and the controller; operating the portable computing device in a first mode of operation or a second mode of operation based on the connection status; receiving an image signal from the imaging device; processing the image signal by the image processor to generate image data when the portable computing device is operating in the first mode of operation; providing the image signal to the controller for processing to generate the image data when the portable computing device is operating in the second mode of operation. Including, Portable computing devices.

2. To determine the connection state, the operation comprises: determining a physical connection between the portable computing device and the controller; determining a communicative connection status between the portable computing device and the controller in response to determining that a physical connection exists between the portable computing device and the controller based on the physical connection status; 10. The portable computing device of claim 1, comprising:

3. 3. The portable computing device of claim 2, wherein if the physical connection status indicates that there is no physical connection between the portable computing device and the controller, the portable computing device is operated in the first operating mode.

4. 3. The portable computing device of claim 2, wherein to determine the communicative connection state, the actions include initiating a negotiation process between the portable computing device and the controller to determine whether a communicative connection threshold is met.

5. In response to determining that the communicative connection threshold has not been met, the portable computing device is operated in the first operating mode; In response to determining that the communicative connection threshold is met, the portable computing device is operated in the second operating mode.

5. The portable computing device of claim 4.

6. The portable computing device of any one of claims 1 to 5, wherein the connection state between the portable computing device and the controller is determined periodically at predefined intervals.

7. 6. The portable computing device of claim 1, wherein the connection state between the portable computing device and the controller is determined in response to detecting a trigger event, and at least one trigger event includes receiving the image signal from the imaging device.

8. When the portable computing device is operating in the first mode of operation, the operation comprises: displaying the generated image data on a display of the portable computing device; or providing the generated image data for display on an external display device connected to the portable computing device. The portable computing device of any one of claims 1 to 7, further comprising one or more of:

9. When the portable computing device is operating in the first operational mode, the portable computing device provides a first user interface on a display of the portable computing device, the first user interface including a subset of imaging-related function controls corresponding to a subset of operations performable by the portable computing device; When the portable computing device is operating in the second mode of operation, the portable computing device provides a second user interface on the display of the portable computing device, the second user interface including a plurality of imaging-related function controls corresponding to a plurality of operations performable by the controller. A portable computing device according to any one of claims 1 to 8.

10. When the portable computing device is operating in the second mode of operation and the image signal is provided to the controller for processing to generate the image data, the operation comprises: converting said image signal into a standard protocol; sending the standard protocol to the controller for processing; The portable computing device of any one of claims 1 to 9, further comprising:

11. A portable computing device according to any preceding claim, wherein the image processor of the controller is configured to perform more advanced image processing than the image processor of the portable computing device.

12. When the portable computing device is operating in the second mode of operation, the operation comprises: receiving the generated image data from the controller for display on one or more of a display of the portable computing device or an external display device connected to the portable computing device; The portable computing device of any one of claims 1 to 11, further comprising:

13. The portable computing device of any one of claims 1 to 12, further comprising a connector receptacle configured to receive a connector of the controller to connect the portable computing device to the controller.

14. The portable computing device of any preceding claim, wherein the portable computing device is configured to receive one or more connector cables that connect the portable computing device to the controller.

15. The portable computing device of claim 14 , wherein the one or more connector cables connect the portable computing device to the controller via a mount configured to receive the portable computing device.

Images