Systems and methods for 3D vascular ultrasound

Abstract

Methods, systems, and apparatuses for capturing two-dimensional (2D) ultrasound images of a location on an individual and motion and orientation data as an imaging device traverses the location on the individual and generating a three-dimensional (3D) reconstruction of the 2D images based on the motion and orientation data are described. The imaging device may be synchronized with one or more sensor devices affixed to the image device or the individual by aligning the 2D images captured by the imaging device with the motion and orientation data generated by the one or more sensor devices. Based on synchronizing the imaging device with the one or more sensor devices, the images may be combined with the motion and orientation data in order to generate the 3D reconstruction of the 2D images.

Description

ATTORNEY DOCKET NO.: 37759.0660P1SYSTEMS AND METHODS FOR 3D VASCULAR ULTRASOUNDCROSS REFERENCE TO RELATED PATENT APPLICATION

[0001] This Application claims priority to U.S. Provisional Application No. 63 / 686,022, filed August 22, 2024, which is herein incorporated by reference in its entirety.BACKGROUND

[0002] Chronic Kidney Disease (CKD) progressing to End Stage Renal Disease (ESRD) is a disabling condition resulting in patients requiring hemodialysis thrice weekly to stay alive. It has been shown that integrating vascular access care can reduce patient costs. Maintaining dialysis vascular access is essential for functional recovery for patients with end-stage renal disease (ESRD), where independence on hemodialysis requires reliable vascular access. When a surgically created access, usually an arteriovenous (AV) fistula, fails to mature (e.g., has insufficient blood flow or vessel wall strength for dialysis) or fails (e.g., thromboses, bleeds excessively, or cannot tolerate dialysis flow rates), dialysis treatments cannot be delivered. This leads to the use of dialysis catheters which are prone to infection, clotting, central vein stenosis, and inferior dialysis clearance for vascular access.

[0003] Thus, ultrasound imaging is widely used to assist in vascular access procedures, such as placing peripheral intravenous lines, central venous catheters, and arterial lines. By providing real-time, dynamic visualization of blood vessels and surrounding structures, ultrasound imaging allows clinicians to identify vessel location, size, and depth, as well as to assess patency and avoid nearby nerves or arteries. This improves first-attempt success rates, reduces the risk of complications (e g., inadvertent arterial puncture, hematoma, or pneumothorax), and is especially useful in patients with difficult anatomy — such as children, obese individuals, or those with collapsed or thrombosed veins. Ultrasound guidance can also minimize the number of needle passes and procedural time in complex cases.

[0004] However, ultrasound-guided vascular access has several limitations. While three-dimensional (3D) ultrasound scanners are available for obstetrical and other specialized uses, it is cost prohibitive in certain settings, especially in the dialysis settingATTORNEY DOCKET NO.: 37759.0660P1 for point of care use. 3D ultrasound scanners are typically expensive and may not always be readily available, particular in emergency or resource-limited settings. In addition.3D ultrasound scanners have a limited field of view that can be too small to cover different regions of interest, especially for dialysis fistula applications. Also, image quality can be operator-dependent and can vary with different patient body -types.

[0005] However, lower cost two-dimensional (2D) ultrasound scanners are increasingly available in the point of care setting. These 2D ultrasound scanners can generate 3D information to assist caregivers in detecting stenosis or other abnormalities, as well as provide a guide for cannulation in the dialysis setting. In addition, these 2D ultrasound scanners may provide a roadmap for interventional procedures. However, these 2D ultrasound scanners suffer from inadequate integration of motion and orientation measurements with the 2D ultrasound scan imaging stick in order to generate precise 3D images.SUMMARY

[0006] It is to be understood that both the following general description and the following detailed description are exemplary and explanatory only and are not restrictive.

[0007] Methods, apparatuses, and systems are described for capturing two-dimensional (2D) ultrasound images of a location on an individual, in addition to motion and orientation data, as an imaging device traverses the location on the individual and generating a three-dimensional (3D) reconstruction of the 2D ultrasound images based on the motion and orientation data. The imaging device may be synchronized with one or more sensor devices affixed to the image device or the individual by aligning the 2D images captured by the imaging device with the motion and orientation data generated by the one or more sensor devices. Based on synchronizing the imaging device with the one or more sensor devices, the images may be combined with the motion and orientation data in order to generate the 3D reconstruction of the 2D ultrasound images.

[0008] In an embodiment, disclosed are methods comprising based on initial movement data associated with an imaging device, determining, by a computing device, subsequent movement data received from one or more sensor devices and image sequence data received from the imaging device as the imaging device traverses a location on an individual, causing, based on the subsequent movement data and the image sequenceATTORNEY DOCKET NO.: 37759.0660P1 data, the imaging device to synchronize with the one or more sensor devices, combining, based on the synchronization of the imaging device with the one or more sensor devices, the image sequence data with the subsequent movement data, and generating, based on the image sequence data combined with the subsequent movement data, a three- dimensional representation of an interior portion of the individual.

[0009] In an embodiment, disclosed are methods comprising receiving, by a computing device, movement data from one or more sensor devices and image sequence data from an imaging device as the imaging device traverses a location on an individual, combining, based on synchronizing the imaging device with the one or more sensor devices, the image sequence data with the movement data, generating, based on the image sequence data combined with the movement data, a vessel distribution map, identifying, in the image sequence data, based on the vessel distribution map, one or more blood vessel locations within an interior portion of the individual, and generating, based on the identification of the one or more blood vessel locations and the image sequence data combined with the movement data, a three-dimensional representation of the interior portion of the individual.

[0010] In an embodiment, disclosed are methods comprising receiving, by a computing device, movement data from one or more sensor devices and image sequence data from an imaging device as the imaging device traverses a location on an individual, generating, based on combining the image sequence data with the movement data, a three-dimensional representation of an interior portion of the individual, generating, based on the three-dimensional representation of the interior portion of the individual, one or more measurements associated with the interior portion of the individual, detecting, based on the one or more measurements associated with the interior portion of the individual, one or more medical conditions associated with the individual, and causing, based on the one or more medical conditions, one or more treatments of the individual.

[0011] In an embodiment, disclosed are methods comprising receiving, by a computing device, movement data from one or more sensor devices and image sequence data from an imaging device as the imaging device traverses a location on an individual, generating, based on combining the image sequence data with the movement data, a three-dimensional representation of an interior portion of the individual, generating a visual interface comprising the three-dimensional representation of the interior portion ofATTORNEY DOCKET NO.: 37759.0660P1 the individual overlaid onto the image sequence data, and outputting the visual interface comprising the three-dimensional representation overlaid onto the image sequence data.

[0012] Additional advantages will be set forth in part in the description which follows or may be learned by practice. The advantages will be realized and attained by means of the elements and combinations particularly pointed out in the appended claims.BRIEF DESCRIPTION OF THE DRAWINGS

[0013] The accompanying drawings, which are incorporated in and constitute a part of the present description serve to explain the principles of the methods and systems described herein:Figure 1 shows an example system for generating 3D ultrasound images;Figure 2 shows an example process for generating 3D ultrasound images;Figures 3A-3E shows an example smartcable and smartcable components;Figure 4 shows an example gesture movement environment;Figure 5 show an example imaging and sensor device trajectory;Figure 6 shows an example L-shape feature wall;Figures 7A-7B shows an example vessel edge visualization;Figures 8A-8E show image sequence data projections and 3D representations;Figures 9A-9B shows an example visual interface process;Figure 10 shows a flowchart of an example method: Figure 11 shows a flowchart of an example method: Figure 12 shows a flowchart of an example method; and Figure 13 shows a flowchart of an example method.DETAILED DESCRIPTION

[0014] As used in the specification and the appended claims, the singular forms “a,” “an,’" and “the” include plural referents unless the context clearly dictates otherwise. Ranges may be expressed herein as from “about” one particular value, and / or to “about” another particular value. When such a range is expressed, another configuration includes from the one particular value and / or to the other particular value. Similarly, when values are expressed as approximations, by use of the antecedent “about,” it will be understood that the particular value forms another configuration. It will be further understood that the endpoints of each of the ranges are significant both in relation to the other endpoint, and independently of the other endpoint.ATTORNEY DOCKET NO.: 37759.0660P1

[0015] “Optional” or “optionally” means that the subsequently described event or circumstance may or may not occur, and that the description includes cases where said event or circumstance occurs and cases where it does not.

[0016] As used herein the terms “individual,” “patient,” “subject,” “user,” or “person” may indicate a person associated with the determination of one or more physiological parameters, such as an interstitial insulin elimination rate, a hepatic insulin elimination rate, a renal insulin elimination rate, or a permeability constant related to insulin diffusion in the human body.

[0017] Throughout the description and claims of this specification, the word “comprise” and variations of the word, such as “comprising” and “comprises,” means “including but not limited to,” and is not intended to exclude, for example, other components, integers or steps. “Exemplary” means “an example of and is not intended to convey an indication of a preferred or ideal configuration. “Such as” is not used in a restrictive sense, but for explanatory purposes.

[0018] It is understood that when combinations, subsets, interactions, groups, etc. of components are described that, while specific reference of each various individual and collective combinations and permutations of these may not be explicitly described, each is specifically contemplated and described herein. This applies to all parts of this application including, but not limited to, steps in described methods. Thus, if there are a variety of additional steps that may be performed it is understood that each of these additional steps may be performed with any specific configuration or combination of configurations of the described methods.

[0019] As will be appreciated by one skilled in the art, hardware, software, or a combination of software and hardware may be implemented. Furthermore, the methods and systems may take the form of a computer program product on a computer-readable storage medium (e.g., non-transitory) having processor-executable instructions (e.g., computer software) embodied in the storage medium. More particularly, the present methods and systems may take the form of web-implemented computer software. Any suitable computer-readable storage medium may be utilized including hard disks, CD- ROMs, optical storage devices, magnetic storage devices, memrsistors, Non-Volatile Random Access Memory (NVRAM), flash memory, or a combination thereof.

[0020] Throughout this application reference is made to block diagrams and flowcharts. It will be understood that each block of the block diagrams and flowcharts, andATTORNEY DOCKET NO.: 37759.0660P1 combinations of blocks in the block diagrams and flowcharts, respectively, may be implemented by processor-executable instructions. These processor-executable instructions may be loaded onto a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the processor-executable instructions which execute on the computer or other programmable data processing apparatus create a device for implementing the functions specified in the flowchart block or blocks. In addition, some of these functions may be carried out using complex programmable logic devices (CPLDs) or other programmable logic devices.

[0021] The processor-executable instructions may also be stored in a computer-readable memory that may direct a computer or other programmable data processing apparatus to function in a particular manner, such that the processor-executable instructions stored in the computer-readable memory' produce an article of manufacture including processorexecutable instructions for implementing the function specified in the flowchart block or blocks. The processor-executable instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer- implemented process such that the processor-executable instructions that execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks. In addition, some of these functions may be carried out using logic devices which do not operate by sequential operations of programmed steps.

[0022] Blocks of the block diagrams and flowcharts support combinations of devices for performing the specified functions, combinations of steps for performing the specified functions and program instruction means for performing the specified functions. It will also be understood that each block of the block diagrams and flowcharts, and combinations of blocks in the block diagrams and flowcharts, may be implemented by special purpose hardware-based computer systems that perform the specified functions or steps, or combinations of special purpose hardware and computer instructions and logic circuitry.

[0023] FIG. 1 shows an example system 100 for generating three-dimensional (3D) ultrasound images based on combining two-dimensional (2D) ultrasound (e.g., B-mode) images captured by an imaging device with movement data (e.g., motion and positional data) measured by one or more sensor devices affixed to the imaging device and / or oneATTORNEY DOCKET NO.: 37759.0660P1 or more locations on an individual (e.g., patient). The system 100 may comprise a computing device 101, an imaging device 102, one or more sensor devices 103, an electronic device 104, and / or a server 106. The computing device 101 may comprise a comprise hardware and software configured to process ultrasound images. For example, the computing device 101 may comprise a computer, a mobile phone, a smart phone, a tablet computer, a laptop, a smartwatch, and the like. As an example, the computing device 101 may comprise a computing device configured to control an imaging device for capturing 2D ultrasound images of an individual, wherein the computing device 101 may generate 3D ultrasound images from the 2D ultrasound and movement data generated by the one or more sensor devices 103. In an example, the computing device 101 may be integrated with a medical treatment device, wherein the computing device 101 and the medical treatment device may be configured to administer one or more treatments to an individual based on determining one or more medical conditions from the 3D ultrasound images. The computing device 101 may include a bus 110, one or more processors 120, a memory 140, an input / output interface 160, a display 170, and a communication interface 180. In an example, the computing device 101 may omit at least one of the aforementioned constitutional elements or may additionally include other constitutional elements.

[0024] The bus 110 may include a circuit for connecting the bus 110, the one or more processors 120, the memory 140, the input / output interface 160, the display 170, and / or the communication interface 180 to each other and for delivering communication (e.g., a control message and / or data) between the bus 110, the one or more processors 120, the memory 140, the input / output interface 160, the display 170, and / or the communication interface 180.

[0025] The one or more processors 120 may include one or more of a Central Processing Unit (CPU), an Application Processor (AP), and a Communication Processor (CP). The one or more processors 120 may control, for example, at least one of the bus 110, the memory 140, the input / output interface 160, the display 170, and / or the communication interface 180 and / or may execute an arithmetic operation or data processing for communication. The processing (or controlling) operation of the one or more processors 120 according to various embodiments is described in detail with reference to the following drawings.ATTORNEY DOCKET NO.: 37759.0660P1

[0026] The processor-executable instructions executed by the one or more processor 120 may be stored and / or maintained by the memory 140. The memory 140 may include a volatile and / or non-volatile memory. The memory 140 may store, for example, a command or data related to at least one different constitutional element of the electronic device 101. According to various exemplary' embodiments, the memory 140 may store a software and / or a program 150. The program 150 may include, for example, a kernel 151, a middleware 153. an Application Programming Interface (API) 155, and / or an application program (or an '‘application”) 157, or the like, configured for controlling one or more functions of the device 101 and / or an external device. At least one part of the kernel 151, middleware 153, or API 155 may be referred to as an Operating System (OS). The memory 140 may include a computer-readable recording medium having a program recorded therein to perform the method according to various embodiments by the processor 120.

[0027] The kernel 151 may control or manage, for example, system resources (e.g.. the bus 110, the processor 120, the memory 140, etc.) used to execute an operation or function implemented in other programs (e.g., the middleware 153, the API 155, or the application program 157). Further, the kernel 151 may provide an interface capable of controlling or managing the system resources by accessing individual constitutional elements of the device 101 in the middleware 153, the API 155. or the application program 157.

[0028] The middleware 153 may perform, for example, a mediation role so that the API 155 or the application program 157 can communicate with the kernel 151 to exchange data. Further, the middleware 153 may handle one or more task requests received from the application program 157 according to a priority7. For example, the middleware 153 may assign a priority of using the system resources (e.g.. the bus 110. the processor 120, or the memory 140) of the device 101 to at least one of the application programs 157. For instance, the middleware 153 may process the one or more task requests according to the priority' assigned to the at least one of the application programs, and thus may perform scheduling or load balancing on the one or more task requests.

[0029] The API 155 may include at least one interface or function (e.g., instruction), for example, for file control, window control, video processing, or character control, as an interface capable of controlling a function provided by the application 157 in the kernel 151 or the middleware 153.ATTORNEY DOCKET NO.: 37759.0660P1

[0030] The application program 157 may include logic (e.g., hardware, software, firmware, etc.) that may be implemented by the computing device 101 to generate three- dimensional (3D) representations (e.g., images, models, etc.) of an interior portion of an individual (e.g., patient) based on image sequence data captured and output by an imaging device 102 and movement data (e.g., motion and positional data) determined (e.g., measured) and output by one or more sensor devices 103 as the imaging device traverse a location on an individual. The imaging device 102 may comprise an ultrasound scanner. For example, the imaging device 102 may comprise a probe comprising an ultrasound scanner. For example, the imaging device 102 may generate the image sequence data as a user of the imaging device 102 performs one or more sweeping motions back and forth across the location on the individual. The interior portion of the individual may comprise one or more of one or more blood vessels, one or more organs, or one or more layers of tissue. The image sequence data may comprise a series of one or more two-dimensional (2D) B-mode images of the interior portion of the individual. For example, the imaging device 102 may comprise a 2D ultrasound scanner configured to output 2D B-mode (e.g.. ultrasound) images. The one or more sensor devices 103 may comprise one or more of an internal measurement unit (IMU), an electromagnetic sensor, an optical sensor, an airborne ultrasound sensor, a time of flight sensor, a light detection and ranging (LiDAR) sensor, a sonomicrometry sensor, a smartcable, a depth camera, a mechanical tracking system, or a tactile sensor. As an example, the one or more sensor devices 103 may be affixed to one or more of the imaging device 102 or the location on the individual. The movement data may comprise positional data and motion data, wherein the positional data may comprise spatial data and orientation data. For example, an IMU may be affixed to the imaging device 102. wherein the IMU may output movement data as the imagine device 102 traverse the location on the individual. For example, the IMU may output the movement data as the user of the imaging device 102 performs the one or more sweeping motions back and forth across the location on the individual.

[0031] As an example, an ultrasound scanning process may be initiated and deactivated based on an initial movement of the imaging device 102 and an ending movement of the imaging device 102, respectively. For example, the scanning process may be initiated based on an initial gesture movement signaling the computing device 101 the initiation of the scanning process. In addition, the scanning process may be deactivated based onATTORNEY DOCKET NO.: 37759.0660P1 an ending gesture movement signaling the computing device 101 to end the scanning process. For example, one or more of the sensor devices 103 may be affixed to the imaging device 102. The gesture movement may be captured by the sensor device(s) 103 affixed to the imaging device 102, wherein the computing device 101 may identify the gesture movement as a movement for initiating the scanning process. In an example, ultrasound speckle tracking may be utilized to identify the bulk, in-plane motion created by the gesture movement. Based on the initial gesture movement, the application program 157 may cause the computing device 101 to synchronize the imaging device 101 with the one or more sensor devices 103. For example, the imaging device 102 may be synchronized with the one or more sensor devices 103 by aligning the image sequence data output by the imaging device 102 with the movement data output by the one or more sensor devices 103. Once the image sequence data is aligned with the movement data, the application program 157 may cause the computing device 101 to combine the image sequence data with the movement data. For example, the image sequence data and the movement data may be combined into image digital imaging and communications in medicine (DICOM) files, wherein the movement data may be stored in a private field of the image DICOM file. As an example, this may enable a standard data interface. In addition, this may enable storage and communication with systems such as picture archiving and communication systems (PACS).

[0032] The application program 157 may then cause the computing device 101 to process the image sequence data. For example, the application program 157 may cause the computing device 101 to perform image segmentation and decorrelation processing of the image sequence data in order to identify different properties of the interior portion of the individual. For example, one or more vessel (e g., blood vessel) locations of the interior portion of the individual may be identified based on the image segmentation and decorrelation processing. As an example, the application program 157 may cause the computing device 101 to generate a vessel distribution map (VDM) based on the image sequence data combined with the movement data. For example, the VDM may be used to identify a high probability of the one or more vessel locations within the image sequence data. In an example, the application program 157 may cause the computing device 101 to further process the VDM to generate a final VDM. For example, a series of filters (e.g.. morphological filtering, media filters, etc.) may be applied to the VDM, selecting subsets of VDM data that have certain shape attributes (e g., a minimumATTORNEY DOCKET NO.: 37759.0660P1 number of pixels along the path of vessel tracking and a maximum shift in location perpendicular to the tracking path). The application program 157 may cause the computing device 101 to identify the one or more vessel locations within the interior portion of the individual in the image sequence data based on the VDM.

[0033] The application program 157 may then cause the computing device 101 to generate a 3D representation of an interior portion of the individual. For example, the interior portion may comprise an interior portion of the location on the individual. For example, the imaging device 102 may be used to scan the forearm portion of the individual. Based on receiving image sequence data and movement data associated with scanning the individual's forearm, a 3D representation of the interior portion of the individual’s forearm may be generated. The 3D representation of the interior portion of the individual may comprise a 3D reconstruction of the one or more 2D B-mode images of the interior portion of the individual based on the movement data. For example, the 3D representation of the interior portion of the individual may comprise real-time 3D images of the interior portion of the individual as the imaging device 102 traverses the location of the individual.

[0034] The application program 157 may then cause the computing device 101 to display the 3D representation of the interior portion of the individual. In an example, the application program 157 may cause the computing device 101 to output the 3D representation of the interior portion of the individual to a second computing device (e.g.. electronic device 104). The second computing device may display the three- dimensional representation of the interior portion of the individual. In an example, the application program 157 may cause the computing device 101 to generate a visual interface comprising the 3D representation of the interior portion of the individual overlaid onto the image sequence data. The application program 157 may cause the computing device 101 to display the visual interface. The visual interface may comprise a virtual reality interface or an augmented reality interface. In an example, the display 170 of the computing device 101 may comprise a head-mounted display (HMD). In an example, the second computing device (e.g., the electronic device 104) may comprise a HMD. The HMD may be configured to display the location of the individual’s body as a user of the HMD and the imaging device 102 images the location of the individual’s body. The HMD may display the visual interface comprising the 3D representation of the interior portion of the individual overlaid onto the image sequence data in real-timeATTORNEY DOCKET NO.: 37759.0660P1 to provide a real-time view of the location of the individual’s body overlaid with the 3D representation of the interior portion of the individual as the user of the HMD and the imaging device 102 performs the one or more sweeping motions back and forth across the location on the individual.

[0035] In an example, the application program 157 may cause the computing device 101 to generate one or more measurements associated with the interior portion of the individual based on the three-dimensional representation of the interior portion of the individual. The application program 157 may cause the computing device 101 to detect one or more medical conditions associated with the individual based on the one or more measurements associated with the interior portion of the individual. The one or more medical conditions may comprise one or more of chronic allograft nephropathy, deep venous thrombosis, peripheral artery disease, stenosis, aneurisms, or abnormalities in one or more blood vessels of the individual. The application program 157 may cause the computing device 101 to cause one or more treatments of the individual based on the one or more medical conditions. For example, the computing device 101 may cause the medical treatment device to administer the one or more treatments to the individual based on the determination of the one or more medical conditions. In an example, the computing device 101 may cause the medical treatment device to adjust an administration of the one or more treatments to the individual based on the determination of the one or more medical conditions.

[0036] The input / output interface 160 may be configured as an interface for delivering an instruction or data input from a user or a different external device(s) to the processor 120, the memory 140, the display 170, and the communication interface 180. For example, the input / output interface 160 may receive user input for programming the computing device 101. For example, the input / output interface 160 may receive user input adjusting one or more settings of the computing device 101 (e.g., settings of the medical treatment device). Further, the input / output interface 160 may output an instruction or data received from the processor 120, the memory' 140, the input / output interface 160, the display 170, and / or the communication interface 180 to a different external device (e.g., imaging device 102. one or more sensor devices 103, electronic device 104, server 106, etc.).

[0037] The display 170 may include various ty pes of displays, for example, a Liquid Crystal Display (LCD) display, a Light Emitting Diode (LED) display, an Organic Light-ATTORNEY DOCKET NO.: 37759.0660P1Emitting Diode (OLED) display, a MicroElectroMechanical Systems (MEMS) display, or an electronic paper display. The display 170 may display, for example, a variety of contents (e.g., text, image, video, icon, symbol, etc.) to the user. The display 170 may include a touch screen. For example, the display 170 may receive a touch, gesture, proximity, or hovering input by using a stylus pen or a part of a user’s body. In an example, the display 170 may be configured to display the 3D representation of the interior portion of the individual. In an example, the display 170 may output the visual interface comprising the 3D representation of the interior portion of the individual overlaid onto the image sequence data. For example, the visual interface may comprise a virtual reality interface or an augmented reality interface, wherein the display 170 may comprise a HMD. The HMD may be configured to display the location of the individual’s body as a user of the HMD and the imaging device 102 images the location of the individual’s body.

[0038] The communication interface 180 may establish, for example, communication between the computing device 101 and an external device (e.g., an imaging device 102, one or more sensor devices 103, an electronic device 104, and / or a server 106). For example, the communication interface 170 may communicate with the external device (e.g., the imaging device 102, the one or more sensor devices 103, the electronic device 104, and / or the server 106) by being connected to a network 162 through wireless communication or wired communication. For example, as a cellular communication protocol, the wireless communication may use at least one of Long-Term Evolution (LTE), LTE Advance (LTE-A), Code Division Multiple Access (CDMA), Wideband CDMA (WCDMA). Universal Mobile Telecommunications System (UMTS), Wireless Broadband (WiBro), Global System for Mobile Communications (GSM), and the like. In an example, the network 162 may include at least one of a telecommunications network, a computer network (e.g., LAN or WAN), the internet, and a telephone network.

[0039] In addition, the communication interface 180 may communicate with an external device (e.g., the imaging device 102, the one or more sensor devices 103, and / or the electronic device 104) via communication connections 164, 165, 166 such as wireless communications and / or wired communications. The wireless communications may include, for example, near-distance communications 164, 165, 166. The near-distance communications 164, 165, 166 may include, for example, at least one of WirelessATTORNEY DOCKET NO.: 37759.0660P1Fidelity (WiFi), Bluetooth, Near Field Communication (NFC). Global Navigation Satellite System (GNSS). and the like. According to a usage region or a bandwidth or the like, the GNSS may include, for example, at least one of Global Positioning System (GPS), Global Navigation Satellite System (Glonass), Beidou Navigation Satellite System (hereinafter, "Beidou"). Galileo, the European global satellite-based navigation system, and the like. Hereinafter, the “GPS” and the “GNSS” may be used interchangeably in the present document. The wired communication may include, for example, at least one of Universal Serial Bus (USB), High Definition Multimedia Interface (HDMI), Recommended Standard-232 (RS-232), power-line communication, Plain Old Telephone Service (POTS), and the like.

[0040] The server 106 may include a group of one or more servers. In an example, all or some of the operations executed by the computing device 101 may be executed in a different one or a plurality of electronic devices (e.g., the imaging device 102, the one or more sensor devices 103. the electronic device 104, and / or the server 106). In an example, if the computing device 101 needs to perform a certain function or service either automatically or based on a request, the computing device 101 may request at least some parts of functions related thereto alternatively or additionally to a different electronic device (e g., the imaging device 102, the one or more sensor devices 103, the electronic device 104, and / or the server 106) instead of executing the function or the service autonomously. The different electronic devices (e g., the imaging device 102, the one or more sensor devices 103, the electronic device 104, and / or the server 106) may execute the requested function or additional function, and may deliver a result thereof to the computing device 101. In one example, the electronic device 104 may comprise one or more complex programmable logic devices (CPLDs) or other programmable logic devices. In another example, the electronic device 104 may comprise a smart phone, a mobile device, a tablet computing device, a laptop computing device, a smartwatch, and the like. As an example, the electronic device 104 may be configured to process the image sequence data and the movement data in order to generate the 3D representation of the interior portion of the individual. In example, the electronic device 104 may receive the image sequence data and the movement data from the computing device 101 after the computing device 101 receives the image sequence data and the movement data from the imaging device 102 and the one or more sensor devices 103. respectively. In an example, the electronic device 104 may receive the image sequence data and theATTORNEY DOCKET NO.: 37759.0660P1 movement data directly from the imaging device 102 and the one or more sensor devices 103, respectively. As an example, the server 106 may be configured to process the image sequence data and the movement data in order to generate the 3D representation of the interior portion of the individual. In an example, the server 106 may receive the image sequence data and the movement data from the computing device 101 after the computing device 101 receives the image sequence data and the movement data from the imaging device 102 and the one or more sensor devices 103. respectively. In an example, the server 106 may receive the image sequence data and the movement data directly from the imaging device 102 and the one or more sensor devices 103, respectively.

[0041] Each of the constitutional elements described in the present document may consist of one or more components, and names thereof may vary depending on a ty pe of an electronic device. The computing device 101 may include at least one of the constitutional elements described in the present document. Some of the constitutional elements may be omitted, or additional other constitutional elements may be further included. Further, some of the constitutional elements of the computing device 101 according to various exemplary' embodiments may be combined and constructed as one entity’, so as to equally perform functions of corresponding constitutional elements before combination.

[0042] FIG. 2 shows an example process 200 for generating three-dimensional (3D) ultrasound images (e.g., 3D representations). An ultrasound system 210 may comprise an imaging device 102 and one or more sensor devices 103. In addition, the ultrasound system 210 may7further comprise a computing device (e.g., computing device 101) for processing movement data 220 generated and output by the one or more sensor devices 103 and image sequence data 230 generated and output by the imaging device 102. For example, the ultrasound system 210 may generate movement data 220 and image sequence data 230 of an interior portion of an individual (e.g., patient) as the imaging device traverses a location on a an individual. The imaging device 102 may comprise an ultrasound scanner. For example, the imaging device 102 may comprise a probe comprising an ultrasound scanner. For example, the imaging device 102 may generate the image sequence data 230 of an interior portion of the individual as a user of the imaging device 102 performs one or more sweeping motions back and forth across the location on the individual. The interior portion of the individual may comprise one orATTORNEY DOCKET NO.: 37759.0660P1 more of one or more blood vessels, one or more organs, or one or more layers of tissue. The image sequence data 230 may comprise a series of one or more two-dimensional (2D) B-mode images of the interior portion of the individual. For example, the imaging device 102 may comprise a 2D ultrasound scanner configured to output 2D B-mode (e.g., ultrasound) images. The one or more sensor devices 103 may comprise one or more of an internal measurement unit (IMU), an electromagnetic sensor, an optical sensor, an airborne ultrasound sensor, a time of flight sensor, a light detection and ranging (LiDAR) sensor, a sonomicrometry sensor, a smartcable, a depth camera, or a tactile sensor. As an example, the one or more sensor devices 103 may be affixed to one or more of the imaging device 102 or the location on the individual. The movement data 220 may comprise positional data and motion data, wherein the positional data may comprise spatial data and orientation data.

[0043] In one example, an IMU may be affixed to the imaging device 102, wherein the IMU may output movement data 220 as the imagine device 102 traverses the location on the individual. For example, the IMU may output the movement data 220 as the user of the imaging device 102 performs the one or more sweeping motions back and forth across the location on the individual. For example, the movement data 220 may comprise probe pose data calculated from the IMU affixed to the probe.

[0044] In another example, a tracking system (e.g., a smartcable) may be attached to an electric wire connecting the imaging device 102 to the computing device, or an ultrasound cart, wherein the tracking system may generate and output the movement data 220. As an example, as shown in FIGS. 3A-3C, a smartcable design 300 may comprise a high-strength wire 302 connected to a plurality of lightweight hollow carbon fiber rods / linkages 304, wherein the plurality of lightweight hollow carbon fiber rods / linkages 304 may be connected via a lightweight, universal joint mechanism 310. The unique joint mechanism 310 may enable, via the contact point 308 unrestricted three-dimensional rotation of adjacent linkages and constraints relative to translational motion, as shown in FIGS. 3B-3C. In addition, the unique joint mechanism 310 may reduce bulk and friction, allowing smooth motion by reducing the number of moving parts and surface contact area. As an example, as shown in FIGS. 3A-3C, the unique joint mechanism 310 may comprise one or more parts such as one or more nozzle heads 306 (e.g., 3D printed nozzle heads), a high-strength wire 302, and a wire tensioner 312. The one or more nozzle heads 306 may be attached at the ends of the hollow carbon fiberATTORNEY DOCKET NO.: 37759.0660P1 rods 304 with the high-strength wire 302 passing through the nozzle heads 306. In an example, the wire 302 may be kept in a tensed state using the wire tensioner 312. When the wire 302 is tensed, it may bring the adjacent links closer, aligning their centers to a contact point. For example, the tension of the wire 302 may help compensative for gravity, making the linkage system behave more like a lightweight wire. The smartcable 300 may be attached to the imaging device 102 using an attachment 314 (e.g., probe attachment), as shown in FIG. 3E. As such, the movements / motion of the imaging device 102 may be tracked using forward kinematics of serially connected linkages of the smartcable 300. An orientation of the links may be obtained using either a mechanical / optical encoder, a 3D depth camera, and / or an IMU. In an example, IMUs on the linkages of the smartcable may be used to obtain an absolute orientation of the individual linkages in 3D space. This information, along with length of the linkage, may be used to accurately track a pose and orientation (e.g., movement data 220) of the imaging device 102 to generate volumetric reconstruction from the image sequence data 230 output by the imaging device 102. In an example, the smartcable may comprise ultra-light rigid segments, connected lightweight universal j oints that measure changes in angles, where one end of the assembly may be attached to the imaging device 102 and the other end may have a fixed (or if in motion a determined, known or measured location), whereby the measured angle changes are combined to calculate or determine the position and angle of the imaging device 102. In an example, the smartcable may comprise segments that are connected by a high-strength wire arrangement. In an example, the smartcable may comprise adjacent linkages that are suspended in the air rather than having the adjacent linkages fall due to their weight, allowing the smartcable to feel lighter and more like a cable.

[0045] At 240, the imaging device 102 may be synchronized with the one or more sensor devices 103. For example, synchronization of the imaging device 102 with the one or more sensor devices 103 may allow the movement data 220 to be precisely associated with stacked frames / images of the image sequence data 230 (e.g., as the imaging device 102 sweeps a location of the individual). As an example, a scanning process may be initiated and deactivated based on an initial movement of the imaging device 102 and an ending movement of the imaging device 102, respectively. For example, the scanning process may be initiated based on an initial gesture movement signaling to the computing device the initiation of the scanning process (e g., as shownATTORNEY DOCKET NO.: 37759.0660P1 in FIG. 4) In addition, the scanning process may be deactivated based on an ending gesture movement signaling the computing device to end the scanning process. The gesture may comprise a movement of the imaging device 102 in a designated way (e.g., a movement distinct from the sweeping movements used during the scanning process). For example, the movement may comprise linear and / or angular reciprocation, vibration, or other velocity-controlled motion. For example, as shown in FIG. 4, one or more of the sensor devices 103 may be affixed to the imaging device 102, wherein the gesture movement may comprise a pushing motion that is repeated to compress tissue (e g., of the individual) several times. The gesture movement may be captured by the sensor device(s) 103, wherein the computing device 101 may identify the gesture movement as a movement for initiating the scanning process. For example, ultrasound speckle tracking may be utilized to identify the bulk, in-plane motion created by the gesture movement. For example, as shown in FIG. 4, as movement data 402 is captured by the sensor device 103, the gesture movement 404 may be identified by the computing device. In an example, the gesture movement 404 may be identified by elastographic strain imaging 406 (e.g., based on images captured by the imaging device 102). For example, as shown in FIG. 4, two points, in the elastographic strain image sequence data 406, may be selected and their distance may be tracked over time, wherein the computing device may identify the gesture movement 408 from the elastographic strain image sequence data 406. Based on the initial gesture movement, the imaging device 101 may be synchronized with the one or more sensor devices 103. For example, the imaging device 102 may be synchronized with the one or more sensor devices 103 by aligning the image sequence data 230 output by the imaging device 102 with the movement data 220 output by the one or more sensor devices 103. For example, as shown in FIG. 4, the gesture movement 404 identified from the movement data 402 and the gesture movement 408 identified from the elastographic strain image sequence data 406 may be correlated to synchronize the movement data 402 and the image sequence data 406 over time. As shown in FIG. 5, a trajectory 500 of a sensor device 103 (e.g.. an IMU) may be aligned with a trajectory of the imaging device 102 (e.g., ultrasound probe), wherein the alignment of the trajectories may increase an accuracy of the ultrasound system 210 to a sub-millimeter position and a sub-degree rotation.

[0046] In an example, optical tracking methods may be implemented to measure a position and orientation of the imaging device 102 as an additional or alternative methodATTORNEY DOCKET NO.: 37759.0660P1 to the use of the movement data 220 for tracking the imaging device 102 as the imaging device 102 traverses the location on the individual. These methods may provide significant advantages for computer vision tracking, such as access to a tracked object (e.g., via the imaging device 102), such as blood vessels, organs, or layers of tissue, and for control of the environment, such as imaging device 102 positioning with respect to object and lighting. Compared to tracking features and textures provided by the tracked object, marker-based tracking methods provide significant improvement in robustness and accuracy. As an example, both fiducial pattern and point constellation may be combined tracking methods. For fiducial pattern tracking, a standard marker used in computer vision (e.g.. Aruco or April pattern) may be attached to the imaging device 102 and a standard HD camera may be positioned to image the marker and / or the imaging device 102 while the imaging device 102 traverses the location on the individual. As an example, for point constellation, tracking point markers (e.g., brightly colored dots) or light sources (e.g., light emitting diode set on a circuit board) may be attached to the imaging device 102, and the HD camera can record the tracking point markers while the imaging device 102 captures the images as the imaging device 102 traverses the location on the individual.

[0047] In an example, one of the sensor devices 103 may comprise a 3D depth camera that may be affixed to the imaging device 102. RGB-D image data output by the 3D depth camera and the movement data 230 (e.g., IMU data) may be used to determine imaging device 102 motion estimation. As an example, as shown in FIG. 6, an accuracy of the ultrasound system 210 may be increased based on an implementation of a portable L-shaped feature wall 602 equipped with high-contrast features and fiducial markers (e.g., ArUco tags), which may provide stable and reliable visual reference, improving pose estimation precision and minimizing drift during 3D image reconstruction. For example, since the fiducial markers lie on a flat plane, planar homography may be utilized to increase the accuracy of the pose estimation and loop closure. The 3D depth camera may be affixed to the imaging device 230 to capture a surrounding environment as the imaging device 102 moves. The image data output by the 3D depth camera may be used with the IMU data in order to increase an accuracy of the image device’s 102 position and orientation in real-time by providing additional spatial information. For example, the fiducial markers, along with natural landmarks (e.g.. features) of the individual, may be used to estimate imaging device 102 pose data / information,ATTORNEY DOCKET NO.: 37759.0660P1 represented by transformation Tcam. which includes both the rotation R and translation T[R T~\j. For a given pixel (x, y) in the image, the corresponding 3D point in the camera coordinate system Pcammay be computed as:

[0048] where K is an intrinsic camera matrix of the imaging device 102 and D x,y) represents a depth value at a specific pixel. This 3D point is then transformed into the world coordinate system using the estimated pose Tcam, providing a constraint on possible locations and orientations of the imaging device 102, increasing an accuracy of the pose estimation. As an example. IMU data may be used to further enhance the accuracy of the pose estimation. The IMU may provide measurements of linear acceleration (aimu) and angular velocity (<x)imu), which may be used to predict the image device’s 103 motion over time. The rotational motion may be calculated as: Rimu=exp( t . u>jmu) where exp(. ) denotes matrix exponential

[0049] and the translation motion may be estimated by:

[0050] These predictions are then fused with visual and depth data using an Extended Kalman Filter (EKF), which optimally combines the different sources of information (e.g., measurements of the different sensor devices 103) to produce a refined estimate of the image device’s 102 pose. In an example, a relationship betw een the views of the flat plane of the fiducial markers may be described by the homography matrix H. which is decomposed to extract rotation R and translation T

[0051] where n' may represent a transpose of a normal vector to the flat plane and d may represent a distance from the imaging device 102 to the plane. This geometric constraint may improve a consistency and accuracy of the ultrasound system 210, particularly in reducing drift during a 3D reconstruction process. This can also be used for interventional procedures.

[0052] In an example, ultrasound speckle tracking (e.g., speckle correlation tracking) may be implemented to measure a position and orientation of the imaging device 102 asATTORNEY DOCKET NO.: 37759.0660P1 an additional or alternative method to the use of the movement data 220 for tracking the imaging device 102 as the imaging device 102 traverses the location on the individual. For example, ultrasound speckle tracking relies on regional pattern matching of ultrasound signals between frames / images to determine local tissue motion. From these measurements, mechanical and dynamic properties of the tissue may be determined bycalculating deformation (e.g., strain), tissue velocity, and displacement. For correlation based speckle tracking, a kernel size may be adjusted based on a desired resolution and noise robustness of a resulting motion field. For example, normalized cross correlation may be used to determine (e.g., measure) a motion and degree of speckle pattern match:

[0053] where T and k may denote mean subtracted images. A correlation lag may be represented by indices l,m. This expression describes taking a subset from image I about location x.y. and evaluating a pattern match in image k about position x,y. The subset size may be determined by an extent of the summation i,j (e.g., window size W x H). A shift of the image k, or equivalent search location, may be set by the lags 1 and m. To calculate a regional motion between ultrasound frames / images, image I may be set to a current frame and the image k may be set to a subsequent frame / image. The lag indices of the peak correlation coefficient (e.g.. maximum correlation magnitude) may describe the motion at pixel location x,y.

[0054] In an example, this method may be repeated for all image / frame pairs and pixel locations (or subset of pixel locations) to measure the tissue and flow motion through an ultrasound image loop. This algorithm, in addition to the Kanade Lucas Tomasi (KLT) and Famebeck algorithms provide an in-plane motion estimation to determine an imaging device 102 position. In an example, images may be acquired with a slightly- angled image plane with respect to the imaging device 102 out-of-plane motion in order to estimate out-of-plane imaging device 102 motion, which may produce a motion component in-plane that may be tracked using the above methods. The imaging device 102 motion may be calculated from an in-plane motion measurement using speckle tracking and the tilt angle. Additionally, an interframe correlation of the image sequence data 230 may be utilized to measure out-of-plane motion. For homogeneous static tissues, the speckle decorrelation (e.g., 1 -correlation) may be proportional to the out-of-ATTORNEY DOCKET NO.: 37759.0660P1 plane displacement and the elevational (e.g., out-of-plane) ultrasound beam characteristics.

[0055] In an example, in-plane motion may be used to register imaging device 102 position in the x and y (e.g., lateral and vertical translation) and theta (e.g., rotation within the plane), and a single direction of out-of-plane motion information may be provided by at least one of the one or more sensor devices 103 (e.g., passive robot, IMU, electromagnetic sensor, optical sensor, airborne ultrasound sensor, time of flight sensor, LiDAR)sensor, sonomicrometry sensor, smartcable, depth camera, a tactile sensor, etc.). The out-of-plane motion may be orthogonal to the ultrasound plane or at a known (e.g., determined) direction to allow image registration for 3D image reconstruction.

[0056] In an example, for in-plane motion, the imaging device 102 may be moved in the out-of-plane motion at a known (e.g., predetermined) velocity, wherein the velocity may contain a component of out-of-plane motion, which me be orthogonal or approximately orthogonal, or may not be orthogonal but at a known out-of-plane angle from the ultrasound plane.

[0057] In an example, in-plane motion tracking associated with a first sensor device 103 of the one or more sensor devices 103 may be used to improve an accuracy of measurements (e.g., tracking, positioning) made be a second sensor device 103 of the one or more sensor devices 103 (e.g., positioning).

[0058] In an example, a robotic ultrasound system 210 may be utilized to augment and / or control the imaging device 102 positioning. For example, a robot may be attached the imaging device 102. wherein the robot may cause the imaging device 102 to travers the location on the individual by causing the imaging device 102 to sweep back and forth across the location on the individual.

[0059] At 250, the image sequence data 230 may be combined with the movement data 220. For example, the image sequence data 230 and the movement data 220 may be combined into image digital imaging and communications in medicine (DICOM) files, wherein the movement data 220 may be stored in a private field of the image DICOM file. For example, this may enable a standard data interface. In addition, this may enable storage and communication with systems such as picture archiving and communication systems (PACS).

[0060] At 260, image segmentation 262 and decorrelation processing 264 may be performed on the image sequence data 230 in order to identify different properties of theATTORNEY DOCKET NO.: 37759.0660P1 interior portion of the individual. For example, one or more object (e.g., blood vessel) locations of the interior portion of the individual may be identified based on the image segmentation 262 and the decorrelation processing 264. As an example, singular value decomposition (SVD) may be implemented to perform the image segmentation 262 processing, leveraging its ability to decompose a matrix representation of an image into its constituent components, which can be used to identify and isolate different image regions or objects within the image. By analyzing singular values and vectors. SVD may be implemented to effectively separate foreground and background elements, enhancing the image segmentation 262 process. As an example, a vessel distribution map (VDM) may be generated based on the image data combined with the movement data. For example, the VDM may be used to identify a high probability of the blood vessels within the image sequence data. As an example, the VDM may be represented as a 3D matrix (array) containing values corresponding to the probability a vessel is located at a location within the array. As an example, grayscale ultrasound data (e.g., backscatter intensitydata) in addition to the image sequence data 230 may contain data from both lumen and non-lumen structures, where lumen locations within the image sequence data 230 may comprise a different intensify than non-lumen structures. As an example, the image segmentation 262 procedure may be used to separate potential lumen from non-lumen structures in the grayscale ultrasound data. As an example, a rule or set of rules where the VDM values derived from the movement data 220 may be used to select probable lumen segments from the grayscale ultrasound data based on the lumen segments’ relative location (e.g., nearness) to each other or their coincident location. In an example, the VDM may be processed to generate a final VDM. For example, a series of filters (e.g., morphological filtering, media filters, etc.) may be applied to the VDM, selecting subsets of VDM data that have certain shape attributes (e.g., a minimum number of pixels along the path of vessel tracking and a maximum shift in location perpendicular to the tracking path). One or more vessel (e.g., blood vessel) locations may be identified within an interior portion of the individual in the image sequence data 230 based on the VDM.

[0061] In an example, the movement data 220 in the combined image data may be utilized to establish probable locations of vessel lumen in the image sequence data 230, wherein one or more characteristics may be added to these locations For example, the one or more characteristics may comprise geometric properties (e.g., constraints)ATTORNEY DOCKET NO.: 37759.0660P1 exhibited by the vessels, such as sufficient size (e.g., size = number of pixels exhibiting motion that are adjacent to each other or near each other) or smoothness constraints, which may apply real world knowledge about vessels (e.g., blood vessels) into the image sequence data 230. The movement data 220 and the constraints may be utilized to generate the VDM identifying a high probability of a vessel location within one or more images of the image sequence data 230. The constraints may include further processing of the VDM to generate the final VDM after applying the series of filters (e.g.. morphological filtering, median filters, etc.) and selecting the subsets of VDM data that have certain shape attributes (e.g., a minimum number of pixels along a tracking path of the blood vessel and a maximum shift in location perpendicular to the tracking path). In an example, the constraints may be derived based on machine learning and artificial intelligence models.

[0062] In an example, unsuppressed grayscale data, or data, or subsets of data, that do not include movement data 220 may be obtained. For example, this data may be received from conventional ultrasound, or from a research scanner (e.g., Verasonics) where data channels may be customized to include movement data 220 in one channel and not in another channel (e.g., 3 channels of 8, but unsigned integer data [values 0 to 255] are in each channel). When Doppler signals are utilized, two sets of image sequence data 230 (e.g., one with the Doppler signals and one without the Doppler signals) may be obtained. In an example, alternatively, decorrelation data processing schemes may utilize a single set of image sequence data 230. Thresholding may be performed based on ultrasound system 210 parameters and / or user pre-sets for grayscale data. Segmentation algorithms may be utilized based on rule sets using nearness, union, and intersection of binarized segments with the VDM. Segments fulfilling rule set criteria may be utilized to perform image sequence data 230 final filtering and registration. The utilization of the movement data 220 and unsuppressed grayscale data may allow for the selection of segments from grayscale data that will exclude nonluminal structures, and more accurately include intra-luminal regions than by simply using grayscale data alone.

[0063] In an example, segment selection from grayscale data may be automated based on a spatial distribution of velocity signals in the movement data 220 and the image sequence data 230. This may reduce a sensitivity to thresholding values and may allow for standardized threshold values to be used instead of performing segmentation multipleATTORNEY DOCKET NO.: 37759.0660P1 times, re-thresholding, and repeating the process. The VDM may be used on the segmented data, which may routinely create a plurality of segments per data-stack. to automate segment selection. As an example, 2D image planes of the image sequence data 230 may be assembled into a 3D data block with the temporal axis corresponding to the longitudinal axis along the vessel (e.g., the z-axis, orthogonal to the 2D x-y ultrasound image plane).

[0064] In an example, speckle decorrelation may be implemented as a means of vessel edge detection for augmenting methods of image segmentation, especially since B-mode signals within the blood vessel lumen are often dominated by electronic noise and blood scattering. For example, for image segmentation, a temporal (e.g., frame to frame) correlation of the image sequence data 230 may be utilized to distinguish between regions of vessel lumen and surrounding tissues. The temporal change of these signals is greater than the surrounding tissue B-mode signals, producing higher decorrelation within the blood vessel lumen and consequently contrast between lumen and surrounding tissue. In addition, the correlation measurement may be independent of B-mode signal strength which can be affected by shadowing and other artifacts. This technique can be used, for example, where the blood vessel wall is poorly visualized, which may cause difficulties for edge detection and for identification of the vessel lumen boundary’. As shown in FIGS. 7A-7B, vessel edge visualization may be performed by creating decorrelation isosurfaces to show vessel boundaries. By including decorrelation data into the vessel detection process vessel edge sharpness may be improved. In one example, thresholded grayscale data may be utilized to generate segments in combination with velocity signal data from the same special region and a nearness parameter that allows selection of segments that represent blood vessels. This may allow automation of vessel mapping and 3D vessel image generation (e.g., a 3D representation of the vessel). The use of a nearness parameter (e.g., a constraint) may allow proper segment selection. In another example, in addition to the use of the thresholded grayscale data, morphologic filtering or smoothness parameters, such as a spline curve fitting procedure, may be used to add constraints to selected vessel segments, or to the entire set of combined selected vessel segments to construct a 3D vascular image that more accurately represents a real vessel being imaged. In another example, speckle decorrelation may be utilized to sharpen the vessel edge while using grayscale ultrasound data. As such, certain valuesATTORNEY DOCKET NO.: 37759.0660P1 from the decorrelation signal, such as local maxima or minima, corresponding to systole and diastole, may be used and may also represent the velocity signal.

[0065] As an example, one or more fiducial markers may be utilized during the scanning process when the imaging device 102 generates the image sequence data 230 as the imaging device 102 traverses the location on the individual (e.g., according to a sweeping motion back and forth across the location). In one example, a length of medical adhesive tape may be notch cut at periodic distances and positioned at a lateral edge of the scan region (e.g., location on the individual) such that a small portion of the imaging device 102 field of view overlays the tape during the scanning process. When the imaging device 102 passes over the periodic notched gaps, the image shadowing produced by the tape at the field of view edge will change, providing an indicator of imaging device 102 travel distance. In an example, a user of the imaging device 102 may implement a “sweep and hold” technique for capturing the image sequence data 230. For example, this technique may allow four-dimensional (4D) mapping (e.g.. 3D plus information extracted from a pulsatile motion of a vessel).

[0066] In another example, a visually and acoustically transparent film may be placed on the individual’s arm along the vessel path with acoustically coupling ultrasound gel. The film may contain small acoustic and visual reflectors that are coded with dots and dashes (e.g., analogous to Morse code) at intervals along the film. The user of the imaging device 102 may perform a standard sweep scan but may pause, or hold, at designated interval locations. This may allow the computing device to encode and couple image sequence data 230 output by the imaging device 102 with the movement data 220 output by the one or more sensor devices 103. In addition, this may allow the image sequence data 230 combined with the movement data 220 to be linked with a signature signal encoded in the image sequence data 230.

[0067] In another example, an ultrasound quick response (QR) code / pattem that encodes location and varies with angle of insonification may be utilized to provide additional movement data during the sweep and hold technique. For example, acoustically translucent reflectors may be embedded to encode the QR code on an edge of a thin (e.g., less than 1 cm) ultrasound standoff pad without interfacing with an underlying imaging plane. As an example, spacing between QR code patches across a surface of the imaging device 102 (e.g., probe surface) may be adjusted (e.g., a nominal spacing of less than 20 code patches across the surface of the imaging device 102 inATTORNEY DOCKET NO.: 37759.0660P1 order to minimize shadowing below in an image of the image sequence data 230). As an example, part of the field of view on the edge of the imaging device 102 may be used for QR (or other) position information where subsurface features of interest are in the central areas of an image of the image sequence data 230. In addition to the visible ultrasound markers, a visible or acoustically shadowing accelerometer or IMU device (e.g., one of the sensor devices 103) may be placed on or within tape or the standoff pad on the edge of the field of view of the imaging device 102. providing an IMU signal as the imaging device 102 moves over the sensor device 103. As an example, this may also serve to couple the imaging device 102 movement data 220 (e.g., probe IMU information) with the image sequence data 230.

[0068] At 270, a 3D representation of the interior portion of the individual may be generated based on the image sequence data combined with the movement data and based on the VDM. The 3D representation of the interior portion of the individual may comprise a 3D reconstruction of the one or more two-dimensional B-mode images of the interior portion of the individual based on the movement data. For example, the 3D representation of the interior portion of the individual may comprise real-time 3D images of the interior portion of the individual as the imaging device 102 traverses the location of the individual. For example, as shown in FIGS. 8A-8B, a top-down projection of a sum along the y-direction of the x-z plane slices of binarized, segmented image sequence data 230 of an individual’s Doppler lumen segment, forming an sono-angiogram (e.g., FIG. 8A) with a fistulogram (e.g., FIG. 8B), may be captured in order to produce a 3D representation (e.g., FIGS. 8C-8E) of the interior portion of the individual (e.g., a binarized blood vessel segment). In an example, the computing device of the ultrasound system 210 may be configured to implement one or more machine learning (MU), or artificial intelligence (Al), modules to generate the 3D representation. For example, the ML / AL modules may be trained based on a plurality of 2D images, with slice position and orientation data, for example. The trained ML / AL modules may be configured to generate a 3D reconstruction of the one or more two-dimensional B-mode images (e.g., without sensors cables, tracking, etc.).

[0069] In an example, a visual interface may be generated comprising the 3D representation of the interior portion of the individual overlaid onto the image sequence data 230, and the visual interface may be displayed to a user of the ultrasound system 210. For example, the ultrasound sy stem 210 may include a display device, wherein theATTORNEY DOCKET NO.: 37759.0660P1 visual interface may comprise a virtual reality (VR) interface or an augmented reality (AR) interface that may be displayed via the display device. As an example, the display device may comprise a head-mounted display (HMD). The HMD may be configured to display the location of the individual’s body as a user of the HMD and the imaging device 102 images the location of the individual’s body. The HMD may display the visual interface comprising the 3D representation of the interior portion of the individual overlaid onto the image sequence data in real-time to provide a real-time view of the location of the individual’s body overlaid with the 3D representation of the interior portion of the individual as the user of the HMD and the imaging device 102 performs the one or more sweeping motions back and forth across the location on the individual.

[0070] In an example, the visual interface may be recorded, or stored, for post processing that may be performed after a diagnostic procedure utilizing the ultrasound system 210. There may also be redundant sweep scans from the same or different angles so that the processed data may be combined from multiple scans to increase the accuracy of a rendered (e.g., viewed) target (e.g. blood vessel or other structure).

[0071] In an example, information (e.g., data) may be generated for the target (e.g. blood vessel) for a specific region from data collected from multiple sweeps of the imaging device 102 to average, compound, or synthesize an image from multiple frames / images from the same or similar or overlapping frames / images. As such, the user may view an image of the vessel constructed in real-time (or near real-time) within the user’s field of view as the user gradually moves the imaging device 102 using multiple gradual sweeping motions, such as by moving the imaging device 102 along the arm, leg, or neck of the individual.

[0072] In example, the imaging device 102 may sweep scan areas of interest of the individual such as a narrowing (stenosis) or region of plaque in 3D by the user to increase the input data for improved imaging quality and accuracy, or to gather data for more complex and computationally intensive procedures such as strain or elasticity imaging. In addition to sweeping, the imaging device 102 may be held in a single location on the individual to gather ultrasound information over time. For example, the imaging device 102 may be held over a region of plaque or stenosis in the carotid artery of the individual. Through processing multiple frames / images, 2D strain or elasticity images may be constructed in real-time or in near real-time. In an example, real-time information may be fed back to the user to help improve a quality of image dataATTORNEY DOCKET NO.: 37759.0660P1 collection, such as signaling out of plane motion decorrelation, to allow the user to keep the imaging device 102 plane stable. Thus, the data from a diagnostic procedure may be collected and stored for post processing diagnostic procedures.

[0073] As an example, as shown in FIGS. 9A-9B. an AR device 920 (e.g., HMD) may superimpose image sequence data 230 (e.g.. ultrasound B-mode images, elastography strain images, and 3D sono-angiography) on the individual’s anatomy at corresponding positions and orientations. The image sequence data 230 may be collected by the computing device, or a point-of-care ultrasound system (POCUS). The display location and orientation of the AR features may be determined based on the image sequence data 230 output by the imaging device 102 (e.g.. probe) combined with the movement data 220 output by the one or more sensor devices 103 as the imaging device 102 traverses the location on the individual (e.g., as a user of the image device performs one or sweeping motions across the location). For example, at 902 image sequence data 230 (e.g.. during point of care ultrasound acquisition) may be generated. The image sequence data 230 may comprise a series of one or more two-dimensional (2D) B-mode images of the interior portion of the individual. At 904, elastography strain imaging edge detection may be performed on the image sequence data 230. At 906, movement data 220 may be determined as the imaging device 102 traverses the location on the individual (e.g.. probe motion perspective-n-point (PNP) tracking). For example, the movement data 220 may comprise probe position and orientation data (e g., motion data and positional data such as spatial and orientation data). At 908, the movement data 220 may be synchronized with the image sequence data 230. The synchronized movement data 220 and image sequence data 230may be combined to generate a 3D representation of the interior portion of the individual and output to the AR device 920 to be displayed to the user. For example, the AR device 920 may display a B-mode imaging overlay 942, an elastography strain imaging overlay 944, and / or 3D sono-angiography overlay 946. As shown in FIG. 9B, the AR device 920 may superimpose the B-mode imaging overlay 942, the elastography strain imaging overlay 944, and / or the 3D sono- angiography overlay 946 on the individual’s anatomy at corresponding positions and orientations.

[0074] FIG. 10 shows a flowchart of an example method 1000 for generating a three- dimensional (3D) representation of an interior portion of an individual based on image sequence data and movement data. Method 1000 may be implemented by a computingATTORNEY DOCKET NO.: 37759.0660P1 device 101, an imaging device 102, one or more sensor devices 103, an electronic device 104, a server 106, or any combination thereof. At step 1002, subsequent movement data received from one or more sensor devices and image sequence data received from an imaging device as the imaging device traverses a location on an individual may be determined based on initial movement data associated with the imaging device. For example, a computing device (e.g., the computing device 101) may determine the subsequent movement data received from the one or more sensor devices (e.g.. the one or more sensor devices 103) and the image sequence data received from the imaging device (e.g., the imaging device 102) as the imaging device traverses the location on the individual (e.g., patient) based on the initial movement data associated with the imaging device. The imaging device may comprise an ultrasound scanner. For example, the imaging device may comprise a probe comprising an ultrasound scanner. For example, the imaging device may comprise a 2D ultrasound scanner configured to output 2D B- mode (e.g., ultrasound) images. The one or more sensor devices may comprise one or more of an internal measurement unit (IMU), an electromagnetic sensor, an optical sensor, an airborne ultrasound sensor, a time of flight sensor, a light detection and ranging (LiDAR) sensor, a sonomicrometry sensor, a smartcable, a depth camera, or a tactile sensor. The image sequence data may comprise a series of one or more two- dimensional B-mode images of the interior portion of the individual. As an example, the imaging device may generate the image sequence data of an interior portion of the individual as a user of the imaging device performs one or more sweeping motions back and forth across the location on the individual. The interior portion of the individual may comprise one or more of one or more blood vessels, one or more organs, or one or more layers of tissue. One or more of the initial movement data or the subsequent movement data may comprise positional data and motion data, wherein the positional data may comprise spatial data and orientation data. The initial movement data may be associated with a gesture movement. As an example, the one or more sensor devices may be affixed to one or more of the imaging device or the location on the individual. The one or more sensor devices may determine (e.g.. generate) the subsequent movement data as the user of the imaging device performs the one or more sweeping motions back and forth across the location on the individual.

[0075] In an example, a scanning process may be initiated and deactivated based on an initial movement of the imaging device and an ending movement of the imaging device,ATTORNEY DOCKET NO.: 37759.0660P1 respectively. For example, the scanning process may be initiated based on an initial gesture movement signaling to the computing device the initiation of the scanning process. In addition, the scanning process may be deactivated based on an ending gesture movement signaling the computing device to end the scanning process. The gesture may comprise a movement of the imaging device in a designated way (e.g., a movement distinct from the sweeping motions used during the scanning process). For example, the movement may comprise linear and / or angular reciprocation, vibration, or other velocity-controlled motion. For example, the gesture movement may comprise a pushing motion that is repeated to compress tissue (e.g., of the individual) several times. The gesture movement may be captured by the sensor device(s), wherein the computing device may identify the gesture movement as a movement for initiating the scanning process. In an example, ultrasound speckle tracking may be utilized to identify the bulk, in-plane motion created by the gesture movement.

[0076] At step 1004, the imaging device may be synchronized with the one or more sensor devices based on the subsequent movement data and the image sequence data. For example, the computing device (e.g., computing device 101) may cause the imaging device (e.g., the imaging device 102) to synchronize with the one or more sensor devices (e.g.. the one or more sensor devices 103) based on the subsequent movement data and the image sequence data. As an example, the imaging device may be synchronized with the one or more sensor devices by aligning the image sequence data output by the imaging device with the subsequent movement data output by the one or more sensor devices. As an example, based on the initial movement data associated with the gesture movement, the imaging device may synchronize with the one or more sensor devices.

[0077] At step 1006, the image sequence data may be combined with the subsequent movement data based on the synchronization of the imaging device with the one or more sensor devices. For example, the computing device (e.g., computing device 101) may combine the image sequence data with the subsequent movement data based on the synchronization of the imaging device with the one or more sensor devices. As an example, the image sequence data combined with the subsequent movement data may be stored as a digital imaging and communications in medicine (DICOM) file. In an example, the subsequent movement data may be stored in a private filed of the DICOM file.ATTORNEY DOCKET NO.: 37759.0660P1

[0078] In an example, image segmentation and decorrelation processing may be performed based on the image sequence data. One or more blood vessel locations of the interior portion of the individual may be identified based on the image segmentation and the decorrelation processing. As an example, a vessel distribution map (VDM) may be generated based on the image data combined with the movement data. For example, the VDM may be used to identify a high probability of the blood vessels within the image sequence data. As an example, the VDM may be represented as a 3D matrix (array) containing values corresponding to the probability a vessel is located at a location within the array. As an example, grayscale data (e.g., backscatter intensity data) in addition to the image sequence data may contain data from both lumen and non-lumen structures, where lumen locations within the image sequence data may be a different intensity than non-lumen structures. As an example, image segmentation may be used to separate potential lumen from non-lumen structures in the grayscale data. As an example, a rule or set of rules where the VDM values derived from the movement data may be used to select probable lumen segments from the grayscale data based on lumen segments' relative locations (e.g., nearness) to each other or their coincident location. In an example, the VDM may be processed to generate a final VDM. For example, a series of filters (e.g., morphological filtering, media filters, etc.) may be applied to the VDM, selecting subsets of VDM data that have certain shape attributes (e.g., a minimum number of pixels along a path of vessel tracking and a maximum shift in location perpendicular to the tracking path). The one or more blood vessel locations may be identified within an interior portion of the individual in the image sequence data based on the VDM.

[0079] At step 1008, a three-dimensional representation of an interior portion of the individual may be generated based on the image sequence data combined with the subsequent movement data. For example, the computing device (e.g.. computing device 101) may generate the three-dimensional representation of the interior portion of the individual based on the image sequence data combined with the subsequent movement data. In an example, the three-dimensional representation of the interior portion of the individual may be generated based on the image sequence data combined with the subsequent movement data and based on the identification of the one or more blood vessel locations. The three-dimensional representation of the interior portion of the individual may comprise a three-dimensional reconstruction of the one or more two-ATTORNEY DOCKET NO.: 37759.0660P1 dimensional B-mode images of the interior portion of the individual based on the subsequent movement data. In an example, the three-dimensional representation of the interior portion of the individual may be displayed (e.g., by the computing device). In an example, the three-dimensional representation of the interior portion of the individual may be output to a second computing device, wherein the second computing device may display the three-dimensional representation of the interior portion of the individual.

[0080] In an example, a visual interface comprising the three-dimensional representation of the interior portion of the individual overlaid onto the image sequence data may be generated. In an example, the visual interface comprising the three- dimensional representation of the interior portion of the individual overlaid onto the image sequence data may be displayed (e.g., by the computing device). In an example, the visual interface comprising the three-dimensional representation of the interior portion of the individual overlaid onto the image sequence data may be output to a second computing device, wherein the second computing device may display the visual interface. The visual interface may comprise a virtual reality interface or an augmented reality interface.

[0081] As an example, the second computing device may comprise a head-mounted display (HMD). The HMD may be configured to display the location of the individual’s body as a user of the HMD and the imaging device images the location of the individual's body. The HMD may display the visual interface comprising the three- dimensional representation of the interior portion of the individual overlaid onto the image sequence data in real-time to provide a real-time view of the location of the individual’s body overlaid with the three-dimensional representation of the interior portion of the individual as the user of the HMD and the imaging device performs the one or more sweeping motions back and forth across the location on the individual.

[0082] FIG. 11 shows a flowchart of an example method 1100 for generating a three- dimensional (3D) representation of an interior portion of an individual based on image sequence data and movement data. Method 1100 may be implemented by a computing device 101, an imaging device 102, one or more sensor devices 103, an electronic device 104, a server 106, or any combination thereof. At step 1102, movement data may be received from one or more sensor devices and image sequence data may be received from an imaging device as the imaging device traverses a location on an individual. For example, a computing device (e.g., computing device 101) may receive the movementATTORNEY DOCKET NO.: 37759.0660P1 data from the one or more sensor devices (e.g., the one or more sensor devices 103) and the image sequence data from the imaging device (e.g., the imaging device 102) as the imaging device traverses the location on the individual (e.g., patient). The imaging device may comprise an ultrasound scanner. For example, the imaging device may comprise a probe comprising an ultrasound scanner. For example, the imaging device may comprise a 2D ultrasound scanner configured to output 2D B-mode (e.g., ultrasound) images. The one or more sensor devices may comprise one or more of an internal measurement unit (IMU), an electromagnetic sensor, an optical sensor, an airborne ultrasound sensor, a time of flight sensor, a light detection and ranging (LiDAR) sensor, a sonomicrometry sensor, a smartcable, a depth camera, or a tactile sensor. The image sequence data may comprise a series of one or more two-dimensional B-mode images of the interior portion of the individual. As an example, the imaging device may generate the image sequence data of an interior portion of the individual as a user of the imaging device performs one or more sweeping motions back and forth across the location on the individual. The interior portion of the individual may comprise one or more of one or more blood vessels, one or more organs, or one or more layers of tissue. The movement data may comprise positional data and motion data, wherein the positional data may comprise spatial data and orientation data. As an example, the one or more sensor devices may be affixed to one or more of the imaging device or the location on the individual. The one or more sensor devices may determine (e.g.. generate) the movement data as the user of the imaging device performs the one or more sweeping motions back and forth across the location on the individual.

[0083] In an example, the movement data may be received from the one or more sensor devices and the image sequence data may be received from the imaging device based on data indicative of a gesture movement associated with the imaging device. For example, a scanning process may be initiated and deactivated based on an initial movement of the imaging device and an ending movement of the imaging device, respectively. For example, the scanning process may be initiated based on an initial gesture movement signaling to the computing device the initiation of the scanning process. In addition, the scanning process may be deactivated based on an ending gesture movement signaling the computing device to end the scanning process. The gesture may comprise a movement of the imaging device in a designated way (e.g., a movement distinct from the sweeping motions used during the scanning process). For example, the movement may compriseATTORNEY DOCKET NO.: 37759.0660P1 linear and / or angular reciprocation, vibration, or other velocity-controlled motion. For example, the gesture movement may comprise a pushing motion that is repeated to compress tissue (e.g., of the individual) several times. The gesture movement may be captured by the sensor device(s), wherein the computing device may identify the gesture movement as a movement for initiating the scanning process. In an example, ultrasound speckle tracking may be utilized to identify the bulk, in-plane motion created by the gesture movement.

[0084] At step 1104, the image sequence data may be combined with the movement data based on synchronizing the imaging device with the one or more sensor devices. For example, the computing device (e.g., computing device 101) may combine the image sequence data with the movement data based on synchronizing the imaging device (e g., the imaging device 102) with the one or more sensor devices (e.g., the one or more sensor devices 103). As an example, the image sequence data combined with the subsequent movement data may be stored as a digital imaging and communications in medicine (DICOM) file. In an example, the subsequent movement data may be stored in a private filed of the DICOM file. As an example, the imaging device may be synchronized with the one or more sensor devices by aligning the image sequence data output by the imaging device with the movement data output by the one or more sensor devices. As an example, based on the data indicative of the gesture movement associated with the imaging device, the imaging device may synchronize with the one or more sensor devices.

[0085] At step 1106. a vessel distribution map (VDM) may be generated based on the image sequence data combined with the movement data. For example, the computing device (e.g., computing device 101) may generate the VDM based on the image sequence data combined with the movement data.

[0086] At step 1108, one or more blood vessel locations within an interior portion of the individual may be identified in the image sequence data based on the vessel distribution map. For example, the computing device (e.g., computing device 101) may identify the one or more blood vessel locations within the interior portion of the individual in the image sequence data based on the vessel distribution map. For example, the VDM may be used to identify a high probability of the blood vessels within the image sequence data. As an example, the VDM may be represented as a 3D matrix (array) containing values corresponding to the probability a vessel is located at a locationATTORNEY DOCKET NO.: 37759.0660P1 within the array. As an example, grayscale data (e.g., backscatter intensity data) in addition to the image sequence data may contain data from both lumen and non-lumen structures, where lumen locations within the image sequence data may be a different intensity than non-lumen structures. As an example, image segmentation may be used to separate potential lumen from non-lumen structures in the grayscale data. As an example, a rule or set of rules where the VDM values derived from the movement data may be used to select probable lumen segments from the grayscale data based on lumen segments’ relative locations (e.g., nearness) to each other or their coincident location. In an example, the VDM may be processed to generate a final VDM. For example, a series of filters (e.g., morphological filtering, media filters, etc.) may be applied to the VDM, selecting subsets of VDM data that have certain shape attributes (e.g., a minimum number of pixels along a path of vessel tracking and a maximum shift in location perpendicular to the tracking path). The one or more blood vessel locations may be identified within an interior portion of the individual in the image sequence data based on the VDM.

[0087] At step 1110, a three-dimensional representation of the interior portion of the individual may be generated based on the identification of the one or more blood vessel locations and the image sequence data combined with the movement data. For example, the computing device (e.g.. computing device 101) may generate the three-dimensional representation of the interior portion of the individual based on the identification of the one or more blood vessel locations and the image sequence data combined with the movement data. The three-dimensional representation of the interior portion of the individual may comprise a three-dimensional reconstruction of the one or more two- dimensional B-mode images of the interior portion of the individual based on the subsequent movement data. In an example, the three-dimensional representation of the interior portion of the individual may be displayed (e.g., by the computing device). In an example, the three-dimensional representation of the interior portion of the individual may be output to a second computing device, wherein the second computing device may display the three-dimensional representation of the interior portion of the individual.

[0088] In an example, a visual interface comprising the three-dimensional representation of the interior portion of the individual overlaid onto the image sequence data may be generated. In an example, the visual interface comprising the three- dimensional representation of the interior portion of the individual overlaid onto theATTORNEY DOCKET NO.: 37759.0660P1 image sequence data may be displayed (e.g., by the computing device). In an example, the visual interface comprising the three-dimensional representation of the interior portion of the individual overlaid onto the image sequence data may be output to a second computing device, wherein the second computing device may display the visual interface. The visual interface may comprise a virtual reality' interface or an augmented reality interface.

[0089] As an example, the second computing device may comprise a head-mounted display (HMD). The HMD may be configured to display the location of the individual’s body as a user of the HMD and the imaging device images the location of the individual’s body. The HMD may display the visual interface comprising the three- dimensional representation of the interior portion of the individual overlaid onto the image sequence data in real-time to provide a real-time view of the location of the individual's body overlaid with the three-dimensional representation of the interior portion of the individual as the user of the HMD and the imaging device performs the one or more sweeping motions back and forth across the location on the individual.

[0090] FIG. 12 shows a flowchart of an example method 1200 for generating a three- dimensional (3D) representation of an interior portion of an individual based on image sequence data and movement data. Method 900 may be implemented by a computing device 101, an imaging device 102, one or more sensor devices 103, an electronic device 104, a server 106, or any combination thereof. At step 1202, movement data may be received from one or more sensor devices and image sequence data may be received from an imaging device as the imaging device traverses a location on an individual. For example, a computing device (e.g., computing device 101) may receive the movement data from the one or more sensor devices (e.g., the one or more sensor devices 103) and the image sequence data from the imaging device (e.g., the imaging device 102) as the imaging device traverses the location on the individual (e.g.. patient). The imaging device may comprise an ultrasound scanner. For example, the imaging device may comprise a probe comprising an ultrasound scanner. For example, the imaging device may comprise a 2D ultrasound scanner configured to output 2D B-mode (e.g., ultrasound) images. The one or more sensor devices may comprise one or more of an internal measurement unit (IMU), an electromagnetic sensor, an optical sensor, an airborne ultrasound sensor, a time of flight sensor, a light detection and ranging (LiDAR) sensor, a sonomicrometry sensor, a smartcable, a depth camera, or a tactile sensor. TheATTORNEY DOCKET NO.: 37759.0660P1 image sequence data may comprise a series of one or more two-dimensional B-mode images of the interior portion of the individual. As an example, the imaging device may generate the image sequence data of an interior portion of the individual as a user of the imaging device performs one or more sweeping motions back and forth across the location on the individual. The interior portion of the individual may comprise one or more of one or more blood vessels, one or more organs, or one or more layers of tissue. The movement data may comprise positional data and motion data, wherein the positional data may comprise spatial data and orientation data. As an example, the one or more sensor devices may be affixed to one or more of the imaging device or the location on the individual. The one or more sensor devices may determine (e.g., generate) the movement data as the user of the imaging device performs the one or more sweeping motions back and forth across the location on the individual.

[0091] In an example, the movement data may be received from the one or more sensor devices and the image sequence data may be received from the imaging device based on data indicative of a gesture movement associated with the imaging device. For example, a scanning process may be initiated and deactivated based on an initial movement of the imaging device and an ending movement of the imaging device, respectively. For example, the scanning process may be initiated based on an initial gesture movement signaling to the computing device the initiation of the scanning process. In addition, the scanning process may be deactivated based on an ending gesture movement signaling the computing device to end the scanning process. The gesture may comprise a movement of the imaging device in a designated way (e.g., a movement distinct from the sweeping motions used during the scanning process). For example, the movement may comprise linear and / or angular reciprocation, vibration, or other velocity-controlled motion. For example, the gesture movement may comprise a pushing motion that is repeated to compress tissue (e.g., of the individual) several times. The gesture movement may be captured by the sensor device(s). wherein the computing device may identify the gesture movement as a movement for initiating the scanning process. In an example, ultrasound speckle tracking may be utilized to identify the bulk, in-plane motion created by the gesture movement.

[0092] At step 1204, a three-dimensional representation of an interior portion of the individual may be generated based on combining the image sequence data with the movement data. For example, the computing device (e.g., computing device 101) mayATTORNEY DOCKET NO.: 37759.0660P1 generate the three-dimensional representation of the interior portion of the individual based on combining the image sequence data with the movement data. The three- dimensional representation of the interior portion of the individual may comprise a three- dimensional reconstruction of the one or more two-dimensional B-mode images of the interior portion of the individual based on the movement data. As an example, the image sequence data may be combined with the movement data based on synchronizing the imaging device with the one or more sensor devices. For example, imaging device may be synchronized with the one or more sensor devices by aligning the image sequence data output by the imaging device with the subsequent movement data output by the one or more sensor devices. As an example, based on the data indicative of the gesture movement associated with the imaging device, the imaging device may synchronize with the one or more sensor devices. As an example, the image sequence data combined with the subsequent movement data may be stored as a digital imaging and communications in medicine (DICOM) file. In an example, the subsequent movement data may be stored in a private filed of the DICOM file.

[0093] As an example, a vessel distribution map (VDM) may be generated based on the image sequence data combined with the movement data. For example, the VDM may be used to identify a high probability of the blood vessels within the image sequence data. As an example, the VDM may be represented as a 3D matrix (array) containing values corresponding to the probability a vessel is located at a location within the array. As an example, grayscale data (e.g., backscatter intensity data) in addition to the image sequence data may contain data from both lumen and non-lumen structures, where lumen locations within the image sequence data may be a different intensity than non-lumen structures. As an example, image segmentation may be used to separate potential lumen from non-lumen structures in the grayscale data. As an example, a rule or set of rules where the VDM values derived from the movement data may be used to select probable lumen segments from the grayscale data based on lumen segments’ relative location (e.g., nearness) to each other or their coincident location. In an example, the VDM may be processed to generate a final VDM. For example, a series of filters (e.g., morphological filtering, media filters, etc.) may be applied to the VDM, selecting subsets of VDM data that have certain shape attributes (e.g., minimum number of pixels along a path of vessel tracking and a maximum shift in location perpendicular to the trackingATTORNEY DOCKET NO.: 37759.0660P1 path). The one or more blood vessel locations may be identified within an interior portion of the individual in the image sequence data based on the VDM.

[0094] In an example, the three-dimensional representation of the interior portion of the individual may be output to a second computing device, wherein the second computing device may display the three-dimensional representation of the interior portion of the individual.

[0095] In an example, a visual interface comprising the three-dimensional representation of the interior portion of the individual overlaid onto the image sequence data may be generated. In an example, the visual interface comprising the three- dimensional representation of the interior portion of the individual overlaid onto the image sequence data may be displayed (e.g., by the computing device). In an example, the visual interface comprising the three-dimensional representation of the interior portion of the individual overlaid onto the image sequence data may be output to a second computing device, wherein the second computing device may display the visual interface. The visual interface may comprise a virtual reality7interface or an augmented reality interface.

[0096] As an example, the second computing device may comprise a head-mounted display (HMD). The HMD may be configured to display the location of the individual’s body as a user of the HMD and the imaging device images the location of the individual’s body. The HMD may display the visual interface comprising the three- dimensional representation of the interior portion of the individual overlaid onto the image sequence data in real-time to provide a real-time view of the location of the individual's body overlaid with the three-dimensional representation of the interior portion of the individual as the user of the HMD and the imaging device performs the one or more sweeping motions back and forth across the location on the individual.

[0097] At step 1206, one or more measurements associated with the interior portion of the individual may be generated based on the three-dimensional representation of the interior portion of the individual. For example, the computing device (e.g., computing device 101) may generate the one or more measurements associated with the interior portion of the individual based on the three-dimensional representation of the interior portion of the individual.

[0098] At step 1208, one or more medical conditions associated with the individual may be detected based on the one or more measurements associated with the interiorATTORNEY DOCKET NO.: 37759.0660P1 portion of the individual. For example, the computing device (e.g., computing device 101) may detect the one or more medical conditions associated with the individual based on the one or more measurements associated with the interior portion of the individual. The one or more medical conditions may comprise one or more of stenosis, aneurisms, or abnormalities in one or more blood vessels of the individual.

[0099] At step 1210, one or more treatments may be administered to the individual based on the one or more medical conditions. For example, the computing device (e.g., computing device 101) may cause the one or more treatments to be administered to the individual based on the one or more medical conditions.

[0100] FIG. 13 shows a flowchart of an example method 1300 for generating a three- dimensional (3D) representation of an interior portion of an individual based on image sequence data and movement data. Method 1300 may be implemented by a computing device 101, an imaging device 102, one or more sensor devices 103, an electronic device 104, a server 106, or any combination thereof. At step 1302, movement data may be received from one or more sensor devices and image sequence data may be received from an imaging device as the imaging device traverses a location on an individual. For example, a computing device (e.g., computing device 101) may receive the movement data from the one or more sensor devices (e.g., the one or more sensor devices 103) and the image sequence data from the imaging device (e.g., the imaging device 102) as the imaging device traverses the location on the individual (e.g., patient). The imaging device may comprise an ultrasound scanner. For example, the imaging device may comprise a probe comprising an ultrasound scanner. For example, the imaging device may comprise a 2D ultrasound scanner configured to output 2D B-mode (e.g., ultrasound) images. The one or more sensor devices may comprise one or more of an internal measurement unit (IMU), an electromagnetic sensor, an optical sensor, an airborne ultrasound sensor, a time of flight sensor, a light detection and ranging (LiDAR) sensor, a sonomicrometry sensor, a smartcable, a depth camera, or a tactile sensor. The image sequence data may comprise a series of one or more two-dimensional B-mode images of the interior portion of the individual. As an example, the imaging device may generate the image sequence data of an interior portion of the individual as a user of the imaging device performs one or more sweeping motions back and forth across the location on the individual. The interior portion of the individual may comprise one or more of one or more blood vessels, one or more organs, or one or more layers of tissue.ATTORNEY DOCKET NO.: 37759.0660P1The movement data may comprise positional data and motion data, wherein the positional data may comprise spatial data and orientation data. As an example, the one or more sensor devices may be affixed to one or more of the imaging device or the location on the individual. The one or more sensor devices may determine (e.g., generate) the movement data as the user of the imaging device performs the one or more sweeping motions back and forth across the location on the individual.

[0101] In an example, the movement data may be received from the one or more sensor devices and the image sequence data may be received from the imaging device based on data indicative of a gesture movement associated with the imaging device. For example, a scanning process may be initiated and deactivated based on an initial movement of the imaging device and an ending movement of the imaging device, respectively. For example, the scanning process may be initiated based on an initial gesture movement signaling to the computing device the initiation of the scanning process. In addition, the scanning process may be deactivated based on an ending gesture movement signaling the computing device to end the scanning process. The gesture may comprise a movement of the imaging device in a designated way (e.g., a movement distinct from the sweeping motions used during the scanning process). For example, the movement may comprise linear and / or angular reciprocation, vibration, or other velocity-controlled motion. For example, the gesture movement may comprise a pushing motion that is repeated to compress tissue (e.g., of the individual) several times. The gesture movement may be captured by the sensor device(s), wherein the computing device may identify the gesture movement as a movement for initiating the scanning process. In an example, ultrasound speckle tracking may be utilized to identify the bulk, in-plane motion created by the gesture movement.

[0102] At step 1304, a three-dimensional representation of an interior portion of the individual may be generated based on combining the image sequence data with the movement data. For example, the computing device (e.g., computing device 101) may generate the three-dimensional representation of the interior portion of the individual based on combining the image sequence data with the movement data. The three- dimensional representation of the interior portion of the individual may comprise a three- dimensional reconstruction of the one or more two-dimensional B-mode images of the interior portion of the individual based on the movement data. As an example, the image sequence data may be combined with the movement data based on synchronizing theATTORNEY DOCKET NO.: 37759.0660P1 imaging device with the one or more sensor devices. For example, imaging device may be synchronized with the one or more sensor devices by aligning the image sequence data output by the imaging device with the subsequent movement data output by the one or more sensor devices. As an example, based on the data indicative of the gesture movement associated with the imaging device, the imaging device may synchronize with the one or more sensor devices. As an example, the image sequence data combined with the subsequent movement data may be stored as a digital imaging and communications in medicine (DICOM) file. In an example, the subsequent movement data may be stored in a private filed of the DICOM file.

[0103] As an example, a vessel distribution map (VDM) may be generated based on the image sequence data combined with the movement data. For example, the VDM may be used to identify a high probability of the blood vessels within the image sequence data. As an example, the VDM may be represented as a 3D matrix (array) containing values corresponding to the probability a vessel is located at a location within the array. As an example, grayscale data (e.g., backscatter intensity data) in addition to the image sequence data may contain data from both lumen and non-lumen structures, where lumen locations within the image sequence data may be a different intensity than non-lumen structures. As an example, image segmentation may be used to separate potential lumen from non-lumen structures in the grayscale data. As an example, a rule or set of rules where the VDM values derived from the movement data may be used to select probable lumen segments from the grayscale data based on lumen segments’ relative location (e.g., nearness) to each other or their coincident location. In an example, the VDM may be processed to generate a final VDM. For example, a series of filters (e.g., morphological filtering, media filters, etc.) may be applied to the VDM, selecting subsets of VDM data that have certain shape attributes (e.g., minimum number of pixels along a path of vessel tracking and a maximum shift in location perpendicular to the tracking path). The one or more blood vessel locations may be identified within an interior portion of the individual in the image sequence data based on the VDM.

[0104] At step 1306. a visual interface comprising the three-dimensional representation of the interior portion of the individual overlaid onto the image sequence data may be generated. For example, the computing device (e.g., computing device 101) may generate the visual interface comprising the three-dimensional representation of theATTORNEY DOCKET NO.: 37759.0660P1 interior portion of the individual overlaid onto the image sequence data may be generated.

[0105] At step 1308, the visual interface comprising the three-dimensional representation overlaid onto the image sequence data may be output. For example, the computing device (e.g.. computing device 101) may output the visual interface comprising the three-dimensional representation overlaid onto the image sequence data. For example, the visual interface comprising the three-dimensional representation of the interior portion of the individual overlaid onto the image sequence data may be displayed (e.g., by the computing device). For example, the visual interface comprising the three- dimensional representation of the interior portion of the individual overlaid onto the image sequence data may be output to a second computing device, wherein the second computing device may display the visual interface. The visual interface may comprise a virtual reality interface or an augmented reality interface. In an example, the three- dimensional representation of the interior portion of the individual may be output to a second computing device, wherein the second computing device may display the three- dimensional representation of the interior portion of the individual.

[0106] In an example, the second computing device may comprise a head-mounted display (HMD). The HMD may be configured to display the location of the individual’s body as a user of the HMD and the imaging device images the location of the individual's body. The HMD may display the visual interface comprising the three- dimensional representation of the interior portion of the individual overlaid onto the image sequence data in real-time to provide a real-time view of the location of the individual’s body overlaid with the three-dimensional representation of the interior portion of the individual as the user of the HMD and the imaging device performs the one or more sweeping motions back and forth across the location on the individual.

[0107] The methods and systems can employ artificial intelligence (Al) techniques such as machine learning and iterative learning. Examples of such techniques include, but are not limited to, expert systems, case based reasoning, Bayesian networks, behavior based Al. neural networks, fuzzy systems, evolutionary computation (e.g.. a genetic algorithms), swarm intelligence (e g., an ant algorithms), and hybrid intelligent systems (e.g., expert inference rules generated through a neural network or production rules from statistical learning).ATTORNEY DOCKET NO.: 37759.0660P1

[0108] While the methods and systems have been described in connection with preferred embodiments and specific examples, it is not intended that the scope be limited to the particular embodiments set forth, as the embodiments herein are intended in all respects to be illustrative rather than restrictive.

[0109] Unless otherwise expressly stated, it is in no way intended that any method set forth herein be construed as requiring that its steps be performed in a specific order. Accordingly, where a method claim does not actually recite an order to be followed by its steps or it is not otherwise specifically stated in the claims or descriptions that the steps are to be limited to a specific order, it is in no way intended that an order be inferred, in any respect. This holds for any possible non-express basis for interpretation, including: matters of logic with respect to arrangement of steps or operational flow; plain meaning derived from grammatical organization or punctuation; the number or type of embodiments described in the specification.

[0110] It will be apparent to those skilled in the art that various modifications and variations can be made without departing from the scope or spirit. Other embodiments will be apparent to those skilled in the art from consideration of the specification and practice disclosed herein. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit being indicated by the following claims.

Claims

ATTORNEY DOCKET NO.: 37759.0660P1CLAIMS1 . A method comprising: based on initial movement data associated with an imaging device, determining, by a computing device, subsequent movement data received from one or more sensor devices and image sequence data received from the imaging device as the imaging device traverses a location on an individual; causing, based on the subsequent movement data and the image sequence data, the imaging device to synchronize with the one or more sensor devices; combining, based on the sy nchronization of the imaging device with the one or more sensor devices, the image sequence data with the subsequent movement data; and generating, based on the image sequence data combined with the subsequent movement data, a three-dimensional representation of an interior portion of the individual.

2. The method of claim 1, wherein the initial movement data is associated with a gesture movement.

3. The method of claim 1, wherein the imaging device comprises an ultrasound scanner.

4. The method of claim 1, wherein the imaging device comprises a probe.

5. The method of claim 1, wherein the interior portion comprises one or more of one or more blood vessels, one or more organs, or one or more layers of tissue.

6. The method of claim 1, wherein the image sequence data comprises a series of one or more two-dimensional B-mode images of the interior portion of the individual.

7. The method of claim 6, wherein the three-dimensional representation of the interior portion of the individual comprises a three-dimensional reconstruction of the one or more two-dimensional B-mode images of the interior portion of the individual based on the subsequent movement data.ATTORNEY DOCKET NO.: 37759.0660P18. The method of claim 1, wherein the one or more sensor devices comprise one or more of an internal measurement unit (IMU), an electromagnetic sensor, an optical sensor, an airborne ultrasound sensor, a time of flight sensor, a light detection and ranging (LiDAR) sensor, a sonomicrometry sensor, a smartcable, a depth camera, or a tactile sensor.

9. The method of claim 1, wherein the one or more sensor devices are affixed to one or more of the imaging device or the location on the individual.

10. The method of claim 1, wherein one or more of the initial movement data or the subsequent movement data comprises positional data and motion data.1 1. The method of claim 10, wherein the positional data comprises spatial data and orientation data.

12. The method of claim 1, further comprising synchronizing the imaging device with the one or more sensor devices by aligning the image sequence data output by the imaging device with the subsequent movement data output by the one or more sensor devices.

13. The method of claim 1, further comprising: performing, based on the image sequence data, image segmentation and decorrelation processing; and identifying, based on the image segmentation and the decorrelation processing, one or more blood vessel locations of the interior portion of the individual.

14. The method of claim 13, wherein generating, based on the image sequence data combined with the subsequent movement data, the three-dimensional representation of the interior portion of the individual comprises generating, based on the image sequence data combined with the subsequent movement data and based on the identification of the one or more blood vessel locations, the three-dimensional representation of the interior portion of the individual.

15. The method of claim 1, further comprising storing the image sequence data combined with the subsequent movement data as a digital imaging and communications in medicine (DICOM) file.ATTORNEY DOCKET NO.: 37759.0660P116. The method of claim 15, wherein the subsequent movement data is stored in a private filed of the DICOM file.

17. The method of claim 1, further comprising: displaying the three-dimensional representation of the interior portion of the individual; or sending, to a second computing device, the three-dimensional representation of the interior portion of the individual, wherein the second computing device displays the three-dimensional representation of the interior portion of the individual.

18. The method of claim 1, further comprising generating a visual interface comprising the three-dimensional representation of the interior portion of the individual overlaid onto the image sequence data.

19. The method of claim 18, further comprising: displaying the visual interface comprising the three-dimensional representation of the interior portion of the individual overlaid onto the image sequence data; or sending, to a second computing device, the visual interface comprising the three- dimensional representation of the interior portion of the individual overlaid onto the image sequence data, wherein the second computing device displays the visual interface.

20. The method of claim 18, wherein the visual interface comprises a virtual reality interface or an augmented reality interface.

21. A method comprising: receiving, by a computing device, movement data from one or more sensor devices and image sequence data from an imaging device as the imaging device traverses a location on an individual; combining, based on synchronizing the imaging device with the one or more sensor devices, the image sequence data with the movement data;ATTORNEY DOCKET NO.: 37759.0660P1 generating, based on the image sequence data combined with the movement data, a vessel distribution map; identifying, in the image sequence data, based on the vessel distribution map, one or more blood vessel locations within an interior portion of the individual; and generating, based on the identification of the one or more blood vessel locations and the image sequence data combined with the movement data, a three- dimensional representation of the interior portion of the individual.

22. The method of claim 21, wherein the imaging device comprises an ultrasound scanner.

23. The method of claim 21, wherein the imaging device comprises a probe.

24. The method of claim 21, wherein the interior portion comprises one or more of one or more blood vessels, one or more organs, or one or more layers of tissue.

25. The method of claim 21, wherein the image sequence data comprises a series of one or more two-dimensional B-mode images of the interior portion of the individual.

26. The method of claim 25. wherein the three-dimensional representation of the interior portion of the individual comprises a three-dimensional reconstruction of the one or more two-dimensional B-mode images of the interior portion of the individual based on the movement data.

27. The method of claim 21, wherein the one or more sensor devices comprise one or more of an internal measurement unit (IMU), an electromagnetic sensor, an optical sensor, an airborne ultrasound sensor, a time of flight sensor, a light detection and ranging (LiDAR) sensor, a sonomicrometiy sensor, a smartcable, a depth camera, or a tactile sensor.

28. The method of claim 21, wherein the one or more sensor devices are affixed to one or more of the imaging device or the location on the individual.

29. The method of claim 21, wherein the movement data comprises positional data and motion data.ATTORNEY DOCKET NO.: 37759.0660P130. The method of claim 29, wherein the positional data comprises spatial data and orientation data.

31. The method of claim 21 , wherein receiving the movement data from the one or more sensor devices and the image sequence data from the imaging device as the imaging device traverses the location on the individual comprises based on data indicative of a gesture movement associated with the imaging device, receiving the movement data from the one or more sensor devices and the image sequence data from the imaging device as the imaging device traverses the location on the individual, wherein the method further comprises: synchronizing the imaging device with the one or more sensor devices by aligning the image sequence data output by the imaging device with the movement data output by the one or more sensor devices.

32. The method of claim 21, further comprising storing the image sequence data combined with the movement data as a digital imaging and communications in medicine (DICOM) file.

33. The method of claim 32, wherein the movement data is stored in a private filed of the DICOM file.

34. The method of claim 21 , further comprising: displaying the three-dimensional representation of the interior portion of the individual; or sending, to a second computing device, the three-dimensional representation of the interior portion of the individual, wherein the second computing device displays the three-dimensional representation of the interior portion of the individual.

35. The method of claim 21, further comprising generating a visual interface comprising the three-dimensional representation of the interior portion of the individual overlaid onto the image sequence data.ATTORNEY DOCKET NO.: 37759.0660P136. The method of claim 35, further comprising: displaying the visual interface comprising the three-dimensional representation of the interior portion of the individual overlaid onto the image sequence data; or sending, to a second computing device, the visual interface comprising the three- dimensional representation of the interior portion of the individual overlaid onto the image sequence data, wherein the second computing device displays the visual interface.

37. The method of claim 35, wherein the visual interface comprises a virtual reality interface or an augmented reality interface.

38. A method comprising: receiving, by a computing device, movement data from one or more sensor devices and image sequence data from an imaging device as the imaging device traverses a location on an individual; generating, based on combining the image sequence data with the movement data, a three-dimensional representation of an interior portion of the individual; generating, based on the three-dimensional representation of the interior portion of the individual, one or more measurements associated with the interior portion of the individual; detecting, based on the one or more measurements associated with the interior portion of the individual, one or more medical conditions associated with the individual; and causing, based on the one or more medical conditions, one or more treatments to be administered to the individual.

39. The method of claim 38, wherein the imaging device comprises an ultrasound scanner.

40. The method of claim 38, wherein the imaging device comprises a probe.

41. The method of claim 38, wherein the interior portion comprises one or more of one or more blood vessels, one or more organs, or one or more layers of tissue.ATTORNEY DOCKET NO.: 37759.0660P142. The method of claim 38, wherein the image sequence data comprises a series of one or more two-dimensional B-mode images of the interior portion of the individual.

43. The method of claim 42, wherein the three-dimensional representation of the interior portion of the individual comprises a three-dimensional reconstruction of the one or more two-dimensional B-mode images of the interior portion of the individual based on the movement data.

44. The method of claim 38, wherein the one or more sensor devices comprise one or more of an internal measurement unit (IMU), an electromagnetic sensor, an optical sensor, an airborne ultrasound sensor, a time of flight sensor, a light detection and ranging (LiDAR) sensor, a sonomicrometry sensor, a smartcable, a depth camera, or a tactile sensor.

45. The method of claim 38, wherein the one or more sensor devices are affixed to one or more of the imaging device or the location on the individual.

46. The method of claim 38, wherein the movement data comprises positional data and motion data.

47. The method of claim 46. wherein the positional data comprises spatial data and orientation data.

48. The method of claim 38, wherein receiving the movement data from the one or more sensor devices and the image sequence data from the imaging device as the imaging device traverses the location on the individual comprises based on data indicative of a gesture movement associated with the imaging device, receiving the movement data from the one or more sensor devices and the image sequence data from the imaging device as the imaging device traverses the location on the individual, wherein the method further comprises: synchronizing the imaging device with the one or more sensor devices by aligning the image sequence data output by the imaging device with the movement data output by the one or more sensor devices.

49. The method of claim 38, wherein the one or more medical conditions comprise one or more of stenosis, aneurisms, or abnormalities in one or more blood vessels of the individual.ATTORNEY DOCKET NO.: 37759.0660P150. The method of claim 38, further comprising storing the image sequence data combined with the movement data as a digital imaging and communications in medicine (DICOM) file.

51. The method of claim 50, wherein the movement data is stored in a private filed of the DICOM file.

52. The method of claim 38, further comprising: displaying the three-dimensional representation of the interior portion of the individual; or sending, to a second computing device, the three-dimensional representation of the interior portion of the individual, wherein the second computing device displays the three-dimensional representation of the interior portion of the individual.

53. The method of claim 38, further comprising generating a visual interface comprising the three-dimensional representation of the interior portion of the individual overlaid onto the image sequence data.

54. The method of claim 53, further comprising: displaying the visual interface comprising the three-dimensional representation of the interior portion of the individual overlaid onto the image sequence data; or sending, to a second computing device, the visual interface comprising the three- dimensional representation of the interior portion of the individual overlaid onto the image sequence data, wherein the second computing device displays the visual interface.

55. The method of claim 53. wherein the visual interface comprises a virtual reality interface or an augmented reality interface.ATTORNEY DOCKET NO.: 37759.0660P156. A method comprising: receiving, by a computing device, movement data from one or more sensor devices and image sequence data from an imaging device as the imaging device traverses a location on an individual; generating, based on combining the image sequence data with the movement data, a three-dimensional representation of an interior portion of the individual; generating a visual interface comprising the three-dimensional representation of the interior portion of the individual overlaid onto the image sequence data; and outputting the visual interface comprising the three-dimensional representation overlaid onto the image sequence data.

57. The method of claim 56, wherein the imaging device comprises an ultrasound scanner.

58. The method of claim 56, wherein the imaging device comprises a probe.

59. The method of claim 56, wherein the interior portion comprises one or more of one or more blood vessels, one or more organs, or one or more layers of tissue.

60. The method of claim 56. wherein the image sequence data comprises a series of one or more two-dimensional B-mode images of the interior portion of the individual.

61. The method of claim 60, wherein the three-dimensional representation of the interior portion of the individual comprises a three-dimensional reconstruction of the one or more two-dimensional B-mode images of the interior portion of the individual based on the movement data.

62. The method of claim 56, wherein the one or more sensor devices comprise one or more of an internal measurement unit (IMU), an electromagnetic sensor, an optical sensor, an airborne ultrasound sensor, a time of flight sensor, a light detection and ranging (LiDAR) sensor, a sonomicrometry sensor, a smartcable, a depth camera, or a tactile sensor.

63. The method of claim 56, wherein the one or more sensor devices are affixed to one or more of the imaging device or the location on the individual.ATTORNEY DOCKET NO.: 37759.0660P164. The method of claim 56, wherein the movement data comprises positional data and motion data.

65. The method of claim 64, wherein the positional data comprises spatial data and orientation data.

66. The method of claim 56, wherein receiving the movement data from the one or more sensor devices and the image sequence data from the imaging device as the imaging device traverses the location on the individual comprises based on data indicative of a gesture movement associated with the imaging device, receiving the movement data from the one or more sensor devices and the image sequence data from the imaging device as the imaging device traverses the location on the individual, wherein the method further comprises: synchronizing the imaging device with the one or more sensor devices by aligning the image sequence data output by the imaging device with the movement data output by the one or more sensor devices.

67. The method of claim 56, further comprising storing the image sequence data combined with the movement data as a digital imaging and communications in medicine (DICOM) file.

68. The method of claim 67, wherein the movement data is stored in a private filed of the DICOM file.

69. The method of claim 56, wherein outputting the visual interface comprising the three- dimensional representation overlaid onto the image sequence data comprises: displaying the visual interface comprising the three-dimensional representation of the interior portion of the individual overlaid onto the image sequence data; or sending, to a second computing device, the visual interface comprising the three- dimensional representation of the interior portion of the individual overlaid onto the image sequence data, wherein the second computing device displays the visual interface.ATTORNEY DOCKET NO.: 37759.0660P170. The method of claim 56, wherein the visual interface comprises a virtual realityinterface or an augmented reality- interface.

Images