Identification of anatomical scan window, probe orientation, and / or patient position based on ultrasound images

Abstract

An ultrasound imaging system includes a processor circuit that stores a target parameter representative of a target anatomical scan window in a memory in communication with the processor circuit. The processor circuit receives a first ultrasound image acquired during a first acquisition period by a first ultrasound probe having a first anatomical scan window. The processor circuit determines a first parameter representative of the first anatomical scan window. The processor circuit retrieves the target parameter from the memory. The processor circuit compares the target parameter to the first parameter. The processor circuit outputs a visual representation of the comparison to a display in communication with the processor circuit.

Description

Technical Field

[0001] This disclosure generally relates to ultrasound imaging. For example, an ultrasound system can estimate the anatomical scanning window, ultrasound probe orientation, and / or patient position for one ultrasound image, compare these parameters with those parameters for another ultrasound image, and provide corresponding guidance to match, for example, the parameters for said image. Background Technology

[0002] Medical ultrasound systems may include ultrasound transducer probes coupled to a processing system and one or more display devices. Ultrasound imaging systems are widely used in medical imaging and measurement. For example, ultrasound imaging systems can be used to measure organs, lesions, tumors, or other structures within a patient's anatomy. A physician may want to perform longitudinal studies on any of these structures. Longitudinal studies may require multiple imaging sessions of the patient over a long period. In this way, the physician can track changes in the patient's anatomy over time.

[0003] Ultrasound imaging systems are generally underutilized in longitudinal studies due to the challenges of reproducible ultrasound image acquisition. Specifically, reliably reproducing two-dimensional (2D) ultrasound images is challenging due to the manual nature of image acquisition, including variations in the positioning of both the ultrasound probe and the patient. These variations make it difficult to determine whether changes in the anatomy in a longitudinal study are due to alterations in the patient's body or to ultrasound imaging itself. These variations are also often not recorded from one ultrasound imaging procedure to the next, as they are not easily captured. Furthermore, physiological parameters such as heart rate, respiratory rate, and tissue morphology can make it difficult to identify previously used imaging conditions. For contrast-enhanced ultrasound (CEUS), an imaging technique involving the injection of contrast agents, the timing between injection and imaging presents additional challenges. All of these factors contribute to increased variability in ultrasound-based measurements, specifically for images acquired at different time points. It is also difficult to determine whether differences between images or measurements are due to physiological changes within the patient's anatomy or to variations in the image acquisition technique. However, despite these challenges, ultrasound imaging remains a cheaper and safer imaging method than the more commonly used methods in longitudinal studies. Summary of the Invention

[0004] Embodiments of this disclosure are systems, apparatus, and methods for automatically identifying anatomical scanning windows, ultrasound probe orientation, and / or patient position associated with ultrasound images using deep learning networks. By identifying and recording one or more of these parameters for each ultrasound image acquired during a longitudinal study, they can be retrieved in subsequent imaging processes and used to assist sonographers, physicians, and / or other users in obtaining additional ultrasound images with the same parameters. This advantageously reduces variability in image acquisition and ultrasound measurements between imaging procedures and can help make ultrasound imaging a more feasible, cost-effective, and safer option for longitudinal studies.

[0005] In an imaging process, an ultrasound imaging system can acquire images. A deep learning network can then receive these images as input. The network can also receive inertial measurement unit (IMU) data from the ultrasound probe as additional input. The network can identify the anatomical scanning window and / or probe orientation used to acquire the ultrasound images based on the characteristics of the images themselves. The network can also use the IMU data to determine the patient's position at the time the images were acquired. These parameters can then be stored along with the ultrasound images.

[0006] In subsequent imaging sessions, the ultrasound imaging system can retrieve the same ultrasound images from previous sessions, along with the anatomical scan window, probe orientation, and / or patient position for those images. The ultrasound system outputs the retrieved scan window, probe orientation, patient position, and / or earlier images to, for example, a display to provide guidance to the user (e.g., an ultrasound physician or doctor). The user can then use this guidance to position the patient and ultrasound probe during subsequent imaging sessions. Furthermore, a deep learning network can receive this new image and determine the scan window, probe orientation, and / or patient position for the new image. These new parameters can be compared to those from previous sessions. The ultrasound system can then indicate whether the parameters match or differ. If the parameters differ, the ultrasound physician can be prompted to adjust the patient or probe position and acquire additional images. This process can continue until all parameters match. When the parameters match, the user is able to record images in the new session. In this way, longitudinal studies can be performed more accurately, minimizing or eliminating variations caused by probe or patient positioning. More precisely, changes in ultrasound images from different imaging sessions are due to changes in the patient's body (e.g., disease progression, treatment effectiveness, etc.).

[0007] In an exemplary aspect, an ultrasound imaging system includes: a processor circuit configured to: store in a memory communicating with the processor circuit target parameters representing (i) a target anatomical scanning window and / or probe orientation and (ii) a patient position; receive a first ultrasound image acquired by a first ultrasound probe having a first anatomical scanning window during a first acquisition period; determine a first parameter representing (i) the first anatomical scanning window and / or probe orientation and (ii) the patient position; retrieve the target parameter from the memory; compare the target parameter and the first parameter; and output a visual representation of the comparison to a display communicating with the processor circuit.

[0008] In some aspects, the target parameters include (i) the anatomical scanning window and / or probe orientation and (ii) the patient position of a second ultrasound probe for a second ultrasound image acquired during a second acquisition period prior to a first acquisition period. In some aspects, the processor circuitry is configured to: store the second ultrasound image in a memory such that the target parameters are associated with the second ultrasound image in the memory. In some aspects, the processor circuitry is configured to: receive the second ultrasound image acquired by the second ultrasound probe; determine the target parameters representing (i) the second anatomical scanning window and / or probe orientation and (ii) the patient position; and associate the target parameters with the second ultrasound image. In some aspects, the processor circuitry is configured to: associate a first parameter with a first ultrasound image; store the first parameter and the first ultrasound image in a memory such that the first parameter and the first ultrasound image are associated in the memory; and retrieve the first parameter from the memory for comparison with other parameters corresponding to other ultrasound images. In some aspects, the processor circuitry is configured to communicate with a first ultrasound probe, and wherein the processor circuitry is configured to: control the first ultrasound probe to acquire a first ultrasound image; and, during the acquisition of the first ultrasound image, output a screen display to a display including at least one of a second ultrasound image or a target parameter. In some aspects, the processor circuitry is configured to output a screen display to a display including a simultaneously displayed target parameter, a first parameter, a first ultrasound image, a second ultrasound image, and a comparative visual representation. In some aspects, the first parameter represents a first orientation of the first ultrasound probe during a first acquisition period, and wherein the target parameter represents a second orientation of the second ultrasound probe during a second acquisition period.

[0009] In some aspects, the system further includes a first ultrasound probe, and the first ultrasound probe includes an inertial measurement unit, wherein the processor circuitry is configured to determine a first parameter based on data obtained by the inertial measurement unit. In some aspects, the first parameter includes a continuous variable, and wherein the processor circuitry is configured to output a visual representation of the continuous variable to a display. In some aspects, the first parameter represents the patient's position during a first acquisition period. In some aspects, the processor circuitry is configured to: receive user input selecting a target anatomical scan window; and determine a target parameter based on the user input. In some aspects, the processor circuitry includes a preprocessor and at least one deep learning network. In some aspects, the preprocessor is configured to: receive a plurality of ultrasound images acquired by the first ultrasound probe during the first acquisition period, wherein the plurality of ultrasound images includes the first ultrasound image; cache the plurality of ultrasound images in a memory; filter the plurality of ultrasound images; and output a subset of the plurality of ultrasound images to the deep learning network. In some aspects, for filtering multiple ultrasound images, a preprocessor is configured to: determine a similarity metric for multiple ultrasound images cached in memory; and remove a portion of the multiple ultrasound images from memory based on the similarity metric to generate a subset of the multiple ultrasound images. In some aspects, at least one deep learning network includes multiple deep learning networks respectively corresponding to multiple organs, and the preprocessor is configured to: identify organs in a first ultrasound image; and output the first ultrasound image to the deep learning network corresponding to the organ. In some aspects, at least one deep learning network includes a single deep learning network corresponding to multiple organs. In some aspects, the deep learning network is configured to determine a first parameter. In some aspects, the deep learning network includes a convolutional neural network (CNN). In some aspects, the CNN is trained using a dataset of ultrasound images associated with corresponding training parameters determined based on information from a tracking system. In some aspects, the visual representation of the comparison includes: a first indicator when the target parameter and the first parameter are the same; and a second indicator when the target parameter and the first parameter are different. In some aspects, the processor circuitry is configured to: store a second patient physiological condition during a second acquisition period in a memory; receive a first patient physiological condition during a first acquisition period; retrieve the second patient physiological condition from the memory; compare the second patient physiological condition with the first patient physiological condition; and output a visual representation of the comparison between the second patient physiological condition and the first patient physiological condition to a display.

[0010] In an exemplary aspect, an ultrasound imaging system includes: a processor circuit configured to: store target parameters representing (i) a target anatomical scanning window and / or a target probe orientation and (ii) a patient position in a memory in communication with the processor circuit; receive ultrasound images acquired by an ultrasound probe having (i) the anatomical scanning window and / or the probe orientation and (ii) the patient position during an acquisition period; determine parameters representing the anatomical scanning window and / or the probe orientation and (ii) the patient position using a convolutional neural network; retrieve the target parameters from the memory; compare the target parameters with the parameters; and output a visual representation of the comparison to a display in communication with the processor circuit, wherein the visual representation of the comparison includes: a first indicator when the target parameters and the parameters are the same; and a second indicator when the target parameters and the parameters are different.

[0011] Additional aspects, features, and advantages of this disclosure will become apparent from the following detailed description. Attached Figure Description

[0012] Illustrative embodiments of this disclosure will be described with reference to the accompanying drawings, in which: Figure 1 This is a schematic diagram of an ultrasound imaging system according to various aspects of this disclosure; Figure 2 This is a schematic diagram of the processor circuit according to various aspects of this disclosure; Figure 3 This is a flowchart of a method for determining the anatomical scanning window and / or probe orientation associated with an ultrasound image, according to various aspects of this disclosure; Figure 4 This is a schematic diagram of an ultrasound imaging system with a deep learning network according to various aspects of this disclosure; Figure 5 This is a schematic diagram of a convolutional neural network (CNN) configuration according to various aspects of this disclosure; Figure 6 This is a flowchart of a method for analyzing ultrasound images from an imaging process and a flowchart of a method for analyzing and comparing ultrasound images from an additional imaging process, according to various aspects of this disclosure. Figure 7 This is a graphical view of a graphical user interface of an ultrasound imaging system for comparing parameters of ultrasound images before acquiring images during additional imaging processes, according to various aspects of this disclosure. Figure 8 It is a graphical view of a graphical user interface of an ultrasound imaging system for comparing parameters of ultrasound images and indicating differences between parameters, according to various aspects of this disclosure. Figure 9 It is a graphical view of a graphical user interface of an ultrasound imaging system for comparing parameters of ultrasound images and indicating differences between parameters, according to various aspects of this disclosure. Figure 10 It is a graphical view of a graphical user interface for an ultrasound imaging system that compares ultrasound images and indicates that the parameters are the same, based on various aspects of this disclosure. Figure 11A This is a graphical view of a graphical user interface for annotating ultrasound images during the training phase of a deep learning network, according to various aspects of this disclosure. Figure 11B This is a graphical view of a graphical user interface for annotating ultrasound images during the training phase of a deep learning network, according to various aspects of this disclosure. Figure 12 This is a graphical view of a computed tomography and ultrasound system for automatically annotating images during the training phase of a deep learning network, according to various aspects of this disclosure. Figure 13 This is a schematic diagram of a CNN configuration including inertial measurement unit (IMU) data according to various aspects of this disclosure; Figure 14 This is a schematic diagram of a CNN configuration that includes IMU data and outputs additional labels according to various aspects of this disclosure; Figure 15 It is a diagrammatic view of the patient's position based on various aspects of this disclosure; Figure 16 These are schematic views of ultrasound probes in various orientations according to various aspects of this disclosure. Detailed Implementation

[0013] To facilitate understanding of the principles of this disclosure, reference will now be made to embodiments illustrated in the accompanying drawings, and these embodiments will be described using specific language. However, it should be understood that this is not intended to limit the scope of this disclosure. Any changes and other modifications to the described devices, systems, and methods, as well as any other application of the principles of this disclosure, are fully considered and included within this disclosure, as would normally occur to those skilled in the art to which this disclosure pertains. For example, while the focusing system is described in relation to cardiovascular imaging, it should be understood that it is not intended to be limited to that application. The system is equally well-suited for any application requiring imaging within a confined cavity. Specifically, it is entirely conceivable that features, components, and / or steps described with respect to one embodiment may be combined with features, components, and / or steps described with respect to other embodiments of this disclosure. However, for the sake of brevity, multiple iterations of these combinations will not be described separately.

[0014] Figure 1This is a schematic diagram of an ultrasound imaging system 100 according to various aspects of this disclosure. System 100 is used to scan areas or volumes of a patient's body. System 100 includes an ultrasound imaging probe 110 that communicates with a host 130 via a communication interface or link 120. Probe 110 may include a transducer array 112, a beamformer 114, processor circuitry 116, and a communication interface 118. Host 130 may include a display 132, processor circuitry 134, a communication interface 136, and a memory 138 for storing patient information. Host 130 and / or its processor 134 may additionally communicate with memory 140 and patient sensor 142.

[0015] In some embodiments, probe 110 is an external ultrasound imaging device, including a housing 111 configured for hand-held operation by a user. Transducer array 112 can be configured to acquire ultrasound data when the user grips the housing 111 of probe 110, positioning the transducer array 112 adjacent to or in contact with the patient's skin. Probe 110 is configured to acquire ultrasound data of the anatomy inside the patient's body while probe 110 is positioned externally to the patient's body. In some embodiments, probe 110 can be an external ultrasound probe and / or a transthoracic echocardiography (TTE) probe.

[0016] In other embodiments, probe 110 can be an internal ultrasound imaging device and may include a housing 111 configured to be positioned within a lumen of a patient's body, including the patient's coronary vascular system, peripheral vascular system, esophagus, ventricle, or other body lumen or cavity. In some embodiments, probe 110 can be an intravascular ultrasound (IVUS) imaging catheter or an intracardiac echocardiography (ICE) catheter. In other embodiments, probe 110 can be a transesophageal echocardiography (TEE) probe. Probe 110 can be any suitable form for any suitable ultrasound imaging application including external and internal ultrasound imaging.

[0017] In some embodiments, aspects of this disclosure can be implemented using medical images of a patient acquired using any suitable medical imaging device and / or modality. Examples of medical images and medical imaging devices include X-ray images (angiographic images, fluoroscopic images, images with or without angiography) acquired by X-ray imaging devices, computed tomography (CT) images acquired by CT imaging devices, positron emission tomography-computed tomography (PET-CT) images acquired by PET-CT imaging devices, magnetic resonance imaging (MRI) images acquired by MRI devices, single-photon emission computed tomography (SPECT) images acquired by SPECT imaging devices, optical coherence tomography (OCT) images acquired by OCT imaging devices, and intravascular photoacoustic (IVPA) images acquired by IVPA imaging devices. Medical imaging devices are capable of acquiring medical images when positioned outside the patient's body, spaced apart from the patient's body, adjacent to the patient's body, in contact with the patient's body, and / or inside the patient's body.

[0018] For an ultrasound imaging device, transducer array 112 transmits ultrasound signals to the anatomical object 105 of the patient and receives echo signals reflected back to transducer array 112 from the object 105. The ultrasound transducer array 112 can include any suitable number of acoustic elements, including one or more acoustic elements and / or multiple acoustic elements. In some cases, transducer array 112 includes a single acoustic element. In some cases, transducer array 112 can include an array of acoustic elements with any number of acoustic elements having any suitable configuration. For example, transducer array 112 can include from 1 acoustic element to 10,000 acoustic elements, including values ​​such as 2 acoustic elements, 4 acoustic elements, 36 acoustic elements, 64 acoustic elements, 128 acoustic elements, 500 acoustic elements, 812 acoustic elements, 1,000 acoustic elements, 3,000 acoustic elements, 8,000 acoustic elements, and / or other values ​​larger and smaller. In some cases, transducer array 112 may comprise an array of acoustic elements having any number of acoustic elements in any suitable configuration, such as linear arrays, planar arrays, curved arrays, wavy arrays, circumferential arrays, ring arrays, phased arrays, matrix arrays, one-dimensional (1D) arrays, 1.5D arrays (e.g., 1.5D arrays), or two-dimensional (2D) arrays. The array of acoustic elements (e.g., one or more rows, one or more columns, and / or one or more orientations) can be uniformly or independently controlled and activated. Transducer array 112 can be configured to acquire one-dimensional, two-dimensional, and / or three-dimensional images of a patient's anatomy. In some embodiments, transducer array 112 may comprise a piezoelectric micromachining ultrasonic transducer (PMUT), a capacitive micromachining ultrasonic transducer (CMUT), a single crystal, lead zirconate titanate (PZT), PZT composite materials, other suitable transducer types, and / or combinations thereof.

[0019] Object 105 may include any anatomical body or feature, such as blood vessels, nerve fibers, airways, mitral valves, cardiac structures, abdominal tissue structures, appendix, large intestine (or colon), small intestine, kidneys, liver, and / or any other anatomical body of the patient. In some aspects, object 105 may include at least a portion of the patient's large intestine, small intestine, cecal pouch, appendix, terminal ileum, liver, upper abdomen, and / or psoas major muscle. This disclosure can be implemented in the context of any number of anatomical locations and tissue types, including, but not limited to, organs including the liver, heart, kidneys, gallbladder, pancreas, and lungs; ducts; intestines; nervous system structures including the brain, dural sac, spinal cord, and peripheral nerves; urinary tract; and valves of blood vessels, blood, chambers or other parts of the heart, abdominal organs, and / or other systems of the body. In some embodiments, object 105 may include malignant tumors, such as tumors, cysts, lesions, hemorrhages, or blood pools within any part of the human anatomy. The anatomical body can be a blood vessel, such as an artery or vein in the patient's vascular system, including the cardiac vascular system, peripheral vascular system, neural vascular system, renal vascular system, and / or any other suitable lumen within the body. In addition to natural structures, this disclosure can also be implemented in the context of artificial structures, such as, but not limited to, heart valves, stents, shunts, filters, implants, and other devices.

[0020] Beamformer 114 is coupled to transducer array 112. Beamformer 114 controls transducer array 112, for example, for transmitting ultrasonic signals and receiving ultrasonic echo signals. In some embodiments, beamformer 114 may apply a time delay to signals sent to individual acoustic transducers within the array in transducer 112, such that the acoustic signals are guided in any suitable direction away from probe 110. Beamformer 114 may also provide image signals to processor circuitry 116 based on the response to received ultrasonic echo signals. Beamformer 114 may include multiple stages of beamforming. Beamforming can reduce the number of signal lines used for coupling to processor circuitry 116. In some embodiments, the combination of transducer array 112 and beamformer 114 may be referred to as an ultrasound imaging assembly.

[0021] Processor 116 is coupled to beamformer 114. Processor 116 may also be described as processor circuitry capable of including other components communicating with processor 116, such as memory, beamformer 114, communication interface 118, and / or other suitable components. Processor 116 may include a central processing unit (CPU), graphics processing unit (GPU), digital signal processor (DSP), application-specific integrated circuit (ASIC), controller, field-programmable gate array (FPGA) device, another hardware device, firmware device, or any combination thereof configured to perform the operations described herein. Processor 116 may also be implemented as a combination of computing devices, such as a combination of a DSP and a microprocessor, multiple microprocessors, one or more microprocessors combined with a DSP core, or any other such configuration. Processor 116 is configured to process beamformed image signals. For example, processor 116 may perform filtering and / or quadrature demodulation to modulate the image signals. Processor 116 and / or 134 may be configured to control array 112 to obtain ultrasound data associated with object 105.

[0022] The probe 110 may include an inertial measurement unit (IMU) 117, which is an electronic device that generates IMU data (e.g., specific force, angular rate, orientation, appropriate acceleration, angular velocity, etc.). The IMU 117 may include one or more accelerometers, gyroscopes, and / or magnetometers disposed within the housing 111 of the probe 110. The IMU data can represent the probe 110 during operation to acquire ultrasound images.

[0023] Communication interface 118 is coupled to processor 116. Communication interface 118 may include one or more transmitters, one or more receivers, one or more transceivers, and / or circuitry for transmitting and / or receiving communication signals. Communication interface 118 may include hardware and / or software components implementing a specific communication protocol suitable for transmitting signals to host 130 via communication link 120. Communication interface 118 may be referred to as a communication device or communication interface module.

[0024] Communication link 120 can be any suitable communication link. For example, communication link 120 can be a wired link, such as a Universal Serial Bus (USB) link or an Ethernet link. Alternatively, communication link 120 can be a wireless link, such as an Ultra Wideband (UWB) link, an IEEE 802.11 WiFi link, or a Bluetooth link.

[0025] At host 130, communication interface 136 can receive image signals. Communication interface 136 can be substantially similar to communication interface 118. Host 130 can be any suitable computing and display device, such as a workstation, personal computer (PC), laptop, tablet, or mobile phone.

[0026] Processor 134 is coupled to communication interface 136. Processor 134 may also be described as processor circuitry capable of including other components communicating with processor 134, such as memory 138, communication interface 136, and / or other suitable components. Processor 134 may be implemented as a combination of software and hardware components. Processor 134 may include a central processing unit (CPU), graphics processing unit (GPU), digital signal processor (DSP), application-specific integrated circuit (ASIC), controller, FPGA device, another hardware device, firmware device, or any combination thereof configured to perform the operations described herein. Processor 134 may also be implemented as a combination of computing devices, such as a combination of a DSP and a microprocessor, multiple microprocessors, one or more microprocessors combined with a DSP core, or any other such configuration. Processor 134 can be configured to generate image data based on image signals received from probe 110. Processor 134 can apply advanced signal processing and / or image processing techniques to the image signals. In some embodiments, processor 134 is capable of forming a three-dimensional (3D) volumetric image from the image data. In some embodiments, the processor 134 is capable of performing real-time processing on image data to provide a streaming video of an ultrasound image of the object 105.

[0027] Memory 138 is coupled to processor 134. Memory 138 can be any suitable storage device, such as a cache (e.g., a cache of processor 134), random access memory (RAM), magnetoresistive RAM (MRAM), read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), flash memory, solid-state storage device, hard disk drive, solid-state drive, other forms of volatile and non-volatile memory, or a combination of different types of memory.

[0028] Memory 138 can be configured to store patient information, measurements, data, or documents related to patient medical history, the history of procedures performed, anatomical or biometric features, characteristics, or medical conditions associated with the patient, computer-readable instructions (such as code, software, or other applications), and any other suitable information or data. Memory 138 may be located within host 130. Patient information may include measurements, data, documents, other forms of medical history, such as, but not limited to, ultrasound images, ultrasound videos, and / or any imaging information related to the patient's anatomy. Patient information may include parameters related to the imaging process, such as anatomical scanning window, probe orientation, and / or the patient's position during the imaging process. The patient may include data, images, measurements, or other information related to other imaging modalities (such as CT imaging). Memory 138 can also be configured to store information related to the training and implementation of deep learning networks (e.g., neural networks). Mechanisms for training and implementing deep learning networks are described in more detail herein.

[0029] Any or all of the aforementioned computer-readable media, such as patient information, codes, software or other applications, or any other suitable information or data, may also be stored in memory 140. Memory 140 may be used for purposes substantially similar to memory 138, but may not be located within host 130. For example, in some embodiments, memory may be a cloud-based server, an external storage device, or any other device for memory storage. Host 130 may communicate with memory 140 in any suitable manner described. Host 130 may communicate continuously with memory 140, or they may communicate intermittently upon request from host 130 or the user of ultrasound system 100.

[0030] The processor 134 of the host 130 can also communicate with the patient sensor 142. The patient sensor 142 can monitor any suitable physiological characteristics of the patient imaged by the imaging system 100. For example, the patient sensor 142 can acquire data related to the patient's heart rate, respiratory rate, blood pressure, body temperature, or any other physiological measurement of the patient, or data related to the patient's positioning. Therefore, the patient sensor 142 can be a heart rate sensor, respiratory sensor, blood pressure sensor, temperature sensor, or orientation sensor, etc. This information can be received by the processor 134 of the host 130 and used to minimize its impact on the images or data acquired by the system 100. The host 130 can communicate with the patient sensor 142 in any suitable manner described. The host 130 can communicate continuously with the memory 142, or they can communicate intermittently upon request from the user of the host 130 or the ultrasound system 100.

[0031] The host 130 can communicate with the memory 140 and / or the patient sensor 142 via any suitable communication method. For example, the host 130 can communicate with the memory 140 and / or the patient sensor 142 via a wired link, such as a USB link or an Ethernet link. Alternatively, the host 130 can communicate with the memory 140 and / or the patient sensor 142 via a wireless link, such as a UWB link, an IEEE 802.11 WiFi link, or a Bluetooth link.

[0032] Display 132 is coupled to processor circuitry 134. Display 132 may be a monitor or any suitable display. Display 132 is configured to display ultrasound images, video images, and / or any imaging information of object 105.

[0033] System 100 can be used to assist an ultrasound physician in performing an ultrasound scan. The scan can be performed in a point-of-care setting. In some cases, the host unit 130 is a console or a mobile cart. In other cases, the host unit 130 can be a mobile device, such as a tablet, mobile phone, or laptop. During the imaging process, the ultrasound system is able to acquire ultrasound images of specific regions of interest within the patient's anatomy. The ultrasound system 100 can then analyze the ultrasound images to identify various parameters associated with the image acquisition, such as scan window, probe orientation, patient position, and / or other parameters. The system 100 can then store the images and these associated parameters in memory 138 and / or memory 140. In subsequent imaging processes, the system 100 can retrieve previously acquired ultrasound images and associated parameters to display to the user, which can be used to guide the user of the system 100 to use the same or similar parameters in subsequent imaging processes, as will be described in more detail below.

[0034] In some aspects, processor 134 may utilize a deep learning-based predictive network to identify parameters of the ultrasound image, including the anatomical scan window, probe orientation, patient position, and / or other parameters. In some aspects, processor 134 may receive measurements or perform various calculations related to the imaged region of interest or the patient's physiological state during the imaging process. These measurements and / or calculations may also be displayed to the sonographer or other user via display 132.

[0035] Figure 2 This is a schematic diagram of the processor circuit according to various aspects of this disclosure. The processor circuit 210 can be implemented in the probe 110. Figure 1The host system 130 or any other suitable location. One or more processor circuits can be configured to perform the operations described herein. Processor circuit 210 can be part of circuits 114 and / or 134, or it can be a separate circuit. In an example, processor circuit 210 can communicate with transducer array 112, circuit 114, communication interface 122, communication interface 140, circuit 134 and / or display 132, and any other suitable components or circuits within the ultrasound system 100. As shown, processor circuit 210 may include processor 260, memory 264, and communication module 268. These elements can communicate directly or indirectly with each other, for example, via one or more buses.

[0036] Processor 260 may include a CPU, GPU, DSP, application-specific integrated circuit (ASIC), controller, FPGA, another hardware device, firmware device, or any combination thereof configured to perform the operations described herein. Processor 260 may also be implemented as a combination of computing devices, such as a combination of a DSP and a microprocessor, multiple microprocessors, one or more microprocessors combined with a DSP core, or any other such configuration. Processor 260 may also include an analysis module, as will be discussed in more detail below. The analysis module may implement various deep learning networks and may be implemented in hardware or software. Processor 260 may additionally include a preprocessor in the hardware or software implementation.

[0037] Memory 264 may include a cache (e.g., a cache of processor 260), random access memory (RAM), magnetoresistive RAM (MRAM), read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), flash memory, solid-state storage devices, hard disk drives, other forms of volatile and non-volatile memory, or combinations of different types of memory. In embodiments, memory 264 includes a non-transitory computer-readable medium. Memory 264 may store instructions 266. Instructions 266 may include, when executed by processor 760, causing processor 260 to perform the actions described herein on reference probe 110 and / or host 130 (…). Figure 1 Instructions 266 describe operations. Instructions 266 can also be referred to as code. The terms "instruction" and "code" should be interpreted broadly to include any type of computer-readable statement. For example, the terms "instruction" and "code" can refer to one or more programs, routines, subroutines, functions, procedures, etc. An "instruction" or "code" can include a single computer-readable statement or many computer-readable statements. Instructions 266 can include aspects of a preprocessor, deep learning network, convolutional neural network (CNN), or various other instructions or codes.

[0038] The communication module 268 can include any electronic and / or logic circuitry to facilitate direct or indirect data communication between the processor circuitry 710, the probe 110, and / or the display 132 and / or the display screen 266. In this respect, the communication module 268 can be an input / output (I / O) device. In some cases, the communication module 268 assists the processor circuitry 210 and / or the probe 110 ( Figure 1 ) and / or host 130 (see Figure 1 Direct or indirect communication between various components.

[0039] Figure 3 This is a flowchart of a method 300 for determining an anatomical scanning window and / or probe orientation associated with an ultrasound image, according to various aspects of this disclosure. As shown, method 300 includes a plurality of enumerated steps, but embodiments of method 300 may include additional steps before, after, or between the enumerated steps. In some embodiments, one or more of the enumerated steps may be omitted, performed in a different order, or performed simultaneously. The steps of method 300 can be performed by any suitable component within the diagnostic system 100, and it is not necessary for all steps to be performed by the same component. In some embodiments, one or more steps of method 300 can be performed by, or at the instruction of, processor circuitry of the diagnostic system 100, including, for example, processor 260 (… Figure 2 ) or any other component.

[0040] It should be noted that method 300 may include procedures that can be applied multiple times at different times or only in one instance. In some embodiments, method 300 may describe a process of identifying anatomical scanning windows and / or probe orientations at a single point in time within a specific imaging process. In other embodiments, method 300 may be repeated in subsequent imaging processes to identify scanning windows and / or probe orientations of images associated with subsequent processes and compare them with previous scanning windows or probe orientations. Additionally, it should be noted that, besides Figure 3 In addition to the scanning window and / or probe orientation described herein, method 300 can also be used to identify and compare parameters. For example, additional parameters may include patient location, which will be described in more detail below. However, for simplicity, Figure 3 Method 300 will be described as identifying and comparing only the scan window and / or probe orientation, although additional parameters may also be identified and compared.

[0041] At step 305, method 300 includes acquiring an ultrasound image. During ultrasound imaging, an ultrasound physician or a user of the ultrasound imaging system 100 may grasp any suitable type of probe 110 and guide it toward the region of interest of the patient's anatomy. When the ultrasound physician is satisfied with the view, or when the region of interest has been properly and adequately depicted by the ultrasound imaging system 100, the ultrasound physician may acquire an image. In some embodiments, data from the inertial measurement unit (IMU) of the ultrasound probe, such as from an accelerometer, gyroscope, or other similar device, may additionally be acquired at step 305, which will be discussed in more detail below.

[0042] At step 310, method 300 includes analyzing the acquired ultrasound images to determine the anatomical scanning window and / or probe orientation. The ultrasound imaging system 100 may employ a deep learning network to determine the scanning window and / or probe orientation based on the acquired ultrasound images. In some embodiments, the deep learning network may be a CNN. In other embodiments, the deep learning network may be any other suitable implementation of an artificial intelligence system or structure, including, for example, random forest deep learning methods, regression analysis methods, or any other suitable method or structure. The deep learning network may be trained prior to the initiation of method 300 to identify the scanning window and / or probe orientation associated with a given ultrasound image. This training will be described in more detail below. The ultrasound imaging system may use a deep learning network to analyze the contents of the acquired ultrasound images and determine the scanning window and / or probe orientation from the images themselves. This additional information is generally helpful to the ultrasound physician but is rarely captured and can vary greatly from process to process.

[0043] Within a patient's anatomy, various high-density structures, such as bones or kidney stones, readily reflect sound waves and appear brightly in ultrasound images. Conversely, less dense structures reflect fewer sound waves and appear weakly or not at all in ultrasound images. An ultrasound physician imaging a region of interest within the patient must not position the ultrasound imaging probe 110 such that a high-density structure is positioned between the probe 110 and the region of interest, as the high-density structure will obscure the region of interest in the final ultrasound image. The ultrasound physician can thus locate the anatomical scanning window, or the area of ​​the anatomy through which a clear image of the region of interest can be acquired. For example, the scanning window may be intercostal, or between two ribs of the patient. In another application, the scanning window may be subcostal, meaning the probe can be positioned below the patient's ribs. The scanning window may also be in the upper abdomen, radial, parasagittal, lateral, or any other suitable scanning window, depending on the region of interest to be imaged and the surrounding structures within the patient's anatomy. Probe orientation can refer to the direction of the probe when acquiring ultrasound images. Specifically, probe orientation can be defined as a relative rotation between the patient coordinate system. For example, the patient coordinate system can be defined using left-right, top-bottom, and front-back axes, and the probe coordinate system can be defined using the major axis, minor axis, and normal axis of the transducer array. The major axis of the transducer array can be defined as the axis along which a conventional 2D image generated using the probe is traced. The minor axis of the transducer array can be defined as an axis perpendicular to the major axis in the plane of the array. The normal axis can be defined as an axis perpendicular to the plane of the array. For example, the probe orientation can be lateral, transverse, longitudinal, tilted, or any other suitable orientation. At step 310, the scan window and / or probe orientation can be estimated using a deep learning network.

[0044] At step 315, method 300 includes comparing the anatomical scan window and / or probe orientation with a desired value. In some ultrasound imaging setups, the desired value for the scan window and / or probe orientation at step 315 may be the scan window or probe orientation of an ultrasound image acquired during a previous ultrasound imaging procedure. The ultrasound system 100 may, for example, compare the scan window of the acquired image determined at step 310 with the scan window of a previously acquired image and determine whether the scan window of the acquired image is the same as or different from the scan window of the previously acquired image. If the two scan windows are different, the system 100 may notify the user of this difference by including any suitable indicator. Similarly, the probe orientation of the acquired image determined in step 310 may be compared with the probe orientation of the same previously acquired image. The system 100 may determine whether the probe orientation of the acquired image is the same as or different from the probe orientation in the previously acquired image. If the two probe orientations are different, the system 100 may notify the user of this difference by including another suitable indicator. For example, at step 315, the imaging system 100 may determine that both the scanning window and probe orientation of the acquired image differ from those of previously acquired images. In this case, the system 100 may identify the difference to the user and return to step 305, where additional images may be acquired. At step 310, the new images may be analyzed to determine new scanning windows and probe orientations that can be compared again in step 315. In another case, the system 100 may determine that the acquired image and the previously acquired image have the same scanning window, but the probe orientations of these images are different. Then, the system 100 may only indicate the difference in probe orientation and return to step 305. Alternatively, the system 100 may determine that the probe orientations are the same, but the acquired image and the previously acquired image have different scanning windows. The system 100 may only identify the difference in the scanning window and return to step 305. However, if the system 100 determines that the scanning window and probe orientation of the acquired image match those of the previously acquired image, the system 100 may indicate that the two parameters match and may proceed to step 320.

[0045] In some embodiments, during the initial ultrasound imaging process, where no previous ultrasound images and corresponding parameters (e.g., anatomical scan window and / or probe orientation) are available, step 315 may not be performed. In some embodiments, during the initial ultrasound imaging process where no previous ultrasound images and parameters are available, a recommended or target scan window and / or probe orientation may be provided to the user of the imaging system 100. For example, in such a first ultrasound imaging process, the system may determine the desired parameters based on, for example, user input selecting the desired scan window and / or probe orientation, presets stored in memory and associated with the specific anatomy being scanned, a portion of a physician's order of ultrasound images stored in memory, etc. In some embodiments, the recommended or target scan window and / or probe orientation (also referred to as target parameters) may be based on previous imaging processes of different patients in which the same region of interest was imaged. The recommended or target parameters may be based on recommendations from experts in the field and may be stored in memory communicating with the system 100 and retrieved based on input related to the region of interest to be imaged, the patient's condition, or any other factors. In some embodiments, the recommended scan window and / or probe orientation may be based on images generated by other imaging modalities, such as those generated by X-ray, magnetic resonance imaging (MRI), and / or computed tomography (CT) scanners. For example, if the patient's anatomy or a similar patient anatomy has already been imaged using a CT scanner, at step 315, the sonographer may use the data acquired by the CT scanner to determine the recommended scan window and / or probe orientation, and may input the determined scan window or probe orientation into system 100 for comparison. Where target parameters are provided from other sources instead of prior image acquisition, they may be provided by a physician, hospital, or other medical expert, or healthcare provider or organization. For example, a physician, hospital, or other medical expert, or healthcare provider or organization may determine the target parameters as a standard or default protocol or preference based on any suitable data or environment. These target parameters may be input into system 100 via user input devices such as a keyboard, mouse, or touchscreen, via auditory signals, or any other suitable input device or interface communicating with processor circuitry.

[0046] As illustrated by the loop created in steps 305-315, method 300 can guide the user of system 100 to acquire ultrasound images in a similar manner to before, or to achieve the most probable comparison using recommended scan windows and / or probe orientations. In this way, changes in the region of interest can be observed more easily and accurately during longitudinal studies and / or imaging.

[0047] At step 320, method 300 includes annotating the image using an anatomical scan window and / or probe orientation. At step 315, after the scan window and / or probe orientation of the acquired image matches the expected value, method 300 may annotate the acquired ultrasound image using the determined scan window or probe orientation. Annotating the image may include graphically overlaying alphanumeric text and / or symbols onto the ultrasound image itself, making the text and / or symbols visible along with the image. Additionally or alternatively, annotating the image may include associating data representing the anatomical scan window and / or probe orientation with the image (e.g., data in the image's metadata and / or image file header). Therefore, annotating the ultrasound image does not necessarily require providing a scan window and / or probe orientation to be displayed on the image. In some embodiments, even if the scan window and / or probe orientation does not match the expected value, a user of system 100 may instruct the system to proceed to step 320 regardless and annotate the acquired image using the new scan window and / or probe orientation. In other cases, even if the scanning window and / or probe orientation matches the desired value, the user of system 100 can instruct the system to return to step 305 to acquire, analyze, and compare additional images.

[0048] At step 325, method 300 includes storing the anatomical scan window and / or probe orientation in memory. The scan window and / or probe orientation of a newly acquired ultrasound image can be stored in conjunction with the ultrasound image itself, so that the image can be displayed together with its corresponding scan window or probe orientation when retrieved later. Data representing the scan window and / or probe orientation and data representing the image can be associated based on additional data stored in memory, the additional data linking the data representing the scan window and / or probe orientation and the data representing the image. The ultrasound image, the corresponding scan window and / or probe orientation, and the linked data can be stored in any suitable location. For example, the image, scan window, and / or probe orientation can be stored in memory 138 within the host 130 itself. In other embodiments, the image, scan window, and / or probe orientation can be stored in memory 140, which communicates with but is spaced apart from the host 130. In other embodiments, data can be stored in both memory 138 and memory 140.

[0049] At step 330, method 300 includes retrieving the anatomical scanning window and / or probe orientation. The processor circuitry is capable of retrieving the scanning window and / or probe orientation along with the ultrasound image (or vice versa) because they are associated in memory. In some embodiments, the scanning window and / or probe orientation stored in conjunction with the image acquired at step 325 can be specified as the desired value of step 315 in a subsequent imaging process. In this way, step 330 can be performed in a subsequent imaging process, or it can be performed at the end of the same process described above. As indicated by the arrows from step 330 to step 305, the process described in method 300 can be performed iteratively. Method 300 can be performed multiple times consecutively on the same patient in a single imaging process or point-of-care scenario, or it can be performed at different times or in different imaging processes.

[0050] Figure 4 This is a schematic diagram of an ultrasound imaging system 400 with a deep learning network according to various aspects of this disclosure. System 400 can also be referred to as an artificial intelligence framework. In this respect, the artificial intelligence framework 400 may include a deep learning network, and various aspects of the framework may be executed by processor 134, processor circuitry 210, and / or may include instructions similar to the previously described instructions 266. Figure 4 The illustrated embodiment of the artificial intelligence framework 400 includes various input images 407 received from various sources such as an ultrasound scanner 405, a picture archiving and communication system (PACS) 410, or other image storage systems 415. The framework 400 may include an analysis module 420 having a preprocessor 422 and a deep learning network 424. In some embodiments, the analysis module 420 is capable of being implemented on the host 130 (…). Figure 1 ) and / or processor circuit 210 ( Figure 2 In the context of artificial intelligence framework 400, labels 430 can be generated as output. Deep learning network 400 can be trained to recognize multiple organs within a patient's anatomy or any other structure, or can process images derived from multiple organs. In some embodiments, deep learning network 400 may also be or include multiple deep learning networks, each trained to recognize one organ within a patient's anatomy or a structure, or can process images derived from a single organ.

[0051] Input image 407 may be an ultrasound image depicting a region of interest within the patient's anatomy. Input image 407 may include any suitable number of ultrasound images. For example, input image 407 may include only a single ultrasound image, or it may include multiple ultrasound images, including dozens, hundreds, or thousands of ultrasound images, and any higher or lower number of ultrasound images. Input image 407 can be an image stream of frames acquired continuously (e.g., in real time) by an ultrasound imaging probe. Input image 407 can be transmitted from scanner 405 (e.g., the memory of scanner 405) to analysis module 420 while the image is being acquired or after the image is being acquired. Input ultrasound image 407 may include images captured using any suitable scan window and / or probe orientation. For example, image 407 may include a set of images showing the region of interest from one angle, another set of images showing it from a different angle, and so on. Input image 407 may be of any suitable type or file format, and any suitable size or resolution.

[0052] The analysis module 420 can directly receive the input image 407 from the ultrasound scanner 405. The ultrasound scanner 405 can be a probe 110, a transducer array 112, and / or a main unit 130, or have components connected to the probe 110, transducer array 112, and / or main unit 130. Figure 1 The input images 407 received from the ultrasound scanner 405 can be acquired by the ultrasound scanner 405 at any time. For example, the ultrasound scanner 405 may acquire several input images 407 under the guidance of the user of the system 100 or otherwise, and immediately transmit them to the analysis module 420. In other cases, the ultrasound scanner 405 may transmit input images 407 from previous imaging processes to the analysis module 420.

[0053] The input image 407 can also be received from the PACS 410 by the analysis module 420. The PACS 410 may include... Figure 1 Memory 138, Figure 1 Memory 140, Figure 2The memory 264 or any other suitable component described herein. PACS 410 may store multiple files or other data associated with one or more patients. PACS 410 may include wired or wireless storage devices, such as servers, portable media storage devices, or any other form of media storage. PACS 410 may contain patient information from multiple patients and multiple imaging examinations and / or other medical examinations or procedures. In some embodiments, upon request from the user of analysis module 420 and / or system 100, PACS 410 may transmit any suitable number of input images 407 to analysis module 420. PACS 410 may transmit input images 407 from various previous imaging procedures from any patient. In some embodiments, depending on the application and needs of the sonographer and / or patient, input images 407 may be ultrasound images depicting the same region of interest within the same patient to be imaged in the current procedure, or they may be images depicting other regions of the same patient or different patients.

[0054] As illustrated by the other image storage system 415, the analysis module 420 may also additionally receive input images 407 from other sources. For example, the other image storage system 415 may be a portable media storage device, such as a secure digital storage device, a universal serial bus storage device, an optical disc, or any other form of portable media storage. The other storage system 415 may also include a cloud-based storage implementation.

[0055] Analysis module 420 may include a preprocessor 422, which may include hardware (e.g., circuit components) and / or software algorithms (e.g., executed by a processor). Preprocessor 422 may perform various functions to adjust, filter, or otherwise modify the received input image 407 before it is transmitted to deep learning network 424. For example, preprocessor 422 may manipulate the incoming image. Preprocessor 422 may perform various image processing steps for training and prediction purposes. Preprocessor 422 may perform various tasks, such as changing image contrast, image size, resolution orientation, geometry, cropping, spatial transformation, resizing, normalization, histogram modification, or various other processes, to help deep learning network 424 work more efficiently or accurately.

[0056] The preprocessor 422 can also cache and filter the incoming images 407, providing a subset of the incoming images 407 that have passed through the cache or filter as output. For example, the preprocessor 422 can determine a similarity metric (e.g., a normalized correlation coefficient) between the received and cached input images 407, and discard images that are too similar to other images in the cache. In this way, the preprocessor 422 can measure the correlation between the received input images 407 and quantify the similarity using a similarity metric. The preprocessor 422 can be configured to compile input images 407 that have met the similarity metric threshold until a threshold number of N input images 407 is reached. Each time N images 407 that have passed the similarity criterion are stored at the preprocessor, the group or subset can be passed to the deep learning network 424 for processing, and the oldest images can be discarded.

[0057] Using more than a single input image 407 can enhance the identification of regions of interest within the patient's anatomy. By selecting multiple input images 407 that are sufficiently distinguishable from each other and firing them into a deep learning network 424, the deep learning network 424 can generate more robust estimates of the anatomical scanning window, probe orientation, patient position, and / or other parameters. The number of input images 407 fired into the deep learning network 424 in a single group or subset can be any suitable number. For example, the preprocessor can fire a single image 407, or it can fire two, three, four, ten, one hundred, or more images 407, or any number thereof. The process of selecting and firing the input images 407 can occur during the training or inference phase of the illustrated framework 400.

[0058] In some embodiments, the preprocessor 422 may additionally use image processing techniques, deep learning network techniques, or any other suitable techniques to identify any structures depicted in the current image. The preprocessor 422 can use features similar to those described in the following patents to identify structures within ultrasound images: U.S. Patent No. 10,424,044, entitled "Anatomically intelligent echocardiography for point-of-care"; U.S. Patent No. 7,794,398, entitled "Real-time volumetric bi-plane ultrasound imaging and quantification"; U.S. Patent No. 8,698,795, entitled "Interactive image segmentation"; U.S. Patent No. 7,117,026, entitled "Physiological model based non-rigid image registration"; U.S. Patent No. 10,290,076, entitled "System and method for automated initialization and registration of navigation system"; U.S. Patent No. 8,144,950, entitled "Method for facilitating post-processing of images using deformable meshes"; and U.S. Patent No. 322, entitled "3D ultrasound imaging". U.S. Patent No. 10,709,425, concerning the “system,” is incorporated herein by reference in its entirety. The structure can be an anatomy within the patient’s body, such as an organ. In some embodiments, the deep learning network 424 may include multiple artificial intelligence networks, each trained to recognize parameters of ultrasound images of various common structures within the patient’s anatomy. After recognizing the imaged structure, the preprocessor may transmit the input image 407 to the appropriate corresponding deep learning network. For example, if a set of input images 407 analyzed at preprocessor 422 depicts a liver, preprocessor 422 may determine a similarity metric between the input images 407 and select a subset of input images 407 that are sufficiently different from each other, and transmit the images 407 to the deep learning network 424 trained to recognize parameters of ultrasound images depicting a liver. Similarly, a filtered subset of images 407 depicting a heart may be transmitted to a deep learning network 424 trained for images depicting a heart.In other embodiments, a single deep learning network 424 can identify image parameters of an ultrasound image, regardless of any structure depicted in the image.

[0059] The deep learning network 424 can receive a subset of images 407 modified and filtered by the preprocessor 422 as input. The deep learning network 424 can include hardware (e.g., circuit components) and / or software algorithms (e.g., executed by a processor). The deep learning network 424 can then identify various parameters associated with the received ultrasound images 407, which will be described in more detail below. Parameters identified by the deep learning network 424 may include the anatomical scanning window of the ultrasound probe, probe orientation, patient position during imaging, and / or other parameters. In some embodiments, in addition to the input images 407, the deep learning network 424 may also receive other inputs, such as patient IMU data or physiological measurements. These additional inputs can be received from the preprocessor 422, directly from the probe 110 (… Figure 1 ) or patient sensor 142 ( Figure 1 The image can be received from (e.g., from the source) or from any other suitable source. In some embodiments, the deep learning network 424 may determine the scan window, probe orientation, and / or patient location associated with each input image 407. In other embodiments, the deep learning network 424 may determine the scan window, probe orientation, and / or patient location associated with a subset of the input images 407, rather than determining each image individually. The deep learning network 424 may be any suitable form or type as described above.

[0060] The deep learning network 424 can generate labels 430 as output, reflecting parameters of the input image 407 or a subset of the input image 407, including anatomical scanning windows, probe orientation, and / or patient position. Labels 430 can be of any suitable type. For example, labels can include visual representations, such as any type or length of alphanumeric text, symbols, images, or any other representation. Furthermore, labels can include any suitable sound or any other method of communicating the determined parameters to the user of the system. Output 430 can be emitted from the analysis module 420 to a display and / or memory (e.g., as part of or in communication with the scanner 405, PACS 410, and / or memory 415).

[0061] In some embodiments, the deep learning network 424 may include a convolutional neural network (CNN). For example, a CNN may be or include a multi-class classification network, or an encoder-decoder type network. In some cases, the analysis module may implement a random forest algorithm instead of a deep learning network.

[0062] Figure 5This is a schematic diagram of a convolutional neural network (CNN) configuration 500 according to various aspects of this disclosure. For example, the CNN configuration 500 can be implemented as a deep learning network 424 ( Figure 4 In this embodiment, configuration 500 can perform a classification task. Configuration 500 can be of any suitable type and can include any suitable type or number of layers, including but not limited to convolutional layers, fully connected layers, flattened vectors, or any other technique or implementation of an artificial intelligence system. (See reference) Figure 5 The embodiments shown and / or described can be scaled to include any suitable number of CNNs (e.g., about 2, 3 or more). Configuration 500 can be trained to identify various anatomical scanning windows, probe orientations and / or patient locations associated with received ultrasound images, as described in more detail below.

[0063] A CNN may include a set of N convolutional layers 510, where N is any positive integer, each followed by a pooling layer 515. The CNN may also include a set of K fully connected layers 520, where K can be any positive integer. In one embodiment, the fully connected layers 520 include at least two fully connected layers 520. The convolutional layers 510 are shown as 510. （1） up to 510 （N） Pooling layer 515 is shown as 515. （1） Up to 515 （N） The fully connected layer 520 is shown as 520. （1） Up to 520 （K） Each convolutional layer 510 may include a set of filters 512 configured to extract features from input 505 (e.g., ultrasound images or other additional data). Convolutional layers 510 may include convolutional kernels of different sizes and strides. The values ​​N and K, as well as the size of the filters 512, may vary depending on the embodiment. In some cases, convolutional layers 510... （1） up to 510 （N） Pooling layer 515 （1） Up to 515 （N） and fully connected layer 520 （1） By 520 （K-1） S-shaped, rectified nonlinear (ReLU), leakyReLU, softmax, or hyperbolic tangent activation functions can be used. Pooling layer 515 can include max pooling or average pooling techniques. Fully connected layer 520 can gradually reduce the high-dimensional output to the dimension of the prediction result (e.g., classification output 530). Therefore, fully connected layer 520 can also be called a classifier. In some embodiments, fully convolutional layer 510 can be additionally called a perceptron or perceptual layer. Fully connected layer 520 can downsample the received information and map it to a finite number of categories 532. In embodiments, the final fully connected layer 520（K） It can be followed by a final classification layer such as softmax to convert the net activations in the final output layer into a set of values ​​that can be interpreted as probabilities.

[0064] The classification output 530 can indicate a confidence score or probability for each of a plurality of categories 532 based on the input image 505. In this respect, the CNN 500 can be a multi-class classification network. In an exemplary embodiment, each of the plurality of categories 532 is a combination of scanning window and probe orientation, as shown in Table 1 below. Label 1 can be one such combination (e.g., right subcostal transverse), label 2 can be another combination (e.g., right subcostal longitudinal), and so on. The output 530 indicates the probability that the input image 505 belongs to a specific category 532. For example, a high confidence score for label 1 and low confidence scores for the other labels indicate that the output label of the CNN 1400 for the input image 505 is label 1 (right subcostal transverse). The confidence score can be of any suitable type. In one embodiment, the confidence score can be a probability (e.g., a decimal value between 0 and 1). In this way, the output of the CNN can be a probability value for each possible option. The option with the highest probability is assigned to the image. In some cases, the highest probability value is compared with a threshold, and if the value meets the threshold, an output label is assigned to the image. For example, if the highest probability value exceeds a minimum threshold, an output label is assigned to the image. If the highest probability value is below the minimum threshold, no option is selected or assigned, or a dedicated "unknown" label is assigned. If the image is stored / archived, the label can be stored along with the image, for example, in a DICOM header.

[0065] In embodiments of deep learning networks including encoder-decoder networks, the network may include two paths. One path may be a shrinking path, where a large image, such as image 505, may be convolved by several convolutional layers 510, such that the size of image 505 changes in the depth of the network. Image 505 may then be represented in a low-dimensional space or a flat space. From this flat space, an additional path may expand the flat space to the original size of image 505. In some embodiments, the implemented encoder-decoder network may also be referred to as a principal component analysis (PCA) method. In some embodiments, the encoder-decoder network may segment image 505 into patches.

[0066] In some embodiments, the output labels generated by the deep learning network may be in the form of ordinal numbers (1, 2, 3, etc.) representing the anatomical scanning window and probe orientation used during the acquisition of the input image 505. The ultrasound imaging system 100 may access a database or table that associates ordinal values ​​with combinations of scanning windows and probe orientations. Some examples of ordinal labels and corresponding combinations of scanning windows and probe orientations are shown in Table 1 below:

[0067] Table 1: Exemplary Labels Assigned to Scan Window and Probe Orientation Table 1 may correspond to possible scanning windows and probe orientations for imaging a specific structure or region of interest within a patient's anatomy (e.g., the liver). In some embodiments, the ultrasound imaging system 100 may access multiple tables similar to Table 1, which associate sequence labels with corresponding scanning windows and probe orientations. Each table may correspond to a different structure or region of interest within the patient's anatomy. In embodiments where a separate deep learning network is trained for a single region of interest, each deep learning network may retrieve information from a single lookup table associated with that region of interest. In embodiments where a single deep learning network is trained for multiple regions of interest, the deep learning network may access different lookup tables depending on the region of interest being imaged. In other embodiments, a single lookup table may correspond to all regions of interest to be imaged.

[0068] In some embodiments, two CNNs can be implemented to identify the scanning window and probe orientation separately. In this case, the outputs of the CNNs are two distinct values, for example, two separate ordinal labels corresponding to the scanning window and probe orientation, respectively. An example of a table corresponding to the separate scanning window label and probe orientation table is shown in Table 2 below:

[0069] Table 2: Exemplary labels assigned to the scanning window and labels assigned to the probe orientation In another embodiment, a single CNN can be implemented to identify the scan window and probe orientation in two separate sets of labels. In this case, the last fully connected layer 520 （k） It can be connected to two independent sets of classification outputs 530 （1） and 530 （2） Each group has its own group label 532 （1） and 532 （2） , of which 532 （1） This corresponds to scan window label 532. （2） And it can correspond to the probe orientation label.

[0070] In an additional embodiment, probe orientation can be predicted or estimated as three continuous variables representing the rotation of the probe relative to the patient's standard coordinate system, rather than restricting the probe orientation to discrete values ​​represented by ordered number labels. The patient's standard coordinate system can be defined by three axes: left-right axis, front-back axis, and head-to-tail axis. The probe's position at the time of image acquisition can then be determined relative to the patient coordinate system. This orientation determination can be performed using IMU data or optical image data. In embodiments where IMU data is received from an ultrasound probe, the ultrasound probe can be equipped with an orientation measurement tool, such as an accelerometer, gyroscope, or other measuring tool, to determine the probe's orientation relative to the gravity vector. This orientation can then be converted to a patient axis based on the patient's position. Probe orientation as a continuous three-valued coordinate system can also be determined based on optical camera images captured at the time of ultrasound image acquisition. Various image processing techniques can be used to estimate the probe orientation relative to the patient coordinate system. The estimated probe orientation coordinates can also be fine-tuned after the ultrasound image acquisition. For example, the optical camera images can be stored in memory 138 or memory 140 and associated with the acquired ultrasound images for later review. Additional details regarding the orientation of the probe relative to the patient's position will be described in more detail below.

[0071] In the embodiment where the probe orientation is estimated as continuous three-valued coordinates, Figure 5 The deep learning network 500 shown may include a regression task. For example, a fully connected layer 520 may perform a regression task. In some embodiments, the deep learning network may output a continuous ternary coordinate corresponding to orientation and a second continuous ternary coordinate corresponding to translation, as well as any other coordinates related to the position of the probe during image acquisition. Determining probe orientation as a continuous variable can utilize features similar to those described in PCT / EP2020 / 081548, filed November 10, 2020, entitled “Ultrasound Screening Coverage Tool,” which is incorporated herein by reference in its entirety. Determining probe orientation as a continuous variable can also utilize features similar to those described in EP application No. 20150464.4, filed January 7, 2020, entitled “Patient Model Estimation for Interventions,” which is also incorporated herein by reference in its entirety. Figure 6This is a flowchart of a method 600 for analyzing ultrasound images from an imaging process and a flowchart of a method 650 for analyzing and comparing ultrasound images from an additional imaging process, according to various aspects of this disclosure. As shown, methods 600 and 650 include a plurality of enumerated steps, but embodiments of method 600 may include additional steps before, after, or between the enumerated steps. In some embodiments, one or more of the enumerated steps may be omitted, performed in a different order, or performed simultaneously. The steps of methods 600 and 650 can be performed by any suitable component within the diagnostic system 100, and all steps do not need to be performed by the same component. In some embodiments, one or more steps of methods 600 and 650 can be performed by the processor circuitry of the diagnostic system 100 (including, for example, processor 260). Figure 2 (or any other component) to execute, or to execute under its guidance.

[0072] At step 605, method 600 includes acquiring a baseline ultrasound image. As shown, steps 605 through 620 can be performed at time point T1. At time point T1, a baseline ultrasound image can be acquired. This baseline can be associated with an anatomical scan window and / or probe orientation. The scan window and / or probe orientation can be selected by the user of the ultrasound imaging system 100 based on a scan window and / or probe orientation recommended for imaging a specific region of interest. The scan window and / or probe orientation can also be based on an imaging plan determined by the user of the system 100, or can be arbitrarily selected.

[0073] At step 610, method 600 includes analyzing the acquired ultrasound images to determine the anatomical scanning window and / or probe orientation. At step 610, scanning window and / or probe orientation labels can be automatically generated using the previously described deep learning network techniques. The ultrasound imaging system can use a deep learning network to analyze the contents of the acquired ultrasound images and determine the scanning window and / or probe orientation from the images themselves.

[0074] At step 615, method 600 includes annotating the acquired ultrasound images using an anatomical scan window and / or probe orientation. The annotation of the ultrasound images can be done in any suitable manner. For example, visual representations such as alphanumeric text or graphic representations can be embedded within the acquired ultrasound images or created as separate data and associated with the ultrasound images.

[0075] At step 620, method 600 includes storing the acquired ultrasound images together with the anatomical scanning window and / or probe orientation. The ultrasound images and the scanning window and / or probe orientation can be stored on any suitable media storage device, including memory 138 or memory 140 within host 130. Figure 1Ultrasound images and scan windows and / or probe orientations can also be stored in PACS 410. Figure 4 ).

[0076] Figure 6 Method 650 is also shown. The steps of method 650 can be performed after the steps of method 600. For example, method 600, including steps 605 to 620, can be performed at time point T1. Method 650, including steps 655 to 685 as shown, can be performed at time point T2, i.e., at a time point after T1. Steps 655 to 685 of method 650 can also be performed any number of times, including at an additional time point T3 after T2, at a time point T4 after T3, and so on. In some embodiments, Figure 6 The imaging processes at time points T1, T2, T3, etc., illustrated demonstrate a longitudinal study of the region of interest within a patient. Specifically, the same region of interest imaged at time point T1 according to method 600 can be imaged at subsequent time points according to method 650. This longitudinal study allows sonographers or physicians to monitor changes in the region of interest over extended periods. According to methods 600 and 650, any length of time can elapse between consecutive imaging processes. For example, the time between two imaging processes could be minutes, hours, days, weeks, months, or years.

[0077] At step 655, method 650 includes retrieving the anatomical scan window and / or probe orientation of a previous imaging procedure at a previous time point. In some embodiments, the scan window and / or probe orientation, and depending on the applied patient location and / or other parameters, may be the same scan window, probe orientation, and / or other parameters stored in step 620 of method 600 previously described. The scan window and / or probe orientation retrieved at step 655 may be a desired or target value for the procedure described by method 650.

[0078] At step 660, method 650 includes displaying the previous anatomical scan window and / or probe orientation. The scan window and / or probe orientation can be transmitted to the sonographer by any suitable method. For example, the scan window and / or probe orientation can be displayed as a visual representation. Specifically, they can be displayed as alphanumeric text or as a graphical representation such as symbols or images. The scan window and / or probe orientation can also be transmitted to the sonographer via audio. Regarding Figure 6Step 660 illustrates an example of alphanumeric text 631. This alphanumeric text can indicate a scan window, such as “right intercostal” as shown, or any other scan window, and a probe orientation, such as “lateral” as shown, or any other probe orientation. The probe orientation can alternatively or additionally be displayed as three-valued continuous coordinates as described above. The scan window and / or probe orientation can additionally or alternatively be shown as a graphical representation similar to graphical representation 632. Graphical representation 632 can include a depiction of patient 633. In some embodiments, the position of patient 633 can be visually displayed via a depiction of patient 633, allowing the sonographer to identify the patient, for example, in a supine position during previous imaging. System 100 can also display the patient position via alphanumeric text combined with alphanumeric text 631, or the position can be conveyed via any other symbol, sound, or otherwise. The graphical representation can also include an indicator 634. Indicator 634 can convey the scan window and / or probe orientation to the sonographer. Indicator 634 can be shown in conjunction with patient 633. Indicator 634 can indicate a position on the patient's body corresponding to the scan window and / or the patient's position. The indicator can be or includes any suitable symbol, color, alphanumeric text, pattern, or any other feature to specify information to the sonographer.

[0079] At step 665, method 650 includes acquiring additional ultrasound images. The sonographer may use the anatomical scan window, probe orientation, and / or patient position shown at step 660 to position the ultrasound probe in a location similar to the probe position in a previous imaging procedure and may acquire images. In some embodiments, the imaging system including the probe for acquiring images at step 665 may be the same imaging system and probe used for acquiring previous images at step 605, or the sonographer may use a different imaging system and / or probe than previously used at step 605. In some embodiments, step 660 of method 650 may not be performed prior to step 665. For example, the sonographer may acquire images without any guidance or indication of the previous scan window, probe orientation, and / or patient position. In some embodiments, the sonographer may choose to retrieve and view the previous scan window, probe orientation, and / or patient position, or may not choose not to retrieve and view them.

[0080] At step 670, method 650 includes analyzing additional ultrasound images to determine the anatomical scanning window and / or probe orientation. System 100 can use any suitable deep learning network, such as... Figure 4 Network 424 or Reference Figure 5The described structure is used to automatically identify image parameters such as scan window, probe orientation, and / or patient location in the image based on the acquired ultrasound images. Step 670, which analyzes the ultrasound images acquired at step 665, may include any features described in step 310 of the previously referenced method 300 or step 610 of method 600.

[0081] At step 675, method 650 includes comparing the anatomical scan window and / or probe orientation of the additional ultrasound image with the scan window and / or probe orientation of the previous ultrasound image. The scan window, probe orientation, and / or patient position estimated at step 670 can be compared with the scan window and / or probe orientation and / or patient position retrieved at step 655. System 100 can determine whether the scan window of an image acquired at the current time point matches the scan window acquired at time point T1, and similarly compare probe orientation and patient position from different time points.

[0082] At step 680, method 650 includes displaying feedback to the user a comparison of the anatomical scan window and / or probe orientation based on the additional ultrasound images with the scan window and / or probe orientation of the previous ultrasound images. Any suitable indicator can be used to communicate the comparison results to the sonographer. For example, any of the previously mentioned communication methods can be used. If the label corresponding to any parameter of the image is different, the indicator may include a warning to be conveyed to the sonographer. The indicator may indicate which parameters are the same and which parameters are different. For example, the scan window of the image acquired in the current imaging procedure may be different from that of the procedure at time point T1, but the probe orientations of the current procedure and the previous procedure may be matched. As per step 680 Figure 6 As shown in the visual representation 651, the visual representation 652 for parameter-based comparison feedback may include text 652 related to the analyzed parameters and an indicator 653 associated with each parameter. For example, if the scan window of the current image matches the scan window of a previous image retrieved at step 655, an indicator 656 may be displayed. The indicator 656 may be a different color, pattern, shape, or any other visual feature indicating a scan window match. For example, if the probe orientation of the current image differs from the previous probe orientation retrieved at step 655, an indicator 657 may be displayed. The indicator may also be any color, pattern, shape, or other visual feature, but may differ from indicator 656 to distinguish the two. In some embodiments, the visual representation 651 may also include text and / or indicators related to the patient position and the similarity or difference between the current patient position and the patient position in a previous procedure. At step 680, the system 100 may additionally or alternatively display alphanumeric text such as text 658 to convey whether there are differences in parameters between the current ultrasound image and the previous ultrasound image, as shown.

[0083] If one or more parameters of the current ultrasound image acquired at step 665 differ from any parameters of the previous ultrasound image retrieved at step 655, the method can return to step 665, as indicated by the arrow. At step 665, the sonographer can adjust the positioning of the ultrasound probe relative to the patient's anatomy based on the comparison received at step 680. The sonographer can also adjust the patient's position. The sonographer can then acquire additional ultrasound images. At step 670, the system 100 can reanalyze the acquired ultrasound images to determine the anatomical scanning window, probe orientation, and / or patient position. At step 675, the system can compare the newly determined parameters associated with the image reacquired at step 665 with the original image parameters from step 655. At step 680, the system can display feedback based on the new comparison. In this way, a loop is created that allows the sonographer to repeatedly acquire new ultrasound images of the patient's anatomy while adjusting the ultrasound probe until the scanning window, probe orientation, and / or patient position match the scanning window, probe orientation, and / or patient position of the previous imaging procedure retrieved at step 655.

[0084] At step 685, method 650 includes storing an additional ultrasound image and its associated scan window and / or probe orientation. A new image can be stored along with the scan window, probe orientation, and / or patient position when system 100 indicates that the scan window, probe orientation, and / or patient position in some embodiments match the same parameters as a previous ultrasound image. This new image and its associated parameters can be stored in the same location as the previous image and parameters, or they can be stored in a different location. The new image and associated parameters can be stored in such a way that they can be retrieved during subsequent imaging. During subsequent imaging, the new image and parameters can determine desired values ​​to guide the sonographer in the same probe positioning. (See reference...) Figure 3 In some embodiments, even when the parameters do not match those of a previous procedure, the sonographer may still have the ability to override method 650 and store the images and associated parameters. Similarly, although all parameters associated with the image may match those of a previous procedure, the sonographer may direct system 100 back to step 665 of method 650 to acquire, analyze, and compare additional ultrasound images.

[0085] Figure 7 This is an illustrated view of a graphical user interface 700 for an ultrasound imaging system according to various aspects of this disclosure, which compares parameters of ultrasound images prior to image acquisition during additional imaging procedures. Reference Figure 6 Step 660 of method 650 Figure 7The interface shown may correspond to a typical user interface displayed to the user of system 100 at step 660, wherein an anatomical scan window and / or probe orientation of the previous imaging procedure may be displayed. Interface 700 includes ultrasound image 705, biographical data 715, text 725, patient location representation 730, box 710, biographical data 720, and title 740.

[0086] Ultrasound image 705 may be an ultrasound image acquired during a previous imaging process. Image 705 may depict the region of interest to be imaged in the current process. Image 705 may correspond to an anatomical scan window and / or probe orientation that can be used as target parameters for this process. In some embodiments, image 705 may not be displayed within interface 700. Image 705 may be annotated with the scan window and / or probe orientation associated with image 705, or may be additionally annotated with any other data, measurements, notes, or other information.

[0087] Biographical data 715, shown below ultrasound image 705, may indicate data associated with image 705, parameters associated with image 705 such as anatomical scan window and / or probe orientation, or any other data related to the imaging procedure that acquired image 705 and / or associated parameters. For example, as shown, biographical data 715 may indicate the date of the imaging procedure or the date image 705 was acquired. It may also indicate the time image 705 was acquired. Biographical data 715 may include additional information, such as, but not limited to, the equipment used to acquire image 705 and associated parameters, the physician or sonographer performing the imaging procedure, any notes related to the imaging procedure, or any other suitable information. Biographical data 715 may be located anywhere suitable within user interface 700, including any location relative to image 705.

[0088] Text 725 located above ultrasound image 705 can convey parameters associated with ultrasound image 705 to the user. For example, as shown, text 725 can indicate the anatomical scan window and / or probe orientation corresponding to image 705. Text 725 may additionally include information related to other parameters such as patient position. As previously described, in addition to or instead of probe position text, text 725 may also include various continuously variable coordinates indicating probe position. Similar to biographical data 715, text 725 can be located anywhere suitable within user interface 700, including any position relative to image 705.

[0089] The patient location indication 730 shown above the text 725 can also indicate to the user the patient's location at the time of image acquisition 705. The location indication 730 can be any suitable indicator, including symbols, alphanumeric text, or images. In some embodiments, a general image of the model patient can be displayed in the location of the actual patient at the time of imaging. In some embodiments, the indication 730 can be an optical camera image of the actual patient during the imaging process. The text indicating the patient's location may or may not accompany the indication 730.

[0090] The box 710 shown adjacent to image 705 can be reserved space within interface 700 for ultrasound images from the current imaging process. However, as previously mentioned, interface 700 may correspond to a time prior to any new ultrasound data or images being acquired, and therefore the current ultrasound image may not be available.

[0091] Within the context of user interface 700, biographical data 720 can indicate the current date and time to the user. This data can be updated upon receiving additional ultrasound images acquired during the current imaging process. In some embodiments, the graphical user interface 700 may not include biographical information 720. Biographical information 720 may include any information that references biographical information 715 but corresponds to a description of the current imaging process, contrasting with the imaging process of image 705.

[0092] The header 740 may convey additional explicit information to the user of system 100. For example, header 740 may include the patient's name and patient identification number. In some embodiments, header 740 may include additional information, such as the patient's date of birth, age, or other information. Furthermore, header 740 may include the patient's physiological information, including the patient's height, weight, heart rate, respiratory rate, medical condition, allergies, or any other suitable information related to the patient to be imaged.

[0093] Figure 8 This is an illustrated view of a graphical user interface 800 for an ultrasound imaging system according to various aspects of this disclosure, which compares parameters of ultrasound images and indicates differences between parameters. The graphical user interface 800 may reflect an exemplary interface displayed to the user of the ultrasound system 100 after ultrasound images have been acquired during the current imaging process. Figure 8 The interface shown is 700 ( Figure 7 The same image 705 shown, and the same biographical information 715, text 725, patient location indication 730, and title 740 previously shown and described. However, Figure 8It also depicts new ultrasound images 810, updated biographical information 820, text 825, patient location representation 830, text 835, comparison information 840, and indicators 842 and 844.

[0094] The new ultrasound image 810 can be one or more ultrasound images acquired during the current ultrasound imaging procedure (e.g., one or more image frames in an ultrasound image stream). After reviewing the user interface 700 for the anatomical scan window and / or probe orientation for the previous imaging procedure, the sonographer can attempt to place the ultrasound probe in a similar position to obtain an image with the same scan window and / or probe orientation as the previous image 705. Image 810 can be previously acquired by... Figure 7 The box 710 shown is displayed to the user within the space of the interface 800. Once the sonographer is satisfied with the positioning of the ultrasound probe, they can instruct the system 100 to record ultrasound images 810. In some embodiments, as the current image stream is continuously acquired, the processor circuitry compares the scan window and / or probe orientation of the current ultrasound image stream with the scan window or probe orientation of previous ultrasound images. In some embodiments, the processor circuitry compares only the scan window and / or probe orientation of the recorded ultrasound images from the current image stream with the scan window and / or probe orientation of previous ultrasound images.

[0095] After acquiring and / or recording images 810, system 100 can replace them with updated biographical information 820. Figure 7 The updated biographical information 820 can reflect the date and time when image 810 was acquired by system 100. If necessary, system 100 can also update any other information in the updated biographical information 820.

[0096] Similarly, after acquiring and / or recording image 810, as referenced Figure 3 Method 300 and Figure 6 As described in methods 600 and 650, system 100 can perform analysis on image 810 to determine the anatomical scan window and / or probe orientation associated with image 810, as well as any other parameters. Text 825 can instruct the ultrasound physician on the scan window and / or probe orientation of image 810. Similar to text 725 previously described, text 825 can contain any suitable information and be located anywhere suitable within interface 800.

[0097] The patient location indicator 830 can indicate to the user the patient's location when image 810 was acquired. Similar to patient location indicator 730, indicator 830 can be any suitable indicator, including an optical image of the patient, and can be accompanied by or without additional text describing the patient's location. It can also be positioned anywhere suitable within interface 800.

[0098] After image 810 is acquired and displayed to the user, and after the anatomical scan window and / or probe orientation, as well as any other parameters, are automatically determined by system 100, system 100 compares the scan window and / or probe orientation of image 705 with the scan window and / or probe orientation of the newly acquired image 810. The comparison result may be displayed along with comparison information 840. Comparison information 840 may include any suitable type of communication, including auditory, visual, or other representations, including alphanumeric text, symbols, images, or any other means of communication. Figure 8 In the illustrated embodiment, comparison information 840 may include text associated with each parameter being compared. For example, comparison information 840 may include text associated with the scanning window and / or text associated with the probe orientation. In some embodiments, it may also include information associated with the patient's location.

[0099] Each parameter being compared can be accompanied by an indicator. For example, as shown in the figure, the text "Scanning Window" is accompanied by indicator 842, and the text "Probe Orientation" is accompanied by indicator 844. Text corresponding to the patient's location will also be accompanied by similar indicators.

[0100] exist Figure 8 In the example shown, indicator 842 can indicate a mismatch in the associated parameters of images 705 and 810. This difference is also indicated by the differences in text 725 and text 825. Specifically, the anatomical scan window of image 705 and the anatomical scan window of image 810 are different. The scan window of image 705 is the right subcostal region, while the scan window of image 810 is the left subcostal region. Indicator 844 can indicate that the associated parameters of images 705 and 810 do match. This parameter matching is indicated in text 725 and text 825 by the same term "lateral". Specifically, the probe orientation of image 705 and the probe orientation of image 810 are both lateral. Naturally, any scan window or probe orientation can be indicated by text 725 and 825, and can produce varying results in comparison information 840.

[0101] Text 835 can be used as an additional or alternative means of conveying the comparison results of parameters associated with images 705 and 810. Figure 8 In the example shown, text 835 can provide information related to differences in the anatomical scan windows between images 705 and 810. In some embodiments, a warning can be displayed as text 835 if the parameters differ between the images. In some embodiments, text 835 may only convey information related to the different parameters. In other embodiments, text 835 may convey information related to all parameters, regardless of whether these parameters differ between images 705 and 810.

[0102] Figure 9 This is an illustrated view of a graphical user interface for an ultrasound imaging system according to various aspects of this disclosure, which compares parameters of ultrasound images and indicates differences between parameters. Similar to graphical user interface 800, graphical user interface 900 may reflect an exemplary interface displayed to the user of ultrasound system 100 after ultrasound images have been acquired during the current imaging process. Figure 9 The image shown is interface 700 ( Figure 7 The same image 705 shown, and the same biographical information 715, text 725, patient location representation 730, title 740, updated biographical information 820, and patient location representation 830. However, Figure 9 It also depicts newly acquired image 910, text 925, comparison information 940, indicators 942 and 944, and text 935.

[0103] and Figure 8 Similar to image 810, a new ultrasound image 910 can be an ultrasound image acquired during the current ultrasound imaging process. A user of system 100 can acquire a new image 910 based on the anatomical scan window and / or probe orientation of image 705, or in response to acquiring an image with a scan window and / or probe orientation that does not match the target parameters. After acquiring image 910, it can be displayed to the user.

[0104] As previously described, after acquiring image 910, system 100 can perform analysis to determine the anatomical scanning window and / or probe orientation associated with image 910, as well as any other parameters. Text 925 can instruct the ultrasound physician on the scanning window and / or probe orientation of image 910. Text 925 can share any of the features of the previously mentioned text 725 or 825.

[0105] After image 910 is acquired and displayed to the user, and after the anatomical scanning window and / or probe orientation is determined, system 100 compares the scanning window and / or probe orientation of image 705 with the scanning window or probe orientation of the newly acquired image 910. The result of the comparison may be displayed along with comparison information 940. Comparison information 940 may be similar to comparison information 840 and may include any suitable type of communication, including auditory, visual, or other representations. For example, comparison information 940 may include text related to the scanning window and text related to the probe orientation. It may also include information related to the patient's location.

[0106] exist Figure 9In the example shown, indicator 942 can indicate a mismatch in the associated parameters of images 705 and 910. This difference is also shown through the differences in text 725 and text 925. Specifically, the probe orientation of image 705 is different from that of image 910. The probe orientation of image 705 is horizontal, while that of image 910 is vertical. Indicator 944 can indicate a match in the associated parameters of images 705 and 910. This match in parameters is indicated by the same term "right subcostal" in text 725 and text 925. Specifically, the anatomical scan window of image 705 and the anatomical scan window of image 910 are both horizontal.

[0107] Text 935 can be used as an additional or alternative means of conveying the comparison results of parameters associated with images 705 and 910. Figure 9 In the example shown, text 935 can provide information related to the difference in probe orientation between images 705 and 910.

[0108] Figure 10 These are illustrated views of a graphical user interface for an ultrasound imaging system according to various aspects of this disclosure, which compares parameters of ultrasound images and indicates that the parameters are the same. Similar to graphical user interfaces 800 and / or 900, graphical user interface 1000 may reflect an exemplary interface displayed to the user of ultrasound system 100 after ultrasound images have been acquired during the current imaging process. Figure 10 The image depicts many things related to Figure 8 and / or Figure 9 Similar elements. However, Figure 10 The newly acquired image 1010, text 1025, comparison information 1040, indicators 1042 and 1044, and text 1035 are depicted.

[0109] and Figure 8 Image 810 and Figure 9 Similar to image 910, a new ultrasound image 1010 can be an ultrasound image acquired during the current ultrasound imaging process. A user of system 100 can acquire a new image 1010 based on the anatomical scan window and / or probe orientation of image 705, or in response to acquiring an image with a scan window and / or probe orientation that does not match the target parameters. After acquiring image 1010, it can be displayed to the user.

[0110] As previously described, after acquiring image 1010, system 100 can perform analysis to determine the anatomical scanning window and / or probe orientation associated with image 1010, as well as any other parameters. Text 1025 can instruct the ultrasound physician on the scanning window and / or probe orientation of image 1010. Text 1025 can share features of any of the previously mentioned texts 725, 825, or 925.

[0111] After image 1010 is acquired and displayed to the user, and after the anatomical scanning window and / or probe orientation is determined, system 100 compares the scanning window and / or probe orientation of image 705 with the scanning window and / or probe orientation of the newly acquired image 1010. The result of the comparison may be displayed along with comparison information 1040. Comparison information 1040 may be similar to comparison information 840 or 940 and may include any suitable type of communication, including auditory, visual, or other representations. For example, comparison information 1040 may include text related to the scanning window and text related to the probe orientation. It may also include information related to the patient's location.

[0112] exist Figure 10 In the example shown, indicator 1044 can indicate the associated parameter matching of images 705 and 1010. Specifically, the anatomical scan window and probe orientation of image 705 and the scan window and probe orientation of image 1010 are both "right subcostal transverse" in text 725 and text 1025.

[0113] Text 1035 can be used as an additional or alternative means of conveying the comparison results of parameters associated with images 705 and 1010. Figure 10 In the example shown, text 1035 can indicate that the anatomical scan windows and probe orientations of images 705 and 1010 are the same.

[0114] In some embodiments, any of the described graphical user interfaces, including interfaces 700, 800, 900, and 1000, can describe physiological conditions such as heart rate and respiratory rate, as well as other conditions. These physiological conditions may affect the flushing and rinsing phases of contrast agents within the patient's body, phases used for better visualization of anatomical structures. This physiological information can also be recorded by system 100 (e.g., using...). Figure 1The physiological condition can be estimated using the patient sensor 142 in the image, or based on motion observed in the imaging. These physiological conditions can also be compared to previous values ​​and used to guide subsequent acquisitions, thereby taking their impact on the results into account and minimizing them. In some embodiments, physiological condition measurements can be estimated using the deep learning network described previously, or the physiological condition measurements can be additional inputs to the deep learning network. In other embodiments, the estimation or use of physiological condition measurements can be integrated into different deep learning networks, which are separate from the network responsible for estimating ultrasound image parameters such as anatomical scan windows and / or probe orientation. In embodiments where physiological conditions are estimated by a deep learning network, the network can be responsible for detecting motion to output the estimated physiological measurement.

[0115] Training deep learning networks can be accomplished using a variety of techniques. In one embodiment, training a deep learning network can be achieved by creating a large dataset of sample ultrasound images with different scan windows and / or probe orientations. These sample images can be additionally obtained from a large number of patients. In an embodiment where the deep learning network is trained to identify image parameters such as scan window and / or probe orientation from images depicting many different regions of interest, the sample images can depict a wide variety of regions of interest. In an embodiment where multiple deep learning networks are trained, each deep learning network can be responsible for identifying image parameters from ultrasound images depicting only one region of interest. In such an embodiment, the large number of sample images selected for training a single deep learning network can all depict the same type of region of interest, although each sample image will still depict various scan windows and / or probe orientations, and can be obtained from a large number of different patients.

[0116] As an example of a training implementation, each sample image selected for training a deep learning network can be assigned a variable I. k Each image I k Label L can be assigned k ,in, Lk Corresponding to image I k The anatomical scanning window and / or probe orientation for image acquisition. Deep learning networks can use batch tuples (I... k L k ) to conduct batch training, where I k It is a potentially pre-processed ultrasound image, L k It means that in image I k The corresponding labels for the ordinal forms (1, 2, 3, ...) of the anatomical scanning window and / or probe orientation used during acquisition. In some embodiments, the labels for the scanning window and / or probe orientation may also be separated into two independent values ​​[S]. k P k ], where Sk Corresponding to image I k Associated scan window, P k Corresponding to image I k Associated probe orientation. Then use tuples (I k [S] k P k ]) Perform training, where, [S k P k This includes a set of independent values ​​describing the scan window and / or probe orientation. Values ​​can be added related to patient location or image I. k Any other associated parameters corresponding to additional values. The following describes the generation of label L. k The method.

[0117] To train the network, random batches of tuples (I0) can be generated. k L k Images can be propagated forward through a deep learning network, creating temporary labels and assigning L to each image. k A loss function can be defined to measure the L of all labels in a batch. k and L k The error between ' and ' can be measured. This error can then be backpropagated through the network, where the optimizer adjusts the network's parameters to improve subsequent predictions. The training process can continue for a fixed number of iterations, or until a certain convergence criterion is met. For example, the training process can continue until, for a specified number of iterations, the error no longer improves.

[0118] In one embodiment, the loss function used during the training of the deep learning network can measure whether the estimated probe angle and / or the probe's anatomical scanning window is correct or incorrect. For example, given a deep learning network that can be trained to recognize a finite set of labels, the network can assign a probability to each category. For example, the network can assign an 80% probability to one category and a 1% probability to another. The loss function can then determine an error metric that indicates the degree to which the network predicts the correct label with a high probability and the incorrect label with a low probability. The loss function can include various other features or steps. For example, in embodiments where the probe orientation is defined as different three-valued coordinates rather than a finite set of labels, the loss function can include an error metric relating to the angular or radian difference between the predicted angle of the probe on any defined axis and the actual angle of the probe.

[0119] Get tag L k One approach is to have the ultrasound physician assign labels L during image acquisition (prospectively) or during image review (retrospectively). k . refer to Figure 11A and Figure 11BThis method is described in more detail. Figure 11A This is a schematic view of a graphical user interface 1100 for annotating ultrasound images during the training phase of a deep learning network, according to various aspects of this disclosure. Users responsible for annotating ultrasound images selected in a training set of sample ultrasound images, such as experts in the field, can view and use an interface similar to interface 1100 to assign labels to each image. The graphical user interface 1100 depicts ultrasound images 1105, text 1110, graphics 1115 including models 1119 and indicators 1117, and options 1120.

[0120] Ultrasound image 1105 can be obtained using ultrasound system 100 or any other similar ultrasound system. Image 1105 can depict any suitable region of interest. An ultrasound physician can instruct system 100 to retrieve ultrasound image 1105 for display. The ultrasound physician can select from various options 1120 corresponding to aspects of image 1105. For example, the ultrasound physician can indicate the region of interest shown in ultrasound image 1105 and / or the patient's position relative to the ultrasound imaging probe, as well as other parameters of the ultrasound image. As the ultrasound physician identifies parameters from the available options 1120, data corresponding to ultrasound image 1105 can be updated. For example, as... Figure 11A As shown, graph 1115 reflects the decision made by the expert sonographer after selecting the highlighted patient location in option 1120.

[0121] Graphic 1115 can be used as an indication of option 1120 selected by an expert sonographer. It can include any suitable visual representation, including alphanumeric text, symbols, models, and / or images. Figure 11A In the illustrated embodiment, graph 1115 depicts model 1119. Model 1119 may reflect the sonographer's selection of option 1120 and may indicate the patient's position when acquiring ultrasound image 1105. Graph 1115 may additionally include indicator 1117. Indicator 1117 may illustrate any suitable information related to the marking of image 1105. In some embodiments, indicator 1117 may illustrate the estimated position of the ultrasound probe relative to the patient's body when acquiring image 1105. Any appropriate number or type of indicator may be included within graph 1115 and may convey any appropriate information related to image 1105.

[0122] The text 1110 and graphic 1115 shown overlaid on the acquired ultrasound image 1105 can also convey to the user of system 100 the current parameters selected by the expert. For example, text 1110 may indicate the type of region of interest depicted by the expert in the ultrasound image 1105. Text 1110 may also depict any other suitable information, such as the anatomical scan window and / or probe orientation selected by the expert user at that time, or other information related to the ultrasound image 1105.

[0123] It should be noted that although graphic 1115 and text 1110 are shown as overlaying image 1105 in interface 1100 of FIG. 11, graphic 1115 and text 1110 can be positioned anywhere suitable within interface 1100 in relation to image 1105. Similarly, although option 1120 is shown below image 1105, it can also be positioned anywhere suitable within interface 1100.

[0124] Figure 11B This is a graphical user interface 1150 for annotating ultrasound images during the training phase of a deep learning network, according to various aspects of this disclosure. Figure 11B It can be used as a reference. Figure 11A The described interface 1100 is a standalone interface. The graphical user interface 1150 may also be an optional or additional view of the same interface 1100. The graphical user interface 1150 depicts additional or optional options 1155 that can be used by a specialist sonographer responsible for creating labels for the ultrasound image 1105 shown.

[0125] Option 1155 can be any suitable type. In some embodiments, option 1155 can be... Figure 11A The options 1120 shown are alternatives to those available to a specialist sonographer. In some embodiments, option 1120 and / or option 1155 may be optional buttons within the user interface. In some embodiments, the specialist sonographer may enter text reflecting any image parameters, such as the anatomical scan window, probe orientation, patient position, and / or others. Figure 11B Option 1155 shown can be a button containing text. Options can also contain text similar to... Figure 11A The image or symbol for option 1120 shown. Similar to the graphical user interface 1100, an expert in the art responsible for annotating ultrasound images can view and use an interface similar to interface 1150 to assign labels to each image.

[0126] It is possible to use a tracking system to implement image I k Automatically provide tag L kOther methods. In some embodiments, the tracking system can be an optical tracking system. Optical camera tracking can involve a photographic device used in conjunction with an ultrasound imaging system. For example, an optical camera acquires images of the patient and probe during image acquisition. The optical images depict the position of the probe on the patient's body and the patient's body positioning. Based on the probe position and body positioning, the processor circuitry can determine a label L for the anatomical scanning window and / or probe orientation. k For example, using image processing techniques, the processor circuitry can identify the patient's head and feet, as well as the patient's left and right regions, to create a patient coordinate system. The processor can then use similar image processing techniques to identify the probe's coordinate system. The processor circuitry can then compare these two coordinate systems to identify the probe orientation. In some embodiments, the system 100 can also identify various features within images received from the imaging device to determine the scanning window. In such embodiments, the system 100 can use image processing techniques to identify the location of ribs or bones within the patient's anatomy and determine the position and orientation of the probe 1222 relative to those ribs or other bones.

[0127] In other embodiments, electromagnetic tracking can be used with two imaging modalities (e.g., ultrasound and CT, or ultrasound and MRI) to automatically track images. k Provide tag L k If you will refer to Figure 12 A more detailed description. Figure 12 This is a graphical view of a computed tomography (CT) or magnetic resonance imaging (MRI) and ultrasound system 1200 for automatically annotating images during the training phase of a deep learning network, according to various aspects of this disclosure. For example, system 1200 can automatically assign labels L to anatomical scanning windows and / or probe orientations. k System 1200 can be similar to the Philips® PercuNav® system. System 1200 can be called a fusion imaging system because it combines image data from two different imaging modalities (ultrasound and CT / MRI).

[0128] Figure 12 The description includes a patient 1210, an ultrasound system 1220 including an ultrasound probe 1222, a CT or MRI system 1250 including a CT or MRI scanner 1255, and a tracking system 1215. The ultrasound system 1220 and / or probe 1222 may be similar to the ultrasound system 100 and / or probe 110 described herein. Figure 12 Also shown is an ultrasound image 1225 of patient 1210 generated by ultrasound system 1220 based on ultrasound data acquired by probe 1222, and an image 1235 of patient 1210 generated by CT / MRI system 1250 based on CT or MRI data acquired by CT / MRI scanner 1255. Figure 12 The document also illustrates an automatically generated label 1245 based on image 1235, and image and label pairs (or tuples) 1230. Image 1235 can be a CT image or an MRI image. Image 1235 can be a 3D image, although only a two-dimensional axial section is shown for simplicity. In other embodiments, image 1235 can also be a 2D image and can be captured along the coronal plane of patient 1210 or along any other suitable plane. The system 1250 shown can be a CT system or an MRI system. Similarly, scanner 1255 can be a CT scanner or an MRI scanner. Features of both CT and MRI imaging systems can be implemented in this disclosure to obtain image 1235. Image 1235 can be a CT image, an MRI image, or any other suitable image. Processor circuitry automatically segments image 1235 to identify structures 1236 within the patient's body, such as the patient's ribs. An ultrasound physician can place ultrasound probe 1222 at a suitable location on or near the anatomy of patient 1210. The ultrasound system 1220 controls the probe 1222 to acquire ultrasound images 125, wherein the ultrasound image 125 itself can be an image I of tuple 1230. k .

[0129] Tracking system 1215 can be an electromagnetic tracking system that tracks ultrasound probe 1222 based on magnetic field strength. Tracking system 1215 registers and tracks the pose (e.g., position and / or orientation) of ultrasound probe 1222 and ultrasound image 1225 generated by ultrasound system 1220 relative to a 3D image (e.g., CT or MRI image 1235). Ultrasound system 1220 can spatially co-register image 1235 and ultrasound image 1225 by analyzing CT / MRI image 1235 and ultrasound image 1225 (e.g., performing image processing thereon) and automatically matching identical structures (e.g., ribs) within the images. System 1200 can include aspects similar to those described in U.S. Patent No. 10,290,076, entitled “System and method for automated initialization and registration of navigation system,” issued May 14, 2020, which is incorporated herein by reference in its entirety.

[0130] Probe tracking using the electromagnetic tracking system 1215 allows for the identification of the probe's position and orientation relative to the patient's CT or MRI image. This probe orientation relative to the patient's CT or MRI image forms part of label 1245. Due to spatial co-registration between image 1235 and ultrasound image 1225, the probe position (tracked by the electromagnetic tracking system 1215) segmented relative to structures (e.g., ribs, bones) in the image allows for scanning windows. For example, a tracked probe position located below the segmented thoracic cavity would be identified as a subcostal window, while a probe position located on the right side of the thoracic cavity would be identified as a right scanning window. This scanning window forms part of label 1245. An illustration 1237 showing the determined probe position and orientation relative to the patient's body 1210 is provided at the corresponding location in image 1235. Label 1245 is label L of tuple 1230. k Tuple 1230 can be generated for each ultrasound image in the training set. Each tuple 1230 may include ultrasound image 1225 (e.g., image I). k ), and labels 1245 associated with the scanning window and / or probe orientation (e.g., label L) k ), and any other values ​​corresponding to other parameters of the image (e.g., patient location).

[0131] Figure 13 This is a schematic diagram of a CNN configuration 1300 including inertial measurement unit (IMU) data according to various aspects of this disclosure. The CNN configuration 1300 can be similar to the reference... Figure 5 The configuration described is 500.

[0132] The input to configuration 1300 can be an ultrasound image 1305. Similar to network configuration 500, CNN configuration 1300 can include multiple N convolutional layers 1310, each followed by a pooling layer 1315. Configuration 1300 can also include multiple fully connected layers 1320, such as... Figure 13 As shown. As output, a CNN configuration of 1300 can be used in a similar manner. Figure 5 The classification output 530 assigns a label to the input image 1305 at classification step 1330. Any layer shown in configuration 1300, as well as the input and output, can be substantially similar to the reference. Figure 5 The configuration described has 500 layers, inputs, and outputs.

[0133] Figure 13The configuration 1300 shown can receive IMU data vector 1350 as additional input. IMU data vector 1350 can include orientation information (θ1, θ2, θ3) and acceleration information (a1, a2, a3) of the ultrasound probe as two sets of three continuous variables. IMU data vector 1350 can also contain additional information related to the position of the ultrasound probe, such as rotational velocity or magnetic field components. Many ultrasound probes are equipped with IMU sensors to provide accelerometer and gyroscope data in real time. This data can be used to determine the probe's angle relative to gravity and to determine the relative change in the probe's rotation about all three probe axes. This data, in addition to image data 1305, can also be provided as input to network 1300. Network 1300 can, for example, connect the additional data vector to a first fully connected layer 1320. （1） Additional data vectors are embedded using connections. Therefore, this additional IMU data can be used during the training and application / inference phases of network 1300. Including IMU data as input, in addition to image 1305, can be used to improve prediction accuracy. In some embodiments, the IMU data can improve the estimation of probe orientation. In some embodiments, the IMU data can estimate probe orientation over extended time periods. Figure 13 As shown, the output of network 1300 can be compared with... Figure 5 The output of network 500 shown is the same. In some embodiments, IMU data can be compared with a patient coordinate system, such as by referencing... Figure 16 As described in more detail.

[0134] Figure 14 This is a schematic diagram of a CNN configuration 1400, including IMU data, based on various aspects of this disclosure. The CNN configuration 1400 can be similar to that described in the reference. Figure 5 The described configuration 500 and / or reference Figure 13 The configuration described is 1300.

[0135] The input to configuration 1400 can be an ultrasound image 1405. Similar to network configurations 500 and / or 1300, CNN configuration 1400 can include multiple K convolutional layers 1410, each accompanied by a pooling layer 1415. Configuration 1400 may also include multiple fully connected layers 1420, such as... Figure 14 As shown in the image. The output is similar to... Figure 5 The classification output is 530 and Figure 13 The classification output 1330, CNN configuration 1400 can assign labels to the input image 1405 at classification step 1430. Any layer shown in configuration 1400, as well as the inputs and outputs, can be compared with a reference. Figure 5 The configuration described is 500 or reference. Figure 13 The described configuration of the 1300 layer, input, and output are basically similar.

[0136] Similar to configuration 1300 ( Figure 13 ), Figure 14 The configuration 1400 shown can receive IMU data vector 1450 as additional input. For example, network 1400 can connect the additional data vector 1450 to the first fully connected layer 1420. （1） Links are used to embed additional data vectors 1450. Additionally, configuration 1400 allows for classification of the patient location used for input image 1405 (e.g., the patient's location when image 1405 was acquired). Regarding... Figure 15 To describe exemplary patient localization. Regarding Figure 16 This describes the use of IMU data to determine patient location. The labels 1435 of the classification output 1430 additionally include patient location, as well as scan window and probe orientation (e.g., regarding...). Figure 5 and 13 (As described). With classification output 530 ( Figure 5 ) and 1330 ( Figure 13 Compared to classification output 1430, which includes a larger number of labels 1435, classification output 530 generates additional parameter combinations by including patient location as one of the parameters. Figure 5 ) and 1330 ( Figure 13 The classification output includes n labels (e.g., possible combinations of scan windows and probe orientations). By adding m possible patient locations while maintaining all combinations of scan windows and probe orientations, the classification output 1430 includes n*m labels.

[0137] Figure 15 This is a graphical view of an exemplary patient location based on various aspects of this disclosure. In some applications, patient location can be an important parameter for repeatable longitudinal measurements. As a non-limiting example, Figure 15 Many commonly used patient postures for liver scans are described.

[0138] Figure 15Image 1505 depicts an area of ​​a patient in a supine position 1535. Image 1505 also depicts an indicator 1507 corresponding to the position of the scan window for viewing regions of interest (e.g., the liver within the patient shown). Image 1510 depicts an area of ​​a patient in a left posterolateral (LPO) position 1540. Image 1510 also depicts an indicator 1512 corresponding to the position of the region of interest. Image 1515 depicts an area of ​​a patient in a right posterolateral (RPO) position 1545. Image 1515 also depicts an indicator 1517 corresponding to the position of the region of interest. Image 1520 depicts an area of ​​a patient in a left lateral decubitus (LLD) position 1550. Image 1520 also depicts an indicator 1522 corresponding to the position of the region of interest. Finally, image 1525 depicts an area of ​​a patient in a left lateral decubitus (RLD) position 1555. Image 1525 also depicts an indicator 1527 corresponding to the position of the region of interest. Figure 15 Any of the locations shown can be used as Figure 14 Additional labels 1435. Depending on the region of interest to be imaged, the parameters to be recognized by the deep learning network, or any other features, additional patient locations can be available labels.

[0139] Figure 16 This is a schematic view of ultrasound probes in various orientations according to various aspects of this disclosure. Using IMU data measurements from the ultrasound probes, the ultrasound system 100 can determine the patient's position during the ultrasound imaging process. Figure 16 An ultrasonic transducer probe 1605 with three axes is shown, the three axes including an x-axis 1690, a y-axis 1692, and a z-axis 1694. Figure 16 Image 1610, combined with probe 1605 and gravity vector 1615, is also depicted. Figure 16 Image 1620, combined with probe 1605 and gravity vector 1625, is also depicted.

[0140] Figure 16 The ultrasonic probe 1605 shown can be used with Figure 1 probe 110, Figure 12 The probe 1222, or any other probe described herein, is substantially similar. The positioning or orientation of the probe 1605 can be defined by the coordinate system shown. For example, the x-axis 1690 can be defined as the direction toward the left and right sides of the probe. The y-axis can be defined as extending directly in front of and behind the probe 1605, or in the main direction of sound propagation. The z-axis 1694 can be defined as extending directly above and below the ultrasound probe 1605 as shown. Figure 16The origins of these axes 1690, 1692, and 1694 shown can be any suitable location on or within the probe 1605. For example, in one embodiment, the origin can be defined as the midpoint along the ultrasonic transducer array.

[0141] Generally, the patient can rest on the master horizontal surface. This horizontal surface can be used as a reference horizontal surface. Depending on the patient's position, the orientation of the ultrasound probe 1605 will differ relative to the reference horizontal surface, as shown by the probe's orientation relative to the patient in images 1610 and 1620. The IMU readings of probe 1605 can be used to identify the difference in probe orientation relative to the reference horizontal surface. The IMU readings can then be used to learn and estimate the patient's position. For example, if a side view is obtained with the patient in a supine position as shown in image 1610, the IMU readings of probe 1605 will be measured, with the gravitational acceleration (i.e., the "downward" direction) represented by the gravity vector 1615 aligned with the probe's z-axis 1694. However, if the same scanning plane is obtained with the patient in a left lateral decubitus position as shown in image 1620, the gravitational acceleration, represented by the gravity vector 1625, is aligned with the probe's y-axis 1692. Therefore, a reference can be used... Figure 14 The described extended label set 1435, based on corresponding image and IMU data inputs, trains a modified network to predict anatomical scanning windows and probe orientation, as well as patient positioning as output. Alternatively, a second, but similar network can be trained to classify only patient positioning. In this way, the ultrasound system 100 can estimate probe orientation using a combination of IMU data and patient orientation as shown in the captured images, and the IMU data can also be used to estimate patient orientation. Patient position can be recorded and displayed during subsequent imaging so that the sonographer knows how to locate the patient, as previously described.

[0142] Those skilled in the art will recognize that the above-described apparatus, systems, and methods can be modified in various ways. Therefore, those skilled in the art will understand that the embodiments covered by this disclosure are not limited to the specific exemplary embodiments described above. In this regard, although illustrative embodiments have been shown and described, a wide range of modifications, changes, and substitutions are contemplated in the foregoing disclosure. It should be understood that such changes can be made to the foregoing without departing from the scope of this disclosure. Therefore, it is appropriate to interpret the appended claims broadly in a manner consistent with this disclosure.

Claims

1. An ultrasound imaging system, comprising: The processor circuitry is configured as follows: Target parameters representing (i) the target anatomical scanning window and / or probe orientation and (ii) the patient position are stored in a memory that communicates with the processor circuitry; Receive a first ultrasound image acquired by a first ultrasound probe having a first anatomical scanning window during a first acquisition period; For the first ultrasound image, determine a first parameter representing (i) the first anatomical scanning window and / or probe orientation and (ii) the patient position; Retrieve the target parameter from the memory; Compare the target parameter with the first parameter; and The visual representation of the comparison is output to a display that communicates with the processor circuitry.

2. The ultrasound imaging system according to claim 1, wherein, The target parameters include (i) the anatomical scanning window and / or probe orientation and (ii) the patient position of the second ultrasound probe for the second ultrasound image acquired during a second acquisition period prior to the first acquisition period.

3. The ultrasound imaging system according to claim 2, wherein, The processor circuit is configured as follows: The second ultrasound image is stored in the memory such that the target parameter is associated with the second ultrasound image in the memory.

4. The ultrasound imaging system according to claim 3, wherein, The processor circuit is configured as follows: Receive the second ultrasound image obtained by the second ultrasound probe; Determine the target parameters representing (i) the second anatomical scanning window and / or probe orientation and (ii) the patient position; and The target parameter is associated with the second ultrasound image.

5. The ultrasound imaging system according to claim 4, wherein, The processor circuit is configured as follows: Associate the first parameter with the first ultrasound image; The first parameter and the first ultrasound image are stored in the memory such that the first parameter and the first ultrasound image are associated in the memory; and The first parameter is retrieved from the memory for comparison with other parameters corresponding to other ultrasound images.

6. The ultrasound imaging system according to claim 2, wherein, The processor circuit is configured to communicate with the first ultrasound probe, and wherein the processor circuit is configured to: Control the first ultrasound probe to acquire the first ultrasound image; and During the acquisition of the first ultrasound image, a screen display including at least one of the second ultrasound image or the target parameter is output to the display.

7. The ultrasound imaging system according to claim 2, wherein, The processor circuit is configured to output a screen display to the display, the screen display including the target parameter, the first parameter, the first ultrasound image, the second ultrasound image, and the visual representation of the comparison, displayed simultaneously.

8. The ultrasound imaging system according to claim 2, further comprising the first ultrasound probe, wherein, The first ultrasonic probe includes an inertial measurement unit, wherein the processor circuit is configured to determine the first parameter based on data obtained by the inertial measurement unit.

9. The ultrasound imaging system according to claim 8, wherein, The first parameter includes a continuous variable, and the processor circuitry is configured to output a visual representation of the continuous variable to the display.

10. The ultrasound imaging system according to claim 1, wherein, The processor circuit is configured as follows: Receive user input to select the target anatomical scan window; and The target parameters are determined based on the user input.

11. The ultrasound imaging system according to claim 1, wherein, The processor circuitry includes a preprocessor and at least one deep learning network.

12. The ultrasound imaging system according to claim 11, wherein, The preprocessor is configured to: Receive multiple ultrasound images acquired by the first ultrasound probe during the first acquisition period, wherein the multiple ultrasound images include the first ultrasound image; The plurality of ultrasound images are cached in the memory; The multiple ultrasound images are filtered; and A subset of the multiple ultrasound images is output to the deep learning network.

13. The ultrasound imaging system according to claim 12, wherein, In order to filter the multiple ultrasound images, the preprocessor is configured to: Determine a similarity measure for the plurality of ultrasound images cached in the memory; and A portion of the plurality of ultrasound images is removed from the memory based on the similarity metric to generate the subset of the plurality of ultrasound images.

14. The ultrasound imaging system according to claim 11, wherein, The deep learning network is configured to determine the first parameter.

15. The ultrasound imaging system according to claim 14, wherein, The deep learning network includes a convolutional neural network (CNN).

16. The ultrasound imaging system according to claim 1, wherein, The visual representation of the comparison includes: A first indicator when the target parameter and the first parameter are the same; and A second indicator when the target parameter is different from the first parameter.

17. The ultrasound imaging system according to claim 2, wherein, The processor circuit is configured as follows: The memory stores the second patient's physiological status during the second acquisition period; Receive the first patient's physiological status during the first data collection period; Retrieve the second patient's physiological status from the memory; Compare the physiological conditions of the second patient with those of the first patient; and A visual representation of the comparison between the second patient's physiological condition and the first patient's physiological condition is output to the display.

18. An ultrasound imaging system, comprising: Processor circuitry, the processor circuitry being configured to: Target parameters representing the target anatomical scanning window and / or target probe orientation and patient position are stored in a memory that communicates with the processor circuitry. Receive ultrasound images acquired during the acquisition period by an ultrasound probe having an anatomical scanning window and / or probe orientation and the patient's position; A convolutional neural network is used to determine parameters representing the anatomical scanning window and / or the probe orientation and the patient position from the received ultrasound images; Retrieve the target parameter from the memory; Compare the target parameter with the parameter; and The visual representation of the comparison is output to a display that communicates with the processor circuitry, wherein the visual representation of the comparison includes: A first indicator when the target parameter and the parameter are the same; and A second indicator when the target parameter and the parameter are different.

19. A method for determining an anatomical scan window associated with an ultrasound image, comprising the steps of: During the first acquisition period, a first ultrasound image is acquired using an ultrasound probe with a first anatomical scanning window; Analyze the acquired first ultrasound image to determine (i) the first anatomical scanning window and / or probe orientation and (ii) the patient position; The expected values ​​of (i) the scanning window and / or probe orientation and (ii) the patient position are compared with (i) the anatomical scanning window and / or probe orientation and (ii) the patient position from the analysis step to determine whether (i) the first anatomical scanning window and / or probe orientation and (ii) the patient position are the same as or different from the expected values; If (i) the orientation of the first anatomical scan window and / or the probe and (ii) the patient position differ from the expected value, then return and repeat the acquisition, analysis and comparison steps; and If (i) the orientation of the first anatomical scanning window and / or the probe and (ii) the patient position are the same as the expected value, then The first ultrasound image is annotated using the scan window and / or probe orientation determined by (i) and the patient position described in (ii); The scanning window and / or probe orientation determined in (i) and the patient position described in (ii) along with the first ultrasound image are stored in a computer memory for later use, and Retrieve (i) the stored determined scan window and / or probe orientation and (ii) the patient location, along with the first ultrasound image, for use in a subsequent comparison step.

20. The method of claim 19, further comprising the step of displaying the first ultrasound image together with (i) the determined scanning window and / or probe orientation and (ii) the patient position.

Images