Systems, methods, and apparatus for quantitative assessment of organ mobility

Abstract

An ultrasound imaging system can analyze motion of landmarks in a time sequence of image frames to determine mobility of one or more landmarks. In some examples, a sequence of image frames can be analyzed by one or more artificial intelligence models to detect landmarks and generate flow fields for the detected landmarks. The flow fields can be analyzed to determine isotropy of movement of the landmarks. The isotropy analysis can be used to generate a quantitative mobility index.

Description

Technical Field

[0001] This disclosure relates to imaging systems and methods for assessing organ mobility based on ultrasound imaging. In particular, it relates to imaging systems and methods for generating quantitative assessments of ovarian and pelvic organ mobility. Background Technology

[0002] Endometriosis affects about one in ten women during their reproductive years (typically between 15 and 49 years of age), affecting approximately 176 million women worldwide. The lining inside the uterus is called the endometrium. Endometriosis is a clinical condition where tissue resembling the endometrium is found outside the uterus. This triggers a chronic inflammatory process, and the tissue surrounding the uterus begins to scar. This inflammatory process is usually severe during the menstrual cycle, as the endometrium thickens to prepare for implantation of a fertilized egg. If the egg is not fertilized, the endometrium sheds as part of the normal menstrual cycle. However, in some cases, the endometrium does not completely shed during menstruation. Instead, it attaches to organs surrounding the uterus, such as the fallopian tubes, ovaries, or the large and small intestines (rectosigmoid colon, especially in cases like deep invasive endometriosis), causing lesions. These lesions are part of the inflammatory process; therefore, they also shed and cause a variety of symptoms, including extreme pain, periodic pain (dysmenorrhea), chronic pelvic pain, bowel problems, and infertility. The earlier endometriosis is detected and treated, the better the outcome. Treatment for endometriosis typically involves medication or surgery, depending on the spread and size of the lesions. Pain relievers are usually over-the-counter, and hormone therapy is often recommended because it slows the growth of endometrial tissue and prevents new implantation (e.g., hormonal contraceptives). If the lesions are too large or if the endometrial implants are removed before pregnancy, laparoscopic surgery is often an option.

[0003] Detection of endometriosis is primarily performed using transvaginal ultrasound imaging (TVUS). On ultrasound, endometriomas appear as ovarian cysts with homogeneous hypoechoic appearance and no evidence of blood within. Functional cysts disappear or become smaller at different stages of the menstrual cycle, while endometriosis does not shrink without treatment. Standard TVUS guidelines recommend examining the uterus and ovaries during a routine examination. However, in cases of chronic pain, this examination is only part of the procedure. When a patient is referred for in-depth TVUS, pelvic and rectal examinations are necessary to identify and rule out the presence of obstruction or endometriosis. The most important part of the examination for these patients is assessing the mobility of the ovaries and uterus, blind tubes, intestinal wall and rectovaginal septum, and rectouterine pouch. In particular, studies of organ mobility have proven to be a method for detecting intestinal adhesions and, most importantly, for understanding the severity of obstruction of the uterus and ovaries relative to other organs in the presence of endometriosis. Organ mobility (or its lack thereof) can also be used to assess other medical conditions, such as detecting adhesions due to scar tissue (e.g., hypertrophic scar formation). Summary of the Invention

[0004] Movement of organs within an image frame and / or cluster displacement of organs can be assessed to provide one or more mobility maps and / or mobility indices (e.g., organ mobility indices), which may be quantitative in some examples. Mobility indices can be used as indicators of clinical diseases (e.g., endometriosis). However, mobility indices can be used for other medical conditions where the movement of organs relative to each other indicates the condition (e.g., adhesions resulting from hypertrophic scarring).

[0005] The systems, methods, and apparatuses can perform landmark detection of pelvic organs (e.g., uterus, ovaries, internal iliac vessels) on B-mode images. In some examples, artificial intelligence (AI) (e.g., deep learning, machine learning, etc.) can be used for landmark detection. In some examples, B-mode images can be acquired via transvaginal ultrasound (TVUS) images. AI can be used to evaluate the absolute displacement and / or global movement of pelvic landmarks between B-mode frames. The systems, methods, and apparatuses can generate two-dimensional (2D) displacement and / or velocity maps. This map can then be evaluated to determine the isotropic and / or anisotropic nature of the motion. Based on this evaluation, a mobility map can be generated, which allows a user to determine which pelvic structures in the field of view are fixed (e.g., attached to another structure) or moving. In some examples, the systems, methods, and apparatuses can generate an ovarian mobility index (OMI), which can be based on the amount of fixed tissue compared to the amount of moving tissue in the generated map.

[0006] Optionally, in some examples, the system, method, and apparatus may cluster organ displacements based on the distribution of velocity vectors in a velocity map. Optionally, in some examples, inertial measurement unit (IMU) data may be extracted to detect push-pull movements and probe rotation to determine which time period of the collected data to apply OMI processing to, provide user feedback on properly performed probe movements, and / or exclude processing of data with corrupted motion patterns in the images due to sudden / incorrect probe movements.

[0007] According to at least one example of this disclosure, an ultrasound imaging system can be configured to provide quantification of marker mobility, and the system may include a non-transient computer-readable medium and at least one processor, the non-transient computer-readable medium being encoded with instructions and configured to store a time series of image frames, the at least one processor communicating with the non-transient computer-readable medium and being configured to execute the instructions, wherein, when executed, the instructions cause the at least one processor to determine whether one or more markers are present in an individual image frame in the sequence of image frames, output a bounding box indicating the location of the one or more markers determined to be present in the individual image frame, output a flow field for the one or more markers determined to be present in the individual image frame based at least in part on the bounding box, generate a displacement field value based at least in part on the flow field, generate a velocity vector value indicating the direction and magnitude of motion based at least in part on the flow field, calculate a displacement / velocity isotropic value by dividing the magnitude of the spatially average displacement field by the ratio of the magnitude of the displacement field to the velocity vector, and generate at least one of a mobility map or a mobility index based at least in part on the displacement / velocity isotropic value.

[0008] According to at least one example of this disclosure, a method for providing quantization of marker mobility may include receiving a sequence of image frames, determining the presence of one or more markers in individual image frames within the sequence of image frames, providing bounding boxes indicating the locations of the one or more markers determined to be present in the individual image frames, generating a flow field for the one or more markers determined to be present in the individual image frames, at least partially based on the bounding boxes or the entire frame of the sequence of image frames, generating values ​​of a displacement field, at least partially based on the flow field, generating values ​​of a velocity vector indicating the direction and magnitude of motion, calculating a displacement / velocity isotropic value by dividing the magnitude of the spatially averaged displacement field by the ratio of the magnitude of the displacement field to the velocity vector, and generating at least one of a mobility map or a mobility index, at least partially based on the displacement / velocity isotropic value. In some embodiments, the method for providing quantization of marker mobility is a computer-implemented method. In some embodiments, determining the presence of the one or more markers and providing the bounding boxes are performed by a first artificial intelligence (AI) model, and wherein generating the flow field is performed by a second artificial intelligence (AI) model. In an embodiment, the second AI model includes a FlowNet network. In some other embodiments, the method for providing quantification of marker mobility further includes: segmenting the mobility map at least in part based on the displacement / velocity isotropic values; and generating a mobility mask comprising one or more regions based at least in part on the segmentation of the mobility map. In some embodiments, the method for providing quantification of marker mobility further includes: clustering the one or more markers present in individual image frames of the sequence of image frames based on at least one of the relative or absolute movement of the one or more markers. In some embodiments, the method for providing quantification of marker mobility further includes generating at least one of a displacement map based on the values ​​of the displacement field or a velocity map based on the values ​​of the velocity vector. In some embodiments, the method for providing quantification of marker mobility includes: receiving probe movement data for an ultrasound probe from an inertial measurement unit; and excluding individual images of the image sequence associated with at least one of sudden movement or rotational movement of the ultrasound probe from the image sequence. In some embodiments, the mobility index indicates the level of a clinical condition.

[0009] According to at least one example of this disclosure, there exists a computer program product comprising a computer-readable medium. The computer-readable medium has computer-readable code implemented therein, and the computer-readable code is configured to cause, when executed by a suitable computer or processor, the computer or processor to perform a method for providing quantization of flag mobility. Attached Figure Description

[0010] Figure 1 Examples of B-mode frame sequences are shown during push-pull procedures in both endometriosis and normal conditions.

[0011] Figure 2 This is a block diagram of an ultrasound system based on the principles of this disclosure.

[0012] Figure 3 It is a schematic diagram illustrating the input and output of an artificial intelligence model based on the principles of this disclosure.

[0013] Figure 4 This is a schematic diagram of a neural network suitable for implementing an artificial intelligence model according to the principles of this disclosure.

[0014] Figure 5A An example displacement diagram based on the principles of this disclosure is shown.

[0015] Figure 5B An example velocity diagram based on the principles of this disclosure is shown.

[0016] Figure 6 An example mobility diagram based on the principles of this disclosure is shown.

[0017] Figure 7 An example of a mobility mask based on the principles of this disclosure is shown.

[0018] Figure 8 This is an example cluster diagram of pelvic organs based on the principles of this disclosure.

[0019] Figure 9 This is a block diagram of the process for training and deploying a neural network based on the principles of this disclosure.

[0020] Figure 10 It is a flowchart of a method based on the principles of this disclosure.

[0021] Figure 11 This is a block diagram illustrating an example processor based on the principles of this disclosure. Detailed Implementation

[0022] The following description of certain embodiments is merely exemplary in nature and is in no way intended to limit the invention or its application or use. In the following detailed description of embodiments of the system and method, reference is made to the accompanying drawings, which form part of this description and illustrate specific embodiments in which the described system and method can be practiced. These embodiments have been described in sufficient detail to enable those skilled in the art to practice the currently disclosed apparatus, system, and method, and it should be understood that other embodiments can be utilized and structural and logical changes can be made without departing from the spirit and scope of this disclosure. Furthermore, for clarity, detailed descriptions of certain features will not be discussed where they are obvious to those skilled in the art, so as not to obscure the description of the apparatus, system, and method. Therefore, the following detailed description is not limiting, and the scope of the system is defined only by the claims.

[0023] Organ mobility can be an indicator of the degree and / or severity of certain medical conditions. For example, ovarian mobility assessed via TVUS can be used as a “soft marker” indicating the degree and severity of occlusion in the presence of endometriosis. Ovarian fixation due to pelvic adhesions and reduced fallopian tube mobility can be characteristic of pelvic endometriosis.

[0024] Mobility assessment is typically performed during deep ultrasound examination to determine if pelvic organs are adhered to each other. For example, in cases of endometriosis, the ovaries may or may not adhere to other pelvic organs. Knowing the degree of occlusion may be necessary before proceeding with laparoscopic surgery.

[0025] Currently, there is no quantitative assessment of organ mobility. For example, in current clinical practice, the ovary is qualitatively assessed by a specialist sonographer. Ovarian mobility is diagnosed by assessing the mobility of the ovary relative to the uterus and the ipsilateral internal iliac vessels. This assessment is performed by applying gentle pressure (push-pull) to the ovary using a TVUS probe. If the ovary moves freely on the ipsilateral internal iliac vessels or the uterus under gentle pressure, it can be considered "mobile." When the ovary does not move freely relative to the uterus and / or ovarian fossa due to the presence of pelvic adhesions, it is considered "fixed." Currently, this assessment is not quantitative but is supported only by qualitative observation of the flexibility of pelvic organs during TVUS.

[0026] Figure 1Examples of B-mode frame sequences during a push-pull procedure in cases of endometriosis and normal uterus are shown. In the left column, there is severe occlusion because the cyst is attached to adjacent pelvic tissue, and the entire structure is very fixed when the sonographer attempts to push the pelvic structure to double-check its mobility. In the right column, the exact same push-pull procedure is performed on a healthy uterus, where the pelvic organs move very freely (features on the image vary in each frame).

[0027] The earlier endometriosis is detected, the earlier it can be treated. In current practice, especially in its early stages, endometriosis may be missed by inexperienced sonographers. This is a growing problem as the number of trained sonographers is outpaced by the increasing use of ultrasound in both developed and developing countries. Therefore, quantitative measurement of organ mobility (such as ovarian mobility) is desirable. Quantitative assessment of organ mobility can improve the detection and diagnosis of various medical conditions associated with organ mobility, such as endometriosis.

[0028] According to this disclosure, artificial intelligence (AI) can be used to intelligently assess organ mobility (e.g., ovarian mobility) by examining the movement of one or more organs relative to adjacent pelvic organs. As used herein, AI refers to machine learning, deep learning, and combinations thereof. The correlation of movement of pelvic landmarks in B-mode ultrasound images can be converted into an occlusion (e.g., mobility) index. The mobility index can indicate the clinical condition (e.g., endometriosis) and / or the severity / level of the clinical condition. The mobility index can be used to cluster pelvic landmarks that move together (e.g., in the presence of deep invasive pelvic endometriosis, the ovary and uterus move as a single block). Additionally or alternatively, the mobility index can be used to generate a mobility mask. The mobility mask can be displayed to the user side-by-side with the B-mode image to indicate the level of occlusion in the field of view during the organ mobility assessment procedure (e.g., by applying gentle pressure using a TVUS probe).

[0029] Figure 2A block diagram of an ultrasound imaging system 200 constructed according to the principles of this disclosure is shown. The ultrasound imaging system 200 according to this disclosure may include a transducer array 214, which may be included in an ultrasound probe 212 (e.g., an external probe or an internal probe, such as a transvaginal ultrasound (TVUS) probe). The transducer array 214 is configured to transmit ultrasound signals (e.g., beams, waves) and receive echoes in response to the ultrasound signals. Various transducer arrays can be used, such as linear arrays, curved arrays, or phased arrays. For example, the transducer array 214 can include a two-dimensional array of transducer elements (as shown), which can scan in both elevation and azimuth dimensions for 2D and / or 3D imaging. It is well known that the axial direction is perpendicular to the array plane (in the case of a curved array, the axial direction fans out), the azimuth direction is typically defined by the longitudinal dimension of the array, and the elevation direction is transverse to the azimuth direction.

[0030] Optionally, in some examples, the ultrasonic probe 212 may include an inertial measurement unit (IMU) 270. The IMU 270 may include an accelerometer, a gyroscope, a magnetometer, and / or a combination thereof. The IMU 270 can provide data related to the velocity, acceleration, rotation, angular rate, and / or orientation of the probe 212 (collectively, probe motion data).

[0031] In some embodiments, the transducer array 214 may be coupled to a microwave beamformer 116, which may be located within the ultrasonic probe 212, and the transmission and reception of signals may be controlled by the transducer elements in the array 214. In some embodiments, the microwave beamformer 216 may be controlled by active elements in the array 214 (e.g., an active subset of the elements of the array that defines the active aperture at any given time) to control the transmission and reception of signals.

[0032] In some embodiments, the microwave beamformer 216 may be coupled, for example, via a probe cable or wirelessly, to a transmit / receive (T / R) switch 218, which switches between transmitting and receiving and protects the main beamformer 222 from high-energy transmitted signals. In some embodiments, such as in a portable ultrasound system, the T / R switch 218 and other components of the system may be included in the ultrasound probe 212, rather than in an ultrasound system base that may house image processing electronics. An ultrasound system base typically includes software and hardware components, including circuitry for signal processing and image data generation, and executable instructions (e.g., processing circuitry 250 and user interface 224) for providing a user interface.

[0033] The ultrasonic signal transmission from transducer array 214, under the control of microwave beamformer 216, is guided by transmission controller 220, which can be coupled to T / R switch 218 and main beamformer 222. Transmission controller 220 can control the direction in which the beam is manipulated. The beam can be manipulated linearly from transducer array 214 (perpendicular to transducer array 214) or at different angles to obtain a wider field of view. Transmission controller 220 can also be coupled to user interface 224 and receive input from user operation of user control keys. User interface 224 may include one or more input devices, such as control panel 252, which may include one or more mechanical controls (e.g., buttons, encoders, etc.), touch-sensitive controls (e.g., trackpads, touchscreens, etc.), and / or other known input devices.

[0034] In some embodiments, the partially beamformed signal generated by microwave beamformer 216 can be coupled to main beamformer 222, in which partially beamformed signals from individual transducer elements can be combined into a fully beamformed signal. In some embodiments, microwave beamformer 216 is omitted, and transducer array 214 is under the control of main beamformer 222, which performs beamforming of all signals. In embodiments with and without microwave beamformer 216, the beamformed signals from main beamformer 222 are coupled to processing circuitry 250, which may include one or more processors (e.g., signal processor 226, mode-B processor 228, Doppler processor 260, and one or more image generation and processing units 268) configured to generate ultrasound images based on the beamformed signals (e.g., beamformed RF data).

[0035] Signal processor 226 can be configured to process received beamformed RF data in various ways, such as bandpass filtering, decimation, I and Q component separation, and harmonic signal separation. Signal processor 226 can also perform additional signal enhancement, such as speckle suppression, signal recombination, and noise cancellation. The processed signals (also referred to as I and Q components or IQ signals) can be coupled to additional downstream signal processing circuitry for image generation. The IQ signals can be coupled to multiple signal paths within the system, each of which can be associated with a specific arrangement of signal processing components suitable for generating different types of image data, such as B-mode image data and Doppler image data. For example, the system may include a B-mode signal path 258 that couples signals from signal processor 226 to B-mode processor 228 for generating B-mode image data.

[0036] The B-mode processor is capable of employing amplitude detection to image structures within the body. Signals generated by the B-mode processor 228 can be coupled to a scan converter 230 and / or a multi-plane reformer 232. The scan converter 230 can be configured to arrange echo signals into a desired image format based on their spatial relationship of reception. For example, the scan converter 230 can arrange echo signals into a two-dimensional (2D) fan-shaped format, or a pyramidal or other shaped three-dimensional (3D) format. The multi-plane reformer 232 is capable of converting echoes received from points in a common plane within a volumetric region of the body into an ultrasound image (e.g., a B-mode image) of that plane, for example, as described in U.S. Patent No. 6,443,896 (Detmer). In some embodiments, the scan converter 230 and the multi-plane reformer 232 can be implemented as one or more processors.

[0037] Volumetric renderer 234 can generate an image of a 3D dataset as viewed from a given reference point (also referred to as a projection, drawing, or rendering result), for example, as described in U.S. Patent No. 6,530,885 (Entrekin et al.). In some embodiments, volumetric renderer 234 can be implemented as one or more processors. Volumetric renderer 234 can generate drawings (e.g., positive or negative drawings) using any known or future known techniques (e.g., surface drawing and maximum intensity drawing).

[0038] In some embodiments, the system may include a Doppler signal path 262 that couples the output from signal processor 226 to Doppler processor 260. Doppler processor 260 may be configured to estimate the Doppler frequency shift and generate Doppler image data. The Doppler image data may include color data, which is then overlaid with B-mode (i.e., grayscale) image data for display. Doppler processor 260 may be configured, for example, to use a wall filter to filter out unwanted signals (i.e., noise or clutter associated with non-moving tissue). Doppler processor 260 may also be configured to estimate velocity and power using known techniques. For example, the Doppler processor may include a Doppler estimator such as an autocorrelation function, wherein the velocity (Doppler frequency, spectral Doppler) estimation is based on the independent variable of a lag-1 autocorrelation function, while the Doppler power estimation is based on the magnitude of a lag-0 autocorrelation function. Motion can also be estimated using known phase-domain (e.g., parametric frequency estimators such as MUSIC, ESPRIT, etc.) or time-domain (e.g., cross-correlation) signal processing techniques. Instead of a velocity estimator, or in addition to a velocity estimator, other estimators relating to the temporal or spatial distribution of velocity can be used, such as an acceleration estimator, or an estimator of the temporal and / or spatial velocity derivatives. In some embodiments, the velocity and / or power estimates may undergo further thresholding detection to further reduce noise, and also undergo segmentation and post-processing (e.g., filling and smoothing). The velocity and / or power estimates can then be mapped to a desired display color gamut based on the color map. The color data (also referred to as Doppler image data) can then be coupled to a scan converter 230, in which the Doppler image data can be converted to a desired image format and superimposed on a B-mode image of the tissue structure to form a color Doppler or power Doppler image. In some examples, power estimation (e.g., lag-0 autocorrelation) can be used to mask or segment the flow in the color Doppler (e.g., velocity estimation) before superimposing the color Doppler image onto the B-mode image.

[0039] Output from the scan converter 230, the multiplane reformer 232, and / or the volume plotter 234 can be coupled to the image processor 236 for further enhancement, buffering, and temporary storage before display on the image display 238. The graphics processor 240 can generate graphic overlays to be displayed along with the images. These overlays can include, for example, standard identification information (e.g., patient name), the date and time of the image, imaging parameters, etc. For these purposes, the graphics processor can be configured to receive input from the user interface 224, such as a typed patient name or other annotations. The user interface 224 can also be coupled to the multiplane reformer 232 for selecting and controlling the display of multiple multiplane reformatted (MPR) images.

[0040] System 200 may include local memory 242. Local memory 242 may be implemented as any suitable non-transient computer-readable medium (e.g., flash drive, disk drive). Local memory 242 may store data generated by system 200, including ultrasound images, executable instructions, imaging parameters, training datasets, or any other information required for the operation of system 200. In some examples, local memory 242 may include multiple memories that may be the same or different types. For example, local memory 242 may include dynamic random access memory (DRAM) and flash memory.

[0041] As mentioned above, system 200 includes a user interface 224. User interface 224 may include a display 238 and a control panel 252. Display 238 may include a display device implemented using various known display technologies (e.g., LCD, LED, OLED, or plasma display technologies). In some embodiments, display 238 may include multiple displays. Control panel 252 may be configured to receive user input (e.g., check type, imaging parameters). Control panel 252 may include one or more hardware controls (e.g., buttons, knobs, dial pads, encoders, mice, trackballs, etc.). In some embodiments, control panel 252 may additionally or alternatively include software controls (e.g., GUI control elements or simply GUI controls) provided on a touch-sensitive display. In some embodiments, display 238 may be a touch-sensitive display that includes one or more software controls of control panel 252.

[0042] In some embodiments, Figure 2 The various components shown can be combined. For example, the multi-plane reformer 232 and the volume renderer 234 can be implemented as a single processor. In some embodiments, Figure 2 The various components shown can be implemented as separate components. For example, signal processor 226 can be implemented as a separate signal processor for each imaging mode (e.g., B-mode, Doppler). In another example, image processor 236 can be implemented as a separate processor for different tasks and / or parallel processing of the same task. In some embodiments, Figure 2One or more of the processors shown may be implemented as general-purpose processors and / or microprocessors configured to perform specific tasks. In some examples, the processor may be configured by providing instructions for the task from a non-transient computer-readable medium (e.g., from local memory 242). The instructions may then be executed by the processor. In some embodiments, one or more of the processors may be implemented as dedicated circuitry. In some embodiments, one or more of the processors (e.g., image processor 236) may be implemented using one or more graphics processing units (GPUs).

[0043] The principles of this disclosure will be described with reference to the detection and / or grading of ovarian mobility and endometriosis. However, this is merely for illustrative purposes, and the principles of this disclosure can be applied to the detection and / or grading of mobility in other organs and other conditions. For example, the principles of this disclosure can also be applied to the mobility of the bladder and / or intestine. Other conditions may include scarring and / or stretching of supporting tissues (e.g., fascia, ligaments). In another example, the mobility of an implantable device can be assessed to determine appropriate placement / implantation.

[0044] During a mobility examination, the user can apply a push-pull motion to the ultrasound probe 212 to apply pressure to one or more organs. For example, during a transvaginal examination, the TVUS probe can be inserted and removed (or partially removed) to apply pressure to pelvic organs such as the ovaries. The pressure may cause temporary displacement of the organs. The ultrasound system 200 can acquire time-series image frames via the transducer array 214 of the probe 212 during the mobility examination. The image frames may include one or more organs in the field of view of the transducer array 214. For example, in an ovarian mobility examination, one or both ovaries, at least a portion of the uterus, at least a portion of the intestine, and / or at least a portion of the internal iliac vessels may be present in the image frame or at least some of the image frames.

[0045] The image frame sequence can then be analyzed by one or more processors (such as image processor 236). In some examples, image processor 236 may include any one or more machine learning, AI algorithms and / or multiple neural networks (collectively, AI models). In some examples, image processor 236 may include one or more of deep neural networks (DNNs), convolutional neural networks (CNNs), recurrent neural networks (RNNs), autoencoder neural networks and / or single-step detectors to segment and identify organs in image frames (e.g., sign detection), evaluate inter-frame displacement and movement of organs (e.g., motion analysis), and / or generate clusters of organ displacements (e.g., cluster analysis). AI models can be implemented as hardware (e.g., neurons of a neural network are represented by physical components) and / or software (e.g., neurons and paths are implemented in a software application manner) components. The neural network according to embodiments of this disclosure can use a wide variety of topologies and learning algorithms to train the neural network to produce the desired output. For example, a software-based neural network can be implemented using a processor configured to execute instructions (e.g., a single or multi-core CPU, a single GPU or GPU cluster, or a processor arranged for parallel processing). These instructions can be stored in a computer-readable medium and, when executed, cause the processor to perform a trained algorithm. In some examples, image processor 236 can be combined with other image processing methods (e.g., segmentation, histogram analysis) to implement AI.

[0046] In various examples, the AI ​​model can be trained using a wide range of currently known or subsequently developed learning techniques to obtain a neural network (e.g., a trained algorithm or a hardware-based system of nodes) configured to analyze input data in the form of ultrasound images, measurements, and / or statistics. In some embodiments, the AI ​​model can be trained statistically. That is, the AI ​​model can be trained using a dataset and deployed on system 200 and implemented by image processor 236. In some embodiments, the AI ​​model can be trained dynamically. In these examples, the AI ​​model can be trained using an initial dataset and deployed on system 200. However, the AI ​​model can continue to be trained and modified based on ultrasound images acquired by the system after it has been deployed on system 200 and implemented by image processor 236.

[0047] In some examples, the output of one or more AI models can be used to generate displacement maps, velocity maps, mobility maps, and / or mobility masks. These maps can be used to determine the isotropic and / or anisotropic nature of motion / displacement. The determined isotropic and / or anisotropic nature can be used to generate an organ mobility (e.g., occlusion) index (OMI). In some examples, the maps, the determination of isotropic and / or anisotropic nature, and / or the OMI can be generated by image processor 236.

[0048] Optionally, in some examples, probe movement data from IMU 270 may be provided to one or more processors, such as image processor 236, B-mode processor 228, and / or scan converter 230. The probe movement data may be used to determine which image frames of the sequence should be provided to the AI ​​for analysis. For example, image frames containing data associated with impaired motion patterns due to sudden / incorrect probe movement may be excluded from the analysis. Additionally or alternatively, the probe movement data may be used to provide user feedback on the appropriate execution of probe movement for a push-pull technique in mobility testing.

[0049] For consistent mobility measurements, probe 212 should exert a significant push on the surrounding tissue, while simultaneously, probe 212 should not rotate. Rotation can lead to out-of-plane motion. Out-of-plane motion can introduce inaccuracies in motion tracking in AI, for example, due to the faster decorrelation rate of the structure being tracked. Furthermore, if the organ structure changes shape in the elevation dimension, out-of-plane motion may create a false impression of a motion pattern even without actual motion. By using data from IMU 270 to differentiate push-pull movements from probe rotation to selectively process data and / or provide the user with feedback on the correct execution of push-pull movements, operator variability during mobility examinations can be reduced.

[0050] Figure 3 This is a schematic diagram illustrating the inputs and outputs of an artificial intelligence model based on the principles of this disclosure. In some examples, the AI ​​model may be implemented by one or more processors included in an ultrasound imaging system (such as ultrasound imaging system 200).

[0051] The time series of image frames 300 acquired during mobility examination can be provided to a marker detection and / or organ classification AI model 304 that detects organs and / or other physiological features (e.g., blood vessels) (collectively, markers) within the sequence of image frames 300. The sequence of image frames 300 may have already been acquired by an ultrasound probe (such as ultrasound probe 212). Optionally, in some examples, probe motion data 302 can be used to block / remove individual frames from the sequence of image frames 300 before they are provided to the marker detection AI model 304. For example, image frames associated with sudden and / or incorrect (e.g., rotational) motion can be excluded from the sequence of image frames 300 provided to the marker detection AI model 304. In some examples, probe motion data 302 can be provided by an IMU (such as IMU 270) associated with the probe.

[0052] In some examples, the sign detection AI model 304 may include multiple deep learning networks, where the output from one network can be used to initiate the training of a new deep learning network designed to learn different tasks, meaning that multiple networks can be used to obtain the final output of the sign detection AI model 304. Simultaneous detection and segmentation (SDS) (YOLOv2, MobileNet SSD lite, SSD: Single Shot MultiBox Detector networks) is an example of a suitable deep learning network that can be trained to perform sign detection in a sequence of image frames 300. In some examples, the output 306 of the sign detection AI model 304 may include the location of one or more predicted bounding boxes (BBs) in image coordinates (xo, zo, width of BB, height of BB), where each BB is associated with a different sign (e.g., uterus, ovary, and iliac vessels). In some examples, the output 306 of the sign detection AI model 304 may be a sequence of image frames 300, where the bounding boxes are labeled, linked, and / or otherwise associated with appropriate image frames in the sequence of image frames 300. Alternatively, in some examples, sign detection can be performed using other image segmentation techniques (e.g., edge detection, histogram analysis) to generate bounding boxes.

[0053] The output 306 from the sign detection AI model 304 can be provided to the motion analysis AI model 308. In an example where the output 306 may include the coordinates of the bounding box and / or the entire corresponding image frame, a sequence of image frames 300 may also be provided to the motion analysis model 308. The motion analysis AI model 308 may include a deep learning network trained to extract optical flow, which is the motion pattern of one or more signs detected by the sign detection AI model 304. In other words, the motion analysis AI model 308 determines the 2D displacement of organs and / or other physiological features across temporally adjacent image frames. (Refer to...) Figure 4 A more detailed description may include a suitable deep learning network in the motion analysis model 308. Alternatively or concurrently, motion analysis may be performed using other techniques such as speckle analysis.

[0054] The output generated by the motion analysis AI model 308 can be used to generate displacement values ​​and velocity vector values ​​for the displacement field. These values ​​can be used to generate displacement map 310 and / or velocity map 312. Displacement map 310 provides a graphical depiction of the marked displacement field, and velocity map 312 provides a graphical depiction of the magnitude and direction of movement of individual pixels in individual image frames of image sequence 300. (Reference) Figure 5A Example displacement diagrams are shown and described, and references are provided. Figure 5B Example velocity diagrams are shown and described.

[0055] Displacement and velocity data from displacement diagram 310 and velocity diagram 312 can be analyzed separately to quantify displacement / velocity isotropy. The degree of isotropy is the ratio of the spatially average displacement / velocity vector magnitude to the displacement / velocity vector magnitude. The quantified isotropy values ​​can be used to generate mobility diagram 318. (Reference) Figure 6 Example mobility graphs are shown and described. In some examples, mobility graph 318 may be generated by one or more processors, such as image processor 236.

[0056] The isotropic data provided for generating the mobility map 318 can be used to generate a quantitative mobility index (e.g., an occlusion index) 320. Free and fixed tissue can be determined based on the absolute and relative movement of landmarks (e.g., pelvic structures) determined from the mobility analysis AI model 308, and bounding boxes for landmark detection from the landmark detection AI model 304. For example, for ovarian mobility, if the TVUS B-mode image is 300×300 in size and landmarks on adjacent B-mode images (frames N and N+1) occupy 75% of the field of view, how much tissue is freely moving or fixed can be extracted from displacement correlation. In some examples, the mobility index 320 can be the percentage of freely moving tissue relative to total tissue. In these examples, a high percentage can indicate high mobility, and a low percentage can indicate low mobility. In some applications, the mobility index 320 can be used to detect and / or classify medical conditions. For example, a condition can be determined when the mobility index 320 is equal to or below a threshold. In another example, the situation can be classified into a certain level based on the range of values ​​for the mobility index 320. In some examples, the mobility index 320 can be generated by one or more processors (such as image processor 236).

[0057] The isotropic data provided for generating mobility map 318 can be used to generate mobility mask 322. Mobility mask 322 can be a user-friendly depiction of the data from mobility map 318, superimposed on at least one image in a sequence of image frames 300. Mobility mask 322 can depict different regions based on mobility data from mobility map 318 using contours, colors, and / or shading. (Reference) Figure 7 Exemplary mobility masks are shown and described. In some examples, mobility mask 322 may be generated by one or more processors, such as image processor 236.

[0058] Optionally, the output of the motion analysis AI model 308 and the output of the sign detection AI model 304 can be provided to the clustering analysis AI model 314. Image frame 300 can also be provided to the clustering analysis AI model 314 when output 306 does not include image frame 300. The clustering analysis AI model 314 can analyze the spatial coherence of temporally adjacent frame movements of velocity vectors across a sequence of image frames 300. The clustering analysis AI model 314 can be trained to cluster signs based on their absolute and relative movements. In some examples, the clustering analysis AI model 314 includes a deep learning model to determine how many unique groups are included in the trajectory distribution. In some examples, the model can use the G-means clustering algorithm, which can automatically discover the number of clusters using statistical tests. In other examples, other algorithms and / or models can be used. In some examples, trajectory clustering can be used to fade movement at the boundaries of pelvic signs. The clustering analysis AI model 314 can output a cluster map 316. Figure 8 The diagram shows an example clustering diagram of pelvic organs.

[0059] In some examples, Figure 3 All the AI ​​models shown can be implemented by different processors. In some examples, Figure 3 One or more of the AI ​​models shown can be implemented by the same processor. In some examples, one or more of the AI ​​models can be implemented by multiple processors. In some examples, mobility graph 318, mobility index 320, and / or mobility mask 322 can be generated by the same or different processor(s) used to implement at least one AI model.

[0060] Figure 4 This is a schematic diagram illustrating a neural network suitable for implementing an artificial intelligence model according to the principles of this disclosure. In some examples, the motion analysis model 308 may include one or more convolutional neural networks. In some examples, one or more convolutional neural networks in the convolutional neural network may be a FlowNet network 400. FlowNet is a convolutional neural network designed to learn correlations between adjacent frames and depends on the displacement of an object (e.g., a sign) across frames. The input to the FlowNet network 400 may be consecutive image frames 402 (e.g., a time series of image frames 300) and object position information (e.g., output 306 from the sign detection AI model 304). The FlowNet network 400 may output a predicted flow field 404 for objects within image frames 402. The flow field 404 may also be interchangeably referred to as a displacement field. The flow field 404 may allow for the analysis of absolute motion and / or global movement associated with objects within frame 402. In some examples, the flow field 404 may be used to generate displacement and / or velocity maps, such as displacement map 310 and velocity map 312, respectively.

[0061] exist Figure 4 In the example FlowNet network 400 shown, consecutive image frames are input into a single, identical processing stream. Each processing stream includes one or more convolutional layers. After passing through the individual processing stream, the image frames are combined by a correlation layer that compares the feature maps output by the last convolutional layer of the processing stream. The combined image frames are then passed through one or more convolutional layers to generate a predicted flow field 404. Optionally, the output may be passed through a variational refinement layer to generate the predicted flow field 404. Additional details and alternative structures of the FlowNet network 400 can be found in A.

[0062] The paper was found in "FlowNet: Learning Optical Flow with Convolutional Networks" by Dosovitskiy et al. (2015 IEEE International Conference on Computer Vision (ICCV), Santiago, 2015, pp. 2758-2766).

[0063] Flow field 404 can utilize, for example Figure 4 The FlowNet network 400 shown is used for generation. Alternatively, other methods such as speckle tracking can be used to identify the motion of the markers in image frame 402. In some examples, baseline truth generation of the flow pattern can be obtained by generating the flow graph using conventional inter-frame tracking methods, such as speckle tracking used to train the FlowNet network 400.

[0064] Figure 5A Example displacement maps according to the principles of this disclosure are shown. In some examples, displacement maps can be generated based on the values ​​of a displacement field generated by normalizing a flow field output by a motion analysis AI model (such as motion analysis AI model 308), e.g., flow field 404. Displacement maps 500 and 502 are generated based on data collected from ovarian mobility examinations of healthy subjects, and displacement maps 504 and 506 are generated based on data collected from subjects with endometriosis (e.g., unhealthy subjects). In displacement maps 500 and 502 of healthy subjects, the flow field pattern is more heterogeneous than in displacement maps 504 and 506 of unhealthy subjects, indicating that the displacement is very chaotic. Conversely, for unhealthy subjects, where pelvic tissue moves with high correlation entirely in the same direction (as shown by the large areas of uniform shading in displacement maps 505 and 506), the occlusion appears to be severe. Clearly, the inter-frame displacement is more chaotic in healthy subjects.

[0065] Figure 5BExample velocity maps according to the principles of this disclosure are shown. A velocity map can provide a vector illustrated as jittery arrows superimposed on an image frame, where the vector indicates the direction and magnitude of motion of pixels in the image frame. Velocity maps can be generated based on the values ​​of velocity vectors, which are based on tracking the motion of markers in a sequence of image frames, for example, based on a flow field output by a motion analysis AI model (such as motion analysis AI model 308). Velocity maps 508, 510, 516, and 518 are generated based on data collected from ovarian mobility examinations of healthy subjects, and velocity maps 512 and 514 are generated based on data collected from subjects with endometriosis (e.g., unhealthy subjects). In some applications, a healthy condition may be associated with a disordered or random distribution of velocity vectors (e.g., anisotropy), while a more isotropic distribution may be associated with an unhealthy condition. In some applications, velocity maps can help users understand the degree of tissue occlusion. For example, in velocity maps 512 and 514, in unhealthy conditions, the jittery arrows all move in almost the same direction (up and down). Conversely, for healthy objects, there is a more chaotic movement, where the velocity vector points in various different directions as shown in velocity diagrams 508, 510, 516, and 518.

[0066] Used to generate Figure 5A and 5B The displacement and velocity data in the displacement and velocity diagram shown can be used to generate a quantitative measure of organ mobility called displacement / velocity isotropy. Displacement / velocity isotropy is calculated as the magnitude of the spatially averaged displacement / velocity vector versus the magnitude of the displacement / velocity vector (e.g., from...). Figure 5A The average value of the data in the displacement diagram shown and from Figure 5B The ratio of the average value of the velocity data shown in the velocity plot. Displacement / velocity isotropic is a dimensionless parameter and can be used to quantify the displacement / velocity patterns of moving tissues.

[0067] Figure 6 Example mobility diagrams according to the principles of this disclosure are shown. Mobility diagrams 600 and 602 are generated based on data collected from healthy subjects during mobility examinations, and mobility diagrams 604 and 606 are generated based on data collected from subjects suffering from endometriosis (e.g., unhealthy subjects). Mobility diagrams 600, 602, 604, and 606 are based on data used to generate displacement and velocity diagrams (such as... Figure 5A and 5B The displacement / velocity graph shown is a graphical representation of the displacement / velocity isotropic data generated from the data. In some examples, the number of displacement / velocity isotropic values ​​can be equal to the number of pixels in the sequence of image frames from which the analyzed image frames are derived.

[0068] In healthy objects, tissue motion is highly disordered in multiple directions. As seen in mobility diagrams 600 and 602, the spatial mean displacement / velocity vectors cancel each other out, resulting in very low isotropy / high anisotropy (e.g., values ​​close to zero). In some applications, anisotropic motion can be associated with and / or referred to as "mobility." In unhealthy objects, tissue motion is much more uniform in almost the same directions. In mobility diagrams 604 and 606, the spatial mean displacement / velocity vectors are not canceled out at all, resulting in very high isotropy (e.g., values ​​close to one).

[0069] Although displacement / velocity isotropy is a quantitative measure of mobility, mobility maps (such as...) Figure 6 The diagrams shown may be difficult for users (especially novice users) to interpret. Therefore, in some applications, a more user-friendly depiction of mobility can be provided.

[0070] Figure 7 Examples of mobility masks according to the principles of this disclosure are shown. Mobility masks 700, 702 and 704 are generated based on data collected from subjects with endometriosis (e.g., unhealthy subjects) during mobility examinations, and mobility masks 706, 708 and 710 are generated based on data collected from healthy subjects. Figure 7 The mobility mask shown can be obtained by analyzing the corresponding mobility graph (such as...). Figure 6 The regions (as shown in the diagram) are segmented to generate the data. Regions with equal and / or similar displacement / velocity isotropic values ​​can be grouped into different regions, such as regions 712, 714, and 716 indicated in mobility mask 704. In some examples, regions 712, 714, and 716 can be based on a threshold and / or a range of displacement / velocity isotropic values. In some examples, regions with different displacement / velocity isotropic values ​​or ranges thereof can be displayed with different colors, patterns, and / or intensities. In some examples (such as...) Figure 7 In the example shown, a motion mask can be overlaid on one or more image frames from a sequence of image frames (e.g., a sequence of image frames 300). In some examples, the motion mask can be provided side-by-side with a corresponding motion map on a display (such as display 238).

[0071] Information displayed on a mobility mask can provide more easily interpreted information related to the degree of displacement of landmarks in the field of view (compared to a mobility map) and isotropic / anisotropic information related to the distribution of velocity vectors. For example, a freely moving ovary, as shown in mobility masks 706, 708, and 710, shows a low degree of occlusion of pelvic landmarks. In the presence of tissue occlusion and endometriosis, such as in mobility masks 700, 702, and 704, the ovary may appear “fixed,” and some or all landmarks may move together as a block, showing very high displacement isotropic velocity vectors.

[0072] In some applications, visual feedback is generated to the user indicating the degree of occlusion (e.g., using... Figure 7 The motion mask shown can help users visualize the onset of medical conditions (such as endometriosis) related to organ movement at an early stage.

[0073] As mentioned earlier, mobility is based on the amount of “free-moving” organization compared to the amount of “fixed” organization. For example, a mobility index could be based on the percentage of pixels with displacement / velocity isotropic values ​​equal to or below a threshold compared to the total number of pixels. Figure 7 The unhealthy objects on the left can have a mobility index of 15%, while Figure 7 The healthy object on the right can have a mobility index of 75%. In some examples, the mobility index can be combined with a mobility mask and / or mobility... Figure 1 The mobility index may be displayed along with the image frame, without showing the mobility mask and / or mobility map. In some examples, the mobility index may be displayed along with the image frame, without showing the mobility mask and / or mobility map.

[0074] Optionally, the flow field (e.g., flow field 404) and / or the displacement and velocity maps generated based on the flow field (e.g., Figure 5A and 5B The data (displacement and velocity diagrams shown) can be analyzed using clustering analysis AI models (such as clustering analysis AI model 314). Figure 8 This is an example clustering diagram of pelvic organs based on the principles of this disclosure. Clustering diagram 800 clusters pelvic organs (e.g., markers) by displacement. Clustering diagram 800 can provide insight into which organs are fused and / or affect the mobility of other organs.

[0075] Figure 9 A block diagram illustrating the process for training and deploying a neural network according to the principles of this disclosure is shown. Figure 9 The process shown can be used to train AI models implemented by medical imaging systems, such as... Figure 3 The AI ​​model shown. Figure 9The left side (Phase 1) illustrates the training of the AI ​​model. To train the AI ​​model, a training set consisting of multiple instances, including the input array and output classification, can be presented to one or more training algorithms for the AI ​​model (e.g., AlexNet training algorithms, such as Krizhevsky, A., Sutskever, I., and Hinton, GE's "ImageNet Classification with Deep Convolutional Neural Networks").

[0076] (As described in NIPS 2012, or its descendants). Training may involve selecting a starting architecture 912 and preparing training data 914. The starting architecture 912 may be a blank architecture (e.g., an architecture with a defined layer and node arrangement but without any previously trained weights) or a partially trained model (e.g., a pre-trained network), which can then be further customized for ultrasound image classification. The starting architecture 912 (e.g., blank weights) and training data 914 are provided to a training engine 910 (e.g., an ADAM optimizer) for training the model. After a sufficient number of iterations (e.g., when the model performs consistently within acceptable error), model 920 is considered trained and ready for deployment. Figure 9 The intermediate (stage 2) diagram illustrates this situation. Figure 9 On the right side (or in stage 3), (via inference engine 930) a trained model 920 is applied to analyze new data 932, which is data not presented to the model during the initial training (in stage 1). For example, the new data 932 may include unknown images, such as live ultrasound images acquired during patient scanning (e.g., pelvic images during mobility assessment). The trained model 920, implemented via engine 930, is used to classify the unknown images based on the training of model 920 to provide output 934 (e.g., generating bounding boxes, flow fields, cluster maps). Output 934 can then be used by the system in subsequent processes 940 (e.g., generating displacement and / or velocity maps, generating mobility maps and / or masks, generating OMIs).

[0077] In embodiments where a trained model 920 is used to implement a neural network executed by a processor (such as image processor 236), the initial architecture may be a convolutional neural network or a deep convolutional neural network initial architecture, which may be trained to detect markers, analyze movement, and / or perform clustering analysis of markers. Training data 914 may include multiple (hundreds, often thousands, or even more) annotated / labeled images (also referred to as training images). It should be understood that training images do not need to include complete images generated by the imaging system (e.g., representing the complete field of view of an ultrasound probe or the entire MRI volume), but may include slices or portions of images, such as those including organs of interest.

[0078] In various embodiments, the trained AI model may be implemented at least in part in a computer-readable medium comprising executable instructions run by a processor (e.g., image processor 236).

[0079] In some examples, to obtain baseline truth for detecting signs in order to train a sign detection AI model (such as sign detection AI model 304), a trained sonographer can annotate B-mode images in a sequence of B-mode images by placing bounding boxes around each organ (e.g., a sign) present in the field of view (e.g., visible in the image frame). In some applications, such as for endometriosis detection, this labeling procedure can be performed on each TVUS B-mode frame acquired during a push-pull mobility assessment procedure. In some examples, organs labeled with bounding boxes from this set of training images can also be used to generate a baseline truth optical flow mask to train a motion analysis AI model, such as motion analysis AI model 308.

[0080] Figure 10 This is a flowchart of a method based on the principles of this disclosure. In some examples, method 1000 may be wholly or partially comprised of an ultrasound imaging system (such as...) Figure 2 The system 200 shown is used for execution. In some examples, method 1000 may be executed wholly or partially by at least one processor of the ultrasound imaging system (such as image processor 236). Method 1000 can provide quantification of marker mobility. Markers may include organs, blood vessels, other physiological features, and / or implanted devices.

[0081] As indicated at box 1002, at least one processor may receive a sequence of image frames. In some examples, the image frames may be B-mode image frames. In some examples, the sequence may be a time series. As indicated at boxes 1004 and 1006, at least one processor may determine the presence of one or more flags in an individual image frame within the sequence of image frames and provide bounding boxes indicating the locations of the one or more flags determined to be present in the individual image frame. In some examples, determining the presence of one or more flags and providing bounding boxes is performed by an AI model (such as flag detection AI model 304) implemented by one or more processors.

[0082] At least one processor can generate flow fields based at least partially on bounding boxes or alternatively on the entire frame for one or more markers determined to be present in an individual image frame, as indicated by box 1008. In some examples, the generation of flow fields is performed by an AI model (such as motion analysis AI model 308) implemented by at least one processor. In some examples, the AI ​​model includes a FlowNet network, such as FlowNet network 400.

[0083] At least one processor can generate, at least in part, a displacement map including values ​​of the displacement field and a velocity map including values ​​of a velocity vector indicating the direction and magnitude of motion, as indicated by boxes 1010 and 1012, based on the flow field.

[0084] As indicated in box 1014, at least one processor can calculate the displacement / velocity isotropic value. In some examples, this value can be calculated by dividing the magnitude of the spatially averaged displacement field by the ratio of the magnitude of the displacement field to the velocity vector.

[0085] At least one processor can generate a mobility map based at least in part on displacement / velocity isotropic values, as indicated in box 1016.

[0086] Optionally, in some examples, such as indicated in box 1018, at least one processor may calculate the mobility index based on an analysis of the displacement / velocity isotropic values.

[0087] Optionally, in some examples, at least one processor may segment the mobility map at least partially based on displacement / velocity isotropic values, and generate a mobility mask comprising one or more regions based at least partially on the segmentation of the mobility map. The mobility mask can be associated with mobility... Figure 1 It is displayed and / or overlaid on image frames in an image sequence of image frames.

[0088] Optionally, at least one processor may cluster one or more labels present in individual image frames within a sequence of image frames based on at least one of the relative or absolute movements of one or more labels. In some examples, clustering is performed by an AI model (such as cluster analysis AI model 314) implemented by at least one processor. In some examples, the AI ​​model includes the G-means clustering algorithm.

[0089] Optionally, in some examples, at least one processor may receive probe movement data of an ultrasound probe (e.g., ultrasound probe 212) from an inertial measurement unit (e.g., IMU 270) and exclude individual images from the image sequence that are associated with at least one of the sudden movement or rotational movement of the ultrasound probe.

[0090] Figure 11 This is a block diagram of an example processor 1100 based on the principles of this disclosure. Processor 1100 can be used to implement one or more processors and / or controllers described herein, for example, Figure 2 The image processor 236 and / or shown in the figure Figure 2 Any other processor or controller shown. Processor 1100 may be any suitable processor type, including but not limited to microprocessors, microcontrollers, digital signal processors (DSPs), field-programmable arrays (FPGAs) (wherein the FPGA has been programmed to form a processor), graphics processing units (GPUs), application-specific integrated circuits (ASICs) (wherein the ASIC has been designed to form a processor), or combinations thereof.

[0091] Processor 1100 may include one or more cores 1102. Core 1102 may include one or more arithmetic logic units (ALUs) 1104. In some embodiments, in addition to or in place of ALU 1104, core 1102 may include a floating-point logic unit (FPLU) 1106 and / or a digital signal processing unit (DSPU) 1108.

[0092] Processor 1100 may include one or more registers 1112 communicatively coupled to core 1102. Registers 1112 may be implemented using dedicated logic gates (e.g., flip-flops) and / or any memory technology. In some embodiments, registers 1112 may be implemented using static memory. Registers may provide data, instructions, and addresses to core 1102.

[0093] In some embodiments, processor 1100 may include one or more horizontal cache memories 1110 communicatively coupled to core 1102. Cache memory 1110 may provide computer-readable instructions for execution to core 1102. Cache memory 1110 may provide data for processing by core 1102. In some embodiments, the computer-readable instructions may have already been provided to cache memory 1110 by local memory (e.g., local memory attached to external bus 1116). Cache memory 1110 may be implemented using any suitable cache memory type, such as metal-oxide-semiconductor (MOS) memory, such as static random access memory (SRAM), dynamic random access memory (DRAM), and / or any other suitable memory technology.

[0094] Processor 1100 may include controller 1114, which can control other processors and / or components included in the system (e.g., Figure 2 The control panel 252 and scan converter 230 shown in the figure) are inputs to the processor 1100 and / or from the processor 1100 to other processors and / or components included in the system (e.g., Figure 2 The outputs of the display 238 and volume plotter 234 are shown in the diagram. The controller 1114 can control the data paths in the ALU 1104, FPLU 1106, and / or DSPU 1108. The controller 1114 can be implemented as one or more state machines, data paths, and / or dedicated control logic. The gates of the controller 1114 can be implemented as stand-alone gates, FPGAs, ASICs, or any other suitable technology.

[0095] Register 1112 and cache memory 1110 can communicate with controller 1114 and core 1102 via internal connections 1120A, 1120B, 1120C and 1120D. The internal connections can be implemented as a bus, multiplexer, cross switch and / or any other suitable connection technology.

[0096] Inputs and outputs for processor 1100 may be provided via bus 1116, which may include one or more conductive lines. Bus 1116 may be communicatively coupled to one or more components of processor 1100, such as controller 1114, cache memory 1110, and / or register 1112. Bus 1116 may be coupled to one or more components of the system, such as the previously mentioned display 238 and control panel 252.

[0097] Bus 1116 can be coupled to one or more external memories. The external memories may include read-only memory (ROM).

[0098] 1132. ROM 1132 may be a masked ROM, electrically programmable read-only memory (EPROM), or any other suitable technology. External memory may include random access memory (RAM) 1133. RAM 1133 may be static RAM, battery-powered static RAM, dynamic RAM (DRAM), or any other suitable technology. External memory may include electrically erasable programmable read-only memory (EEPROM) 1135. External memory may include flash memory 1134. External memory may include magnetic storage devices, such as disks 1136. In some embodiments, external memory may be included in a system (such as...) Figure 2 In the ultrasound imaging system 200 shown, for example, local memory 242.

[0099] The systems, methods, and apparatus disclosed herein provide a quantitative, operator-independent technique for characterizing organ mobility based on deep learning. Intelligent assessment of organ mobility can be performed by examining the movement of organs relative to adjacent organs. During a sonographer's assessment of mobility, the systems, methods, and apparatus can track global displacement and calculate the displacement / velocity isotropicity of pelvic landmarks in B-mode ultrasound. Metrics called mobility indices and / or occlusion indices can be used to cluster pelvic landmarks that move together (e.g., in the presence of deep invasive pelvic endometriosis, the ovaries and uterus move as a single mass) and generate a mobility mask. The quantification and visualization of organ mobility can help physicians make more confident diagnoses of certain medical conditions, such as endometriosis.

[0100] While the examples described herein discuss the processing of ultrasound image data, it should be understood that the principles of this disclosure are not limited to ultrasound and can be applied to image data from other modalities such as magnetic resonance imaging and computed tomography.

[0101] In various embodiments that implement components, systems, and / or methods using programmable devices (e.g., computer-based systems or programmable logic units), it should be understood that the aforementioned systems and methods can be implemented using various known or later-developed programming languages ​​(e.g., "C", "C++", "C#", "Java", "Python", etc.). Therefore, various storage media (e.g., computer disks, optical disks, electronic storage devices, etc.) capable of containing information that can instruct devices such as computers can be prepared to implement the aforementioned systems and / or methods. Once a suitable device accesses the information and programs contained on the storage medium, the storage medium can provide the information and programs to the device, thereby enabling the device to perform the functions of the systems and / or methods described herein. For example, if a computer disk containing appropriate materials (e.g., source files, object files, executable files, etc.) is provided to a computer, the computer can receive that information, appropriately configure itself, and perform the functions of the various systems and methods outlined in the diagrams and flowcharts above, thereby implementing various functions. That is, a computer can receive portions of various information relating to different units of the aforementioned systems and / or methods from the disk, implement individual systems and / or methods, and coordinate the functions of the aforementioned individual systems and / or methods.

[0102] In light of this disclosure, it should be noted that the various methods and apparatuses described herein can be implemented in hardware, software, and firmware. Furthermore, the various methods and parameters are included by way of example only and are not intended to be limiting. In view of this disclosure, those skilled in the art can implement the teachings to influence these techniques as they determine their own techniques and desired instrumentation, while remaining within the scope of the invention. The functionality of one or more processors in the processors described herein can be incorporated into a smaller number or a single processing unit (e.g., a CPU) and can be implemented using application-specific integrated circuits (ASICs) or general-purpose processing circuitry programmed to perform the functions described herein in response to executable instructions.

[0103] While this system has been specifically described with reference to ultrasound imaging systems, it is also conceivable that it can be extended to other medical imaging systems to systematically acquire one or more images. Therefore, this system can be used to acquire and / or record image information relating to, but not limited to, the kidneys, testes, breasts, ovaries, uterus, thyroid gland, liver, lungs, musculoskeletal system, spleen, heart, arteries and vascular system, as well as other imaging applications related to ultrasound-guided interventions. Additionally, this system may include one or more procedures that can be used with conventional imaging systems, such that said one or more procedures can provide the features and advantages of this system. Certain additional advantages and features of this disclosure may be apparent to those skilled in the art upon review of this disclosure, or those skilled in the art will experience certain additional advantages and features of this disclosure when employing the novel systems and methods of this disclosure. Another advantage of this system and method is the ability to easily upgrade conventional medical imaging systems to encompass the features and advantages of using this system, device, and method.

[0104] Of course, it should be understood that any of the examples, embodiments, or processes described herein may be combined with one or more other examples, embodiments, and / or processes, or may be separated / performed in a separate device or device portion according to the system, device, and method.

[0105] Finally, the foregoing discussion is intended to illustrate the system only and should not be construed as limiting the claims to any particular embodiment or group of embodiments. Therefore, while the system has been described in particular detail with reference to exemplary embodiments, it should be understood that those skilled in the art can devise many modifications and alternative embodiments without departing from the broader and contemplated spirit and scope of the invention as set forth in the claims. Consequently, the specification and drawings should be considered illustrative and not intended to limit the scope of the claims.

Claims

1. An ultrasound imaging system configured to provide quantification of marker mobility, the system comprising: A non-transient computer-readable medium that is encoded with instructions and configured to store a time series of image frames; as well as At least one processor, which communicates with the non-transient computer-readable medium and is configured to execute the instructions, wherein, when executed, the instructions cause the at least one processor to: Determine whether one or more flags exist in an individual image frame within the sequence of image frames; The output indication is determined as the bounding box of the location of one or more markers present in the individual image frame; The flow field is output at least in part based on the bounding box for the one or more markers determined to be present in the individual image frame; The displacement field values ​​are generated at least in part based on the flow field; The velocity vector, indicating the direction and magnitude of motion, is generated at least in part based on the flow field. The isotropic value of displacement is calculated by dividing the magnitude of the spatially averaged displacement field by the magnitude of the displacement field, or the isotropic value of velocity is calculated by dividing the magnitude of the spatially averaged displacement field by the velocity vector; and At least one of a mobility map or a mobility index is generated, based at least in part on the isotropic displacement value or the isotropic velocity value.

2. The system according to claim 1, wherein, The mobility index is based on a percentage of values ​​equal to or below a threshold.

3. The system according to claim 1, wherein, The isotropic displacement value or the isotropic velocity value indicates the clinical condition or the severity of the clinical condition.

4. The system according to claim 3, wherein, The clinical conditions mentioned include endometriosis.

5. The system according to claim 1, wherein, The instruction also causes the at least one processor to: The mobility map is segmented at least in part based on the displacement isotropic value or the velocity isotropic value; and A mobility mask comprising one or more regions is generated, at least in part based on the segmentation of the mobility map.

6. The system according to claim 5, wherein, The one or more regions have at least one of different values ​​or ranges of the isotropic displacement value or the isotropic velocity value.

7. The system according to claim 5, wherein, The instructions also cause the at least one processor to generate display data for overlaying the mobility mask onto image frames in the sequence of image frames.

8. The system according to claim 1, wherein, The instructions also cause the at least one processor to cluster the one or more flags present in the sequence of image frames based on at least one of the relative or absolute movements of the one or more flags.

9. The system according to claim 1, further comprising: An ultrasonic probe, including an inertial measurement unit configured to measure the movement of the ultrasonic probe and output probe movement data. The instructions also cause the at least one processor to exclude individual image frames from the sequence of image frames from analysis based at least in part on the probe movement data.

10. The system according to claim 9, wherein, Based on the probe movement data, the excluded individual image frames are associated with at least one of the sudden movement of the ultrasound probe or the rotational movement of the ultrasound probe.

11. The system according to claim 8, further comprising a user interface, in, The instructions also cause the at least one processor to provide feedback via the user interface on the appropriate probe movements.

12. The system according to claim 1, wherein, The instructions also cause the at least one processor to generate at least one of a displacement map based on the value of the displacement field or a velocity map based on the value of the velocity vector.

13. A method for providing quantification of flag mobility, the method comprising: A sequence of received image frames; Determine whether one or more flags exist in an individual image frame within the sequence of image frames; Provides a bounding box indicating the location of one or more markers determined to exist in the individual image frame; A flow field is generated, at least in part, based on the entire frame of the sequence of the bounding boxes or image frames, targeting the one or more markers identified as existing in the individual image frames; The displacement field values ​​are generated at least in part based on the flow field; The velocity vector, indicating the direction and magnitude of motion, is generated at least in part based on the flow field. The displacement isotropic value is calculated by dividing the value of the spatial average displacement field by the value of the displacement field, or the velocity isotropic value is calculated by dividing the value of the spatial average displacement field by the velocity vector. as well as At least one of a mobility map or a mobility index is generated, based at least in part on the isotropic displacement value or the isotropic velocity value.

14. The method according to claim 13, wherein, Determining the existence of the one or more flags and providing the bounding box is performed by a first artificial intelligence (AI) model, and wherein generating the flow field is performed by a second artificial intelligence (AI) model.

15. A computer program product comprising a computer-readable medium having computer-readable code contained therein, characterized in that, The computer-readable code is configured to cause the computer or processor, when executed by a suitable computer or processor, to perform the method according to claim 13.

Images