Method for Reversing the Directions of Sensors and Emitters in Multi-Sensor Imaging

Abstract

Techniques for orientation determination cause an emitter of a multi-sensor imaging device to emit a first set of ultrasonic signals, measure the arrival times and amplitudes of the first set of ultrasonic signals, estimate an initial position of the sensors, orient each of the sensors towards a center of mass position, use a loss function to fit an initial orientation to each of the sensors and each of the emitters, determine a coarse model based on the initial positions of the sensors and the emitters, calculate new positions and new orientations for each of the sensors and each of the emitters based on the coarse model and a second set of ultrasonic signals, employ full-wave inversion to generate an updated model, and determine the orientation of each of the sensors and emitters based on the updated model. Embodiments can include filtering outlier sensor-emitter pairs.

Description

Technical Field

[0001] <Cross - Reference to Related Applications> This application claims the benefit of U.S. Provisional Application No. 63 / 367,161, filed Jun. 28, 2022, the content of which is incorporated herein by reference.

[0002] <Technical Field> The present disclosure generally relates to ultrasonic imaging systems, and more particularly to determining the position and orientation of ultrasonic emitters and sensors of an ultrasonic imaging system.

Background Art

[0003] <Background> Ultrasound is commonly used in a variety of applications, including those for non - invasively and low - risk scanning of body parts. In a typical application, an array of ultrasonic sensors is mounted on a flat surface. The body part to be scanned is coated with a gel to ensure better alignment between the surface of the sensor and the skin of the body part. The array sweeps the outer surface of the volume of interest and emits a tight beam that is used to generate an image of the internal volume from the returned signals. One or more emitters emit ultrasound at a desired frequency, and the reflected or refracted sound waves are captured by the sensor array. The signals are then interpreted to provide images of the internal organs and bones of the body part. In a typical setting, the patient is positioned at a desired location to enable the use of the array of ultrasonic sensors by a physician. Over time, ultrasonic imaging capabilities have improved, from images that could only be interpreted by a specialized interpreter to being able to provide clearly valuable three - dimensional imaging today, for example, when scanning a fetus in the uterus with 3D detail.

[0004] The drawback of current ultrasonic devices that use an array of sensors / emitters is that the array typically has a small surface area, and thus, when it is necessary to check a larger area of the body, the ultrasonic measurement device must be manually or automatically moved along the patient's skin in order to achieve the required coverage. Further, since the device relies on beamforming, appropriate contact between the ultrasonic array of sensors and the patient's skin must be maintained throughout the process for effective and clear imaging results. That is, ultrasonic imaging used in medical imaging is performed using a single emitter / sensor array mounted on a flat surface that is much larger than the wavelength. The array sweeps the outer surface of the volume of interest and emits a tight beam that is used to generate an image of the internal volume from the returned signals.

Summary of the Invention

Problems to be Solved by the Invention

[0005] Alternative scanning setups can utilize a set of individual emitters and sensors that spread around the volume of interest at a fixed position. The ultrasonic waves emitted by a given emitter propagate through a medium (e.g., the patient's body) and are recorded by all surrounding sensors. Each sensor records data from multiple radiation directions and captures not only the returned signal but also the wave that was initially emitted. A similar emitter / sensor setup is mainly used in geoscience applications for exploring the Earth's subsurface structure and properties. Generally, an inversion method is applied to infer the internal properties of the Earth that best fit the recorded data. In such applications, the positions of the emitters and sensors on the capture surface are known a priori with high certainty. For this purpose, the use of such a similar emitter / sensor setup is limited for industrial and medical applications where direct and accurate measurement of the position in real time is not easily obtained.

[0006] In addition, in many medical applications, the sensor size is larger than the ultrasonic half-wavelength, and thus it may be necessary to determine the orientation of the sensors along with their positions in the space. Current methods can simultaneously locate the positions of the emitters and sensors while optimizing the internal property model of the medium. However, a method for determining the orientation of the sensors on the acquisition surface while simultaneously optimizing the internal property model of the medium has not yet been developed or implemented.

[0007] Therefore, it is advantageous to provide a solution that overcomes the above-described problems.

Means for Solving the Problems

[0008] <Abstract> An overview of some exemplary embodiments of the present disclosure is presented below. This overview is provided for the convenience of the reader to provide a basic understanding of such embodiments and does not fully define the scope of the present disclosure. This overview is not an exhaustive overview of all contemplated embodiments, nor does it identify the main or important elements of all embodiments, nor does it define the scope of any or all aspects. Its sole purpose is to present, in a simplified form, some concepts of one or more embodiments as a prelude to the more detailed description presented later. For convenience, the terms "some embodiments" or "certain embodiments" may be used herein to refer to a single embodiment or multiple embodiments of the present disclosure.

[0009] Certain embodiments disclosed herein include a method for orientation determination. The method includes causing an emitter of a multi-sensor imaging apparatus to emit a first set of ultrasonic signals, wherein the multi-sensor imaging apparatus includes the emitter and sensors; measuring arrival times and amplitudes of the first set of ultrasonic signals; estimating an initial position of the sensors based on the measured arrival times and amplitudes; orienting each of the sensors toward a center mass location; fitting an initial orientation to each of the sensors and each of the emitters using a loss function; determining a coarse model based on the initial positions of the sensors and the emitter; calculating new positions and new orientations for each of the sensors and each of the emitters based on the coarse model and a second set of ultrasonic signals; using full wave inversion (FWI) to generate an updated model based on the new positions and new orientations of the sensors and the emitter; and determining the orientations of each of the sensors and each of the emitters based on the updated model.

[0010] Certain embodiments disclosed herein also include a non-transitory computer-readable medium having instructions stored thereon for causing a processing circuit to execute a process, the process including: causing an emitter of a multi-sensor imaging device to emit a first set of ultrasonic signals; measuring arrival times and amplitudes of the first set of ultrasonic signals; estimating an initial position of a sensor based on the measured arrival times and amplitudes; orienting each of the sensors towards a center of mass position; fitting an initial orientation to each of the sensors and each of the emitters using a loss function; determining a coarse model based on the initial position of the sensor and the initial position of the emitter; calculating new positions and new orientations for each of the sensors and each of the emitters based on the coarse model and a second set of ultrasonic signals; using full-wave inversion (FWI) to generate an updated model based on the new positions and new orientations of the sensors and the emitters; and determining an orientation for each of the sensors and each of the emitters based on the updated model.

[0011] Certain embodiments disclosed herein also include a system for orientation determination. The system includes a processing circuit, a plurality of emitters communicatively coupled to the processing circuit, a plurality of sensors communicatively coupled to the processing circuit, and a memory that, when executed by the processing circuit, causes the system to cause the emitters of the multi-sensor imaging device to emit a first set of ultrasonic signals, wherein the multi-sensor imaging device includes the emitters and sensors; measure the arrival time and amplitude of the first set of ultrasonic signals; estimate an initial position of the sensors based on the measured arrival time and amplitude; direct each of the sensors to a centroid position; fit an initial orientation to each of the sensors and each of the emitters using a loss function; determine a coarse model based on the initial positions of the sensors and the initial positions of the emitters; calculate new positions and new orientations for each of the sensors and each of the emitters based on the coarse model and a second set of ultrasonic signals; generate an updated model based on the new positions and new orientations of the sensors and the emitters using full-wave inversion (FWI); and determine the respective orientations of the sensors and the emitters based on the updated model.

[0012] Certain embodiments disclosed herein include the method, non-transitory computer-readable medium, or system described above or below, wherein at least one outlier sensor-emitter pair is filtered based on signal strength, and each outlier sensor-emitter pair has a signal strength below a threshold value.

[0013] Certain embodiments disclosed herein include the method, non-transitory computer-readable medium, or system described above or below, wherein at least one outlier sensor-emitter pair is filtered based on characteristics of signals passing through non-soft tissue that are greater than a predetermined threshold size.

[0014] Certain embodiments disclosed herein include, or are configured to perform, the above or below method, non-transitory computer-readable medium, or system, further comprising the step of performing gradient descent.

[0015] Certain embodiments disclosed herein include the above or below method, non-transitory computer-readable medium, or system, and the gradient descent is any one of Stochastic Gradient Descent (SGD), Broyden-Fletcher-Goldfarb-Shanoo method, Memory Limited Broyden-Fletcher-Goldfarb-Shanoo method, Adaptive Moment Estimation (ADAM), ADAM-W, Multistage Stochastic Variational Approximation Gradient (M-SVAG), and ADAbelief.

[0016] Certain embodiments disclosed herein include, or are configured to perform, the above or below method, non-transitory computer-readable medium, or system, further comprising the step of filtering out at least one outlier sensor emitter pair based on the estimated initial position of the sensor.

[0017] Certain embodiments disclosed herein further include, or are configured to perform, the steps of sorting the sensors based on the loss values determined for each sensor using a loss function, selecting at least one sensor among the sensors whose determined loss values exceed a predetermined threshold, changing the direction of each of the selected at least one sensor to the direction of maximum intensity, repeating the process consisting of the steps of sorting the sensors, selecting at least one sensor, and changing the direction of each selected sensor until the loss value of each sensor is below a predetermined threshold, in the above or below method, non-transitory computer-readable medium, or system.

[0018] Certain embodiments disclosed herein include or are configured to perform a method, non - transitory computer - readable medium, or system as described above or below, further including or configured to perform a step of using inversion tomography.

[0019] Certain embodiments disclosed herein include or are configured to perform a method, non - transitory computer - readable medium, or system as described above or below, in which FWI is used and the orientation is iteratively determined until the model generated based on the orientation converges.

[0020] Certain embodiments disclosed herein include or are configured to perform a method, non - transitory computer - readable medium, or system as described above or below, further including or configured to perform a step of determining the loss value of each sensor based on a full - amplitude comparison over all frequencies between the observed measurements of the sensor and the calculated signal.

[0021] Certain embodiments disclosed herein include or are configured to perform a method, non - transitory computer - readable medium, or system as described above or below, further including or configured to perform a step of determining the loss value of each sensor based on the ratio of amplitudes at different frequencies. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The subject matter disclosed herein is particularly pointed out and distinctly claimed in the claims filed herewith. The foregoing and other objects, features, and advantages of the disclosed embodiments will become apparent from the following detailed description when taken in conjunction with the accompanying drawings.

[0023]

Figure 1

[0024]

Figure 2

[0025]

Figure 3

[0026]

Figure 4

[0027]

Figure 5A

[0028]

Figure 5B

[0029]

Figure 6

[0030] <DETAILED DESCRIPTION> It is important to note that the embodiments disclosed herein are merely examples of many advantageous uses of the innovative teachings herein. In general, the descriptions made in the specification of this application do not necessarily limit any of the various claimed embodiments. Further, some descriptions may apply to some features of some inventions but not to others. In general, unless otherwise specified, a single element may be plural without loss of generality, and vice versa. In the drawings, like numbers refer to like parts throughout the several figures.

[0031] The various disclosed embodiments provide systems and methods for improved determination of the orientation of sensors and emitters in a multi-sensor imaging device by performing stages of preparation, initial orientation, and improvement. In one embodiment, the preparation stage includes steps of measuring arrival time and amplitude of signals, estimating the position of sensors, filtering outlier sensor-emitter pairs, and as an initial state, orienting each sensor towards the center of mass position of the sensor at the height level of the sensor.

[0032] In one embodiment, the initial orientation stage includes steps of using gradient-based optimization to fit to an initial orientation based on a loss function and optionally adding global maximization, where global maximization includes steps of sorting sensors based on loss values, selecting the sensor with the highest loss and changing the direction of maximum intensity, measuring the loss, and if the loss is improved, returning to the initial orientation stage, and if not, recalculating the positions and orientations of the sensors and emitters and repeating the improvement of the model based on the new positions and orientations to continue the improvement stage.

[0033] The disclosed embodiments include, but are not limited to, systems and methods that provide or utilize techniques for solving the problem of inversion of orientation of ultrasonic sensors and ultrasonic emitters in a multi-sensor imaging device. This enables ensuring an accurate ultrasonic image representation when the orientations of both the emitter and the sensor change relative to each other, such as when such emitters and sensors are attached to an ultrasonic sensing garment.

[0034] Some solutions can hypothetically utilize a human operator to mitigate the inversion of the orientations of the ultrasonic sensors and ultrasonic emitters of a multi-sensor imaging device, but such solutions actually fail. A human operator applies subjective criteria for analyzing, deciding, and classifying, and there is no consistency among different human operators, and even for the same human operator performing the same task repeatedly, there are often inconsistent results, especially at the speed required to provide an operable solution. Further, the number of possible orientations in the space of multiple emitters and multiple sensors far exceeds any practical use of the human mind.

[0035] Various disclosed embodiments utilize a set of predetermined objective criteria, such as steps of measuring the arrival time and amplitude of ultrasonic signals, estimating the positions of the sensors based on the measured arrival times and amplitudes, filtering outlier sensor-emitter pairs, directing each of the sensors to a centroid position, fitting an initial orientation to each of the sensors and emitters using a loss function, determining a rough model based on the initial positions of the sensors and emitters, recalculating the new positions and new orientations of each of the sensors and emitters, using full-wave inversion (FWI) to generate an improved model based on the new positions and new orientations of the sensors and emitters, and improving the orientations of the sensors and emitters based on the improved model. The use of such objective criteria results in solving the problem of inversion of the orientations of the ultrasonic sensors and ultrasonic emitters of a multi-sensor imaging device in a reliable and consistent manner based on objective criteria.

[0036] FIG. 1 shows an exemplary schematic diagram 100 of an ultrasonic sensing garment (or ultrasonic sensing garment / USG) 110 in which a sensor 120, an emitter 130, and a marker 140 are embedded and arranged according to an exemplary embodiment. Such a USG is described, for example, but not limited to, in PCT patent application PCT / IB2021 / 061474, entitled "Wearable Garments Adapted for Ultrasonic Detection and Methods Therefor for Full-Wave Inversion with Incorrect Sensor Positions," assigned to a common assignee, the content of which is incorporated herein by reference.

[0037] The embedding of the sensor 120, the emitter 130, and the marker 140 can be achieved by different techniques such as, but not limited to, weaving, adhesion, mechanical attachment, etc., and any combination thereof. The USG 110 is designed to provide an imaging solution that transmits and receives ultrasonic signals to be processed to generate a high-resolution three-dimensional (3D) image of the scanned body part. In one embodiment, a power supply (not shown) may be provided to elements (such as elements of the sensor 120, the emitter 130, the marker 140, etc.) embedded in the USG 110, and this element may be provided by a mesh of conductive wires (not shown) that are part of the USG 110.

[0038] It should be further understood that the USG 110 may be shaped in various ways such that it is worn on a body part or multiple body parts, or wrapped around it in other ways. In one embodiment, the USG 110 is designed to be placed on a body part rather than worn on or wrapped around a body part. In another embodiment, one or more of the elements embedded in the USG 110 (i.e., the sensor 120, the emitter 130, the marker 140, or a combination thereof) may be coated with a soft polymeric material that allows for sufficient contact with substantially no voids between the element and the body part adjacent to the element. In yet another embodiment, the clothing material is elastic and adapted to closely conform to the contour of the body part.

[0039] The USG110 is further designed to comfortably conform around body parts with the necessary flexibility that is not provided by at least some existing solutions. Such flexibility is achieved, at least in part, by typically avoiding the need for a largely bulging sensor array. Such a sensor array can give the wearable device a rigid feeling.

[0040] To facilitate providing the flexible garment 110, in one embodiment, small ultrasonic sensors 120, e.g., sensors 120-1 to 120-i, where "i" is an integer greater than "1" (the ultrasonic sensors 120 are also simply referred to herein as sensors (plural) 120 or sensor 120 for simplicity) are configured on the USG110. The sensors 120 may be randomly or regularly embedded within the USG110. In a further embodiment, unlike implementations that utilize an array of sensors, the sensors 120 are each small in size relative to the garment 110 such that the garment 110 retains its flexibility in such an embodiment.

[0041] In one embodiment, such sensors 120 do not abut against each other and each sensor 120 can be positioned on the USG110 as needed or desired for a given use case, maintaining at least a predetermined distance from each other. The plurality of sensors 120 on the USG110 may be referred to as a loose array since each sensor 120 is separated from any adjacent sensor 120. The sensors 120 may be, for example, piezoelectric sensors, capacitive microelectromechanical systems (MEMS)-based sensors, capacitive polymer-based sensors, combinations thereof, etc., but are not limited thereto. In addition to the sensors 120, the USG110 also includes ultrasonic emitters 130, also referred to herein as emitters 130, e.g., emitters 130-1 to 130-j, where "j" is an integer greater than "1".

[0042] The USG110 further includes markers 140, for example, markers 140-1 to 140-k, where "k" is an integer greater than or equal to "1". The markers 140 can be used to obtain an initial approximate position of the emitter 130 and the sensor 120. The initial approximation of the positions of the emitter and the sensor will be discussed in more detail herein. The power supply to the USG110 can be provided using various sources such as, but not limited to, a battery, a generator, an electrical outlet, etc.

[0043] In one embodiment, the USG110 may be further configured using fasteners 150, for example, fasteners 150-1 and 150-2, adapted to fix the USG110 around a body part. The fasteners 150 can include, but are not limited to, hooks, Velcro®, buttons, and corresponding button loops or holes, combinations thereof, parts thereof, etc.

[0044] In one embodiment, the USG110 can include an electronic circuit 160 adapted to provide power for consumption by elements embedded in the USG110 (for example, the sensor 120, the emitter 130, the marker 140, or a combination thereof). The electronic circuit 160 can include a combination of digital, analog, and optical components, although not limited thereto, to be adapted to enable proper operation of the USG110. The signal received from the sensor 120 can be further processed by the electronic circuit 160 and processed locally or at a processing device (not shown) to display an image corresponding to the processed signal on a display device (not shown), as will be described in more detail below with respect to FIG. 4. In an exemplary embodiment, the signal is transmitted to the processing device by a wired or wireless connection, for example, after initial or minimal processing, although not limited thereto. The processing of the signal will be further described with respect to FIGS. 2 and 3 of this specification.

[0045] In one embodiment, at least a portion of a process executed via the electronic circuit 160, or via the computing components of the USG 110, or via communicating with the USG 110, includes a. a step of inverting the elastic properties of the medium (e.g., full-wave inversion), and b) a step of obtaining the positions of one or more elements (e.g., one or more of the sensors 120, one or more of the emitters 130, or both).

[0046] For this purpose, in one embodiment, such computing components can be configured to calculate the causal factors (or, causal factor) that generate them from a set of observed signals. This is the reverse of the forward problem that starts with the cause and calculates the result. Inversion (or, inverse / reversal / inversion) is performed to attempt to find the best model that fits the acquired data. As used herein, the term "full waveform inversion (or, full waveform inversion)" (FWI) refers to a method of simulating waves from an emitter (e.g., emitter 130) to a sensor (e.g., sensor 120) and comparing the measured signals when a number of initial models are given. It should be further understood that reference to FWI is interchangeable with any of its variants, including but not limited to adaptive waveform inversion (wavefield reconstruction inversion / AWI), wavefield reconstruction inversion (wavefield reconstruction inversion / WRI), and other similar algorithms.

[0047] The error between the simulated wave and the measured signal is backpropagated to obtain a gradient for each point in the model, and the model is changed (in some implementations, slightly, i.e., by a threshold amount or less than a ratio) in the gradient direction, thereby resulting in a new model. Then, the new model is used instead of the previous model to simulate the wave forward and backward and recalculate the gradient. In one embodiment, the iteration of changing the model based on simulation, measurement, and comparison continues until the model converges. In various implementations, the model is considered to be close to the "true" model or, otherwise, close enough for all practical problems once it has converged.

[0048] In some implementations, starting by assuming a very simple model, the position of the sensor is identified along with the model using a process that compares the simulated time travel of the wave through the medium with the actual travel time. The difference in travel time is used to correct the position of the emitter, e.g., emitter 130. The differences in travel time in various soft tissues are such that even with the high uncertainty in the human body model, a sufficiently good initial position error is achieved. Using the initial position, it is possible to build a better model and thus improve position identification. This alternating process continues until both the expected position of the model (e.g., the position of the sensor shown in the model) and the actual position of the sensor (e.g., the actual position of the sensor determined based on current measurements) have converged sufficiently.

[0049] Note that some initial care is required to handle non-soft tissues (e.g., bone, air, etc.) within the model. While some differences in the attenuation of close frequency signals cause attenuation that is large enough to be substantially meaningless, in at least some implementations, some differences in the attenuation of close frequency signals may need to be carefully considered in the model.

[0050] FIG. 2 is an exemplary flowchart 200 showing a method of model optimization according to one embodiment. The methods described herein can utilize a loose array of ultrasonic emitters (e.g., emitter 130 of FIG. 1) and sensors (e.g., sensor 120 of FIG. 1) that spread around the body with unknown or otherwise potentially inaccurately known geometric shapes. FIG. 2 is described with respect to sensor 120 and emitter 130 of USG 110 of FIG. 1, but note that at least some of the disclosed embodiments utilizing the process of FIG. 2 are not limited to the specific configuration of USG 110 shown in FIG. 1.

[0051] The process begins with a data acquisition stage at S210, where each emitter of emitter 130 emits a known radiation pattern. The emission can be performed, for example but not limited to, continuously (i.e., each emitter in turn) or in parallel (i.e., simultaneously) using encoding (or encoding). During the emission phase, signal arrival data is recorded by each sensor of sensor 120 for all sensors of USG 110 of FIG. 1. After acquiring the raw signal, signal processing can be used to remove noise and artifacts.

[0052] Based on the data acquired at S210, the positions and orientations of elements such as emitters and sensors can be determined iteratively. The position of an element such as an emitter or a sensor refers to its spatial position relative to a medium (e.g., a body part), and the orientation of such an element refers to the angular direction that the element points to (or points to) at that position. For example, a first emitter may be placed at position A above the belly button in an orientation that points to the emitter along the surface of the skin. In the same example, the first emitter can be placed at position A in an orientation that points to the emitter perpendicular to the skin surface and emits to the underlying tissue.

[0053] In one embodiment, each of the coarse location and the coarse orientation can be determined based on the data obtained in S210 and further based on an initial model. The initial model can be based on a predetermined configuration of the model, for example, based on the known location and orientation of the model at a default or starting position.

[0054] In S220, the coarse locations of the emitter 130 and the sensor 120 are determined using the coarse model generated in S220. In one embodiment, each coarse location is determined based on the delay between the time when the emitter 130 transmits a signal and the time when the signal is received by the sensor 120. Each such delay forms a constraint on the distance between the emitter and the sensor, more precisely a distance range. All the delays form a network of constraints that can be solved to find the corresponding locations.

[0055] In S230, the coarse orientations of the emitter 130 and the sensor 120 are determined using the coarse model generated in S220. In one embodiment, the emission patterns of the sensor 120 and the emitter 130 are known in advance to determine the orientations of the emitter 130 and the sensor 120, and it is assumed that the sensor 120 is spread out so that the actual intensities for a plurality of angles can be measured. These measurements can be "fitted" to the known emission pattern to determine the orientation. In practice, the sensor 120 itself also has an orientation and sensitivity pattern for which details of handling are provided herein. In some embodiments, the coarse orientation can be determined as further described below with respect to FIG. 3.

[0056] In S240, a rough model is generated based on the rough position and orientation. For this purpose, when the rough position determined in S220 and the rough orientation determined in S230 are provided, the rough model can be calculated using techniques such as, but not limited to, using the full-wave inversion technique. Next, using this rough model, for each of the emitter 130 and the sensor 120, a new improved position can be determined in S250, and a new improved orientation can be determined in S260. In some embodiments, the improved orientation can be determined as further described below with respect to FIG. 3. Thereafter, in S270, an improved model (or updated model) is generated, for example, by utilizing the FWI technique, but not limited thereto.

[0057] In S280, it is checked whether additional iterations should be performed. If so, the execution continues with S250, and if not, the execution ends. In one embodiment, the improved model (S270) is applied to correct the orientations of the sensor and the emitter for accurate ultrasonic imaging using the multi-sensor imaging device disclosed herein. Therefore, the implementation is provided in two stages, namely, a rough stage (S220, S230, and S240) and an iterative improvement stage (S250, S260, and S270). During the iterative improvement stage, as the model approximation and position are improved, the orientation is also recalculated. Note that the two-stage implementation enables the optimization of the orientations of the sensor and the emitter, and at the same time, optimizes the model for accurate and improved fitting of the ultrasonic signal.

[0058] FIG. 3 is an exemplary flowchart 300 showing a method for determining the orientations of a sensor and an emitter according to one embodiment. In some embodiments, the method of FIG. 3 can be utilized between step S230, step S260, or both of FIG. 2.

[0059] FIG. 3 is described with respect to the sensor 120 and the emitter 130 of the USG 110 of FIG. 1, but it should be noted that at least some of the disclosed embodiments utilizing the process of FIG. 3 are not limited to the specific configuration of the USG 110 shown in FIG. 1.

[0060] In S310, at the sensor 120, a set of signals is acquired and pre - processed from the emissions of the emitter 130. The received signals are pre - processed to remove at least noise and certain artifacts.

[0061] In S320, the lead (first) arrival time and the signal strength are determined for each sensor - emitter pair, for example, sensor 120 - 1 and emitter 130 - 1 (also referred to herein as an s - e pair). The lead arrival signal is determined using cross - correlation. Based on the predetermined positions of the emitter 130 and the sensor 120 (for example, in the current orientation determination iteration of the current model of the USG or other garment, including the sensors and emitter 130 used, such as an initial model or default model, the rough model determined at S240, or the improved model generated at S270), the strength of the received leading (or antecedent) signal recorded by each sensor 120 from each emitter 130 can act as a set of constraints on the orientation of the sensors 120 and the emitter 130, also referred to herein as a transfer matrix.

[0062] In optional S330, one or more irrelevant s - e pairs can be filtered out. Irrelevant s - e pairs can be identified by their respective lead arrival times and intensities that are outside a predetermined range with respect to other s - e pair measurements, predetermined values, or both. In one embodiment, the irrelevant s - e pairs can be, for example, but not limited to, s - e pairs associated with sufficiently large (e.g., exceeding a threshold) attenuation and signal degradation that can be caused by non - soft tissues such as bone, air, etc., and any combination thereof.

[0063] At S340, the orientation loss is determined. In one embodiment, a set of orientations is determined through optimization, and an orientation law is determined using the set of orientations. In a further embodiment, a given radiation pattern I(θ) of the sensor 120 and the emitter 130 is defined as an angle from the sensor axis, defined as an orientation vector, and a set of positions and orientations {p i}, {d i}, when given as a function of, the orientation loss is calculated based on the difference between the actually observed intensity and the simulated intensity as defined in Equation 1: TIFF2025524504000001.tif30170

[0064] In Equation 1, A(x,y) is the attenuation between x and y along a ray trace, and dist(x,y) is the distance along a ray path. In some embodiments, the attenuation function and the distance function can be replaced with a function of the Euclidean distance when the model resolution is low enough. Then, in the improvement stage (e.g., S250 - S270 described with respect to Figure 2), the ray path is used to determine the attenuation.

[0065] At S350, it is determined whether the orientation loss is below a predetermined threshold. If it is below, the execution ends; if not, the execution continues to S360. Note that a sufficiently small (e.g., below the threshold) orientation loss indicates a sufficient match between the simulated signal and the actually observed signal, and thus the orientations of the sensor and the emitter are optimized.

[0066] In S360, an orientation gradient is determined. The orientation gradient is the direction of the maximum change in the intensity of the signal received in S310. In some implementations, the orientation gradient is determined to update the orientation to minimize the loss (e.g., the loss calculated using the loss function described above with respect to S340). If the initial orientation is close enough to the global minimum of the orientation gradient, this can be solved by a gradient-based optimization method that includes: a) performing an iterative process of calculating a loss value from the difference between the calculated (or simulated) signal intensity and the observed signal intensity; and b) reorienting (or repositioning) the orientation of the emitter 130 and the sensor 120 according to the gradient of the loss with respect to the change in the orientation of each emitter and sensor.

[0067] In S370, outliers are removed. Since the model is only roughly known at a coarse stage (see S240), as shown by the orientation gradient calculated in S360, the rate race or signal attenuation of a particular s-e pair may be different (e.g., different by exceeding a predetermined threshold) from what is predicted based on the model. Such an s-e pair can be identified as an outlier and removed. As a non-limiting example, if a given s-e pair with a first emitter (e.g., emitter 130-1) and a first sensor (e.g., sensor 120-1) provides information that is sufficiently different (e.g., exceeding a predetermined threshold) from other s-e pairs of a loose array of ultrasonic emitters and sensors, the given s-e pair is discarded.

[0068] In one embodiment, outlier removal can be weighted in each iteration. The weighting is used to prefer a subset of s-e pairs that are expected to provide better information than other s-e pairs. The weights may be based on prior knowledge (e.g., but not limited to, s-e pairs above and below the pelvis bone) and signal quality (e.g., but not limited to, when the lead signal structure is cleaner and stronger, or both).

[0069] In S380, the orientation information of sensor 120 and emitter 130 is updated, and execution continues at S340, where a new orientation loss is calculated based on the updated orientation information.

[0070] FIG. 4 shows an exemplary electronic circuit 160 adapted to perform model inversion of the positions and orientations of sensors and emitters in a multi-sensor imaging device according to one embodiment. For simplicity, and not to limit the disclosed embodiments, FIG. 4 is described with reference to the elements shown in FIG. 1.

[0071] The processing circuit 410 is communicatively connected to the memory 420. At least a portion of the memory 420 contains instructions that, when executed by the processing circuit 410, enable the USG 110 to perform the functions described herein, particularly the functions described in FIGS. 2 and 3, and their respective descriptions. A sensor control interface (SCI) 430 communicatively connected to the PE 410 is adapted to receive at least the signals sensed by the sensor 120. The SCI 430 can receive signals in parallel from all, some, or just one of the sensors 120. An emitter control interface (ECI) 440 communicatively connected to the PE 410 is adapted to transmit at least control signals to the emitter 130 to activate the emitter 130. The ECI 440 can transmit signals in parallel to all, some, or just one of the emitters 130.

[0072] In one embodiment, an optional marker control interface (MCI) 450 communicatively connected to the PE410 may be used for the active marker 140 and is adapted to at least operate (or activate) the active marker 140. The MCI 450 can send control signals in parallel to all, some, or just one of the active markers 140. A power control unit (PCU) 460 connected to the PE410 is configured to provide the operating power required for any element of the USG110 (e.g., the sensor 120, the emitter 130, the marker 140, or a combination thereof), which can be performed in parallel, in part, or for just one element of the USG110.

[0073] Also, a communication interface unit (CIU) 470 is communicatively connected to the PE410. The CIU 470 is configured to provide communication with the USG110. For example, without limitation, the CIU 470 can communicate with a) means for operating the USG110, b) receive signals from an external device (not shown) that controls the USG110, and c) transmit processed or raw signals captured by the sensor 120, according to any of the embodiments described herein. In one embodiment of the electronic circuit 160, the PE410 and the memory 420 are replaced, for example, without limitation, by combinational logic circuitry adapted to perform the tasks described herein. Such embodiments and similar embodiments should be considered within the scope of the disclosed embodiments.

[0074] FIG. 5A is an exemplary pattern 500A provided in polar coordinates of an emitter radiation pattern and a sensor reception pattern according to one embodiment. FIG. 5B is an exemplary pattern 500B provided in Cartesian coordinates of an emitter radiation pattern and a sensor reception pattern according to one embodiment. Both FIG. 5A and FIG. 5B show a main lobe 510, a side lobe 520, and a back lobe 530, respectively labeled "A" and "B" for each figure (e.g., 510A for FIG. 5A and 510B for FIG. 5B).

[0075] FIG. 6 is an exemplary diagram 600 of a geometry emission function (or geometry radiation function) and orientation according to one embodiment. Sensor 120 and emitter 130 are disposed around body part 610. In the non-limiting exemplary implementation shown in FIG. 6, the sensor includes three sensors 120-1 to 120-3.

[0076] The reception for each sensor 120 is described along a main reception path 640. For example, reception path 640-1 is between emitter 130 and sensor 120-1. Each emitter has its respective emission lobe with a main lobe and side lobes (not separately labeled), e.g., emission lobe 630-1 of emitter 130-1. Each sensor has its respective reception lobe with its corresponding main lobe and side lobes, e.g., reception lobe 620-2 of sensor 120-2. This, of course, is also the case for sensor 120-1 with reception lobe 620-1 and sensor 120-3 with reception lobe 620-3.

[0077] Note that the receiving path 640 is defined based on at least the orientation, internal characteristics of the medium (e.g., body part 610), combinations thereof, etc. The following emission function and orientation are provided for a better understanding of the positioning of the sensor 120 and the emitter 130 with respect to the description provided herein. FIG. 6 shows one emitter 130 for purposes of simplification and illustration, and note that without departing from the scope of the disclosed embodiments, two or more emitter mats may be arranged around the body part 610.

[0078] In an exemplary embodiment, the position of the sensor 120 is known, for example, based on the time of arrival of the signal. To estimate the orientation of the sensor 120, calculations based on the measured signal strength are used. For each pair of emitter-sensor (s-e pair), the intensity is defined by Equation 2 as follows: TIFF2025524504000002.tif18169

[0079] In Equation 2, i, j are emitter-sensor indices, and A ij is the amplitude at position k, and N is the signal length. The calculated amplitude of the signal for each emitter-sensor pair depends on the orientation of the emitter-sensor pair and is defined by Equation 3 as follows: TIFF2025524504000003.tif15169

[0080] In Equation 3, P(t) is the pressure at time t, and K e (r,θ) is the angular kernel function of the emitter 130, r is the emitter-sensor distance, θ is the emitter-sensor spatial angle, and K s is the angular kernel of the sensor 120, φ is the orientation of the sensor 120 relative to the emitter 130, * is the convolution operator. A is a general operator that defines other kernels, such as attenuation on the medium, the frequency response of the emitter 130, and the frequency response of the sensor 120.

[0081] In one embodiment, the calculation of the intensity for a plurality of frequencies uses two or more pulses, each pulse having a different center frequency. In one embodiment (e.g., when the frequencies are not too far apart and the optical paths are similar [e.g., within each other's threshold distances]), the measured value of the intensity in the transfer matrix can be replaced by the ratio of intensities, i.e., the ratio of the amplitudes at different frequencies. The ratio of intensities as a function of the angle is not very dependent on the path itself, and specific artifacts and obstacles along the path are easy to remove, so an improved loss function can be provided.

[0082] Some non-limiting exemplary loss functions based on a set of intensity-orientation constraints are defined in Equations 4, 5, and 6 herein. Loss function Equation 4 is as follows: TIFF2025524504000004.tif19170

[0083] In Equation (4), I ij and S ij are the measured intensity and the simulated intensity of the i,j emitter-sensor pair, respectively. For the multi-frequency embodiment, the associated loss function described in Equation 5 is as follows: TIFF2025524504000005.tif23170

[0084] In Equation 5, f1 and f2 are the different center frequencies of two different emission signals. Finally, loss function Equation 6 is as follows: TIFF2025524504000006.tif20169

[0085] In yet another embodiment, an additional term is added to the loss function (Equation 5). This is used to avoid the high curvature of the surface formed by connecting all the surfaces that are assumed to be on the surface of the organ.

[0086] According to one embodiment, the method of model optimization includes a preparation stage, an initial orientation stage, and an improvement stage. In the preparation stage, the following are performed: the step of measuring the arrival time and signal amplitude; the step of estimating the position of sensor 120; the step of filtering outlier sensor-emitter pairs based on low signal intensity (e.g., below a threshold value) or the characteristic of passing through excessive non-soft tissue, i.e., a characteristic exceeding a predetermined threshold value of non-soft tissue; and, as an initial state, the step of orienting each sensor 120 towards the center of mass position of sensor 120 at the height level of sensor 120.

[0087] In the initial orientation stage, the following are performed: the step of adapting to the initial orientation using an optimization algorithm such as a gradient-based optimization (e.g., but not limited to, Stochastic Gradient Descent SGD) algorithm, Broyden-Fletcher-Goldfarb-Shanno (BFGS) algorithm, Limited-memory BFGS (L-BFGS) used for limited memory applications, Adaptive Moment Estimation (Adam) Adam-W (weighted Adam algorithm), Multistage Stochastic Variational Approximation Gradient (M-SVAG), ADAbelief, and the like, and derivatives thereof, based on, for example, a loss function as described herein; and optionally, the step of adding global maximization; and the step of measuring the loss, and if the loss is improved, returning to the start of the initial orientation stage, otherwise continuing with the improvement stage. The step of adding global maximization may include, but is not limited to, the step of sorting sensor 120 based on the loss value, the step of selecting the sensor with the highest loss (e.g., based on a predetermined threshold value), and the step of changing the direction to the maximum intensity for the selected sensor.

[0088] Finally, in the improvement stage, steps of recalculating the positions and orientations of the sensors and emitters, and steps of executing an improvement loop are performed. The improvement loop may include, but is not limited to, improving the model based on the new positions and new orientations of each sensor 120 and emitter 130 using FWI, and improving the orientation based on the model. When a predetermined threshold of improvement is reached, the iteration ends.

[0089] The various embodiments disclosed herein can be implemented as hardware, firmware, software, or any combination thereof. Further, the software is preferably implemented as an application program tangibly embodied on a program storage unit or computer-readable medium consisting of parts, or specific devices and / or combinations of devices. The application program can be uploaded to and executed by a machine having any suitable architecture. Preferably, the machine is implemented on a computer platform having hardware such as one or more central processing units ("CPUs"), memory, and an input / output interface. The computer platform can also include an operating system and microinstruction code. The various processes and functions described herein can be part of the microinstruction code, part of the application program, or any combination thereof, which can be executed by the CPU, whether or not such a computer or processor is explicitly shown. Additionally, various other peripheral units can be connected to the computer platform, such as additional data storage units and printing units. Further, a non-transitory computer-readable medium is any computer-readable medium except a transitory propagation signal.

[0090] All of the examples and conditional language recited in this specification are for the purpose of education to help the reader understand the principles of the disclosed embodiments and the concepts contributed by the inventors to advance the art, and are not to be construed as limited to such specifically recited examples and conditions. Further, all descriptions in this specification listing the principles, aspects, and embodiments of the disclosed embodiments, as well as specific examples thereof, are intended to encompass both their structural and functional equivalents. Further, such equivalents are intended to include both currently known equivalents and equivalents developed in the future, i.e., any elements developed that perform the same function regardless of structure.

[0091] It should be understood that any reference in this specification to an element using terms such as "first," "second," etc. generally does not limit the quantity or order of those elements. Rather, these designations are generally used herein as a convenient way to distinguish between two or more elements or instances of an element. Thus, a reference to a first and a second element does not mean that only two elements may be used there or that the first element must precede the second element in any way. Also, unless otherwise specified, a set of elements includes one or more elements.

[0092] As used herein, following the phrase "at least one of," a list of items follows means that any of the recited items may be utilized individually or any combination of two or more of the recited items may be utilized. For example, if a system is described as including "at least one of A, B, and C," the system can include A alone, B alone, C alone, 2A, 2B, 2C, 3A, a combination of A and B, a combination of B and C, a combination of A, B, and C, a combination of 2A and C, a combination of A, 3B, and 2C, and the like.

Claims

1. 1. A method for determining the orientation of a sensor disposed on an imaging device, comprising: causing emitters of the sensor imager to emit a first set of ultrasonic signals using an emitter control interface, the sensor imager including a plurality of the emitters and a plurality of sensors; measuring the arrival times and amplitudes of the first set of ultrasound signals at the sensor of the imaging device received at a sensor control interface of the imaging device; estimating an initial position of the sensor along a ray trace based on the measured arrival times and amplitudes at the sensor of the imaging device; directing each of the sensor receive lobes to point at a center mass location based on an initial sensor position by performing a geometry receive function; fitting an initial orientation to each of the sensors and each of the emitters by performing an orientation loss function; generating a coarse model based on an initial position of the sensor and the initial position of the emitter; determining a new position and a new orientation for each of the sensors and each of the emitters by implementing the coarse model on a second set of ultrasound signals received by the imaging device; generating an updated model by performing full wave inversion (FWI) based on the determination of the new positions and the new orientations of the sensors and emitters; and determining an improved orientation of each of the sensors and each of the emitters by implementing at least the updated model.

2. The method of claim 1 , wherein at least one outlier sensor-emitter pair is filtered based on signal strength, each outlier sensor-emitter pair having a signal strength below a threshold.

3. The method of claim 1 , wherein at least one outlier sensor-emitter pair is filtered based on characteristics of a signal passing through non-soft tissue that is greater than a predetermined threshold size.

4. The method of claim 1 , wherein fitting the initial orientations to each of the sensors and each of the emitters further comprises performing gradient descent.

5. 5. The method of claim 4, wherein the gradient descent is based on any of stochastic gradient descent (SGD), Broyden-Fletcher-Goldfarb-Shanno, limited-memory Broyden-Fletcher-Goldfarb-Shanno, adaptive moment estimation (ADAM), ADAM-W, multi-stage stochastic variational approximation gradient (M-SVAG), and ADAbelief.

6. The method of claim 1 , further comprising filtering out at least one outlier sensor-emitter pair based on the estimated initial positions of the sensors.

7. further comprising adding a global maximization; The step of adding a global maximization comprises: using the loss function to sort the sensors based on a loss value determined for each sensor; selecting at least one sensor among the sensors for which the determined loss value is above a predetermined threshold; changing the orientation of each of the selected at least one sensor to a direction of maximum intensity; 7. The method of claim 6, further comprising repeating the process of sorting the sensors, selecting at least one sensor, and changing the direction of each selected sensor until a loss value for each sensor is below the predetermined threshold.

8. The method of claim 1 , wherein determining the coarse model further comprises using inverse tomography.

9. The method of claim 1 , wherein the FWI is used to iteratively determine the orientation until a model generated based on the orientation converges.

10. A non-transitory computer-readable medium having stored thereon instructions for causing a processing circuit to perform a process, the process comprising: causing emitters of an imaging device to emit a first set of ultrasound signals using an emitter control interface, the imaging device including a plurality of the emitters and a plurality of sensors; measuring the amplitude and time of arrival of the first set of ultrasonic signals at the sensor of the imaging device received at a sensor control interface of the imaging device; estimating an initial position of the sensor along a ray trace based on the measured arrival times and amplitudes at the sensor of the imaging device; directing each of the sensor receive lobes to point at a center mass location based on an initial sensor position by performing a geometry receive function; fitting an initial orientation to each of the sensors and each of the emitters by performing an orientation loss function; generating a coarse model based on an initial position of the sensor and the initial position of the emitter; determining a new position and a new orientation for each of the sensors and each of the emitters by implementing the coarse model on a second set of ultrasound signals received by the imaging device; generating an updated model by performing full wave inversion (FWI) based on the determination of the new positions and the new orientations of the sensors and emitters; and determining an improved orientation of each of the sensors and each of the emitters by implementing at least the updated model.

11. The process further comprises: The non-transitory computer-readable medium of claim 10 , comprising filtering out at least one outlier sensor-emitter pair based on the estimated initial positions of the sensors.

12. 1. A system for determining the orientation of a sensor disposed on an imaging device, comprising: a processing circuit; a plurality of emitters communicatively connected to the processing circuitry; a plurality of sensors communicatively connected to the processing circuitry; a memory, The memory, when executed by the processing circuitry, causes the system to: causing the plurality of emitters of the imaging device to emit a first set of ultrasound signals using an emitter control interface; measuring the arrival times and amplitudes of the first set of ultrasonic signals at the sensor of the imaging device received at a sensor control interface of the imaging device; estimating an initial position of the sensor along a ray trace based on the measured arrival times and amplitudes at the sensor of the imaging device; directing each of the receive lobes of the plurality of sensors to a center mass location based on the initial positions of the sensors by performing a geometry receive function; performing a loss function orientation to fit an initial orientation to each of the plurality of sensors and each of the plurality of emitters; generating a coarse model based on initial positions of the plurality of sensors and the initial positions of the plurality of emitters; determining a new position and a new orientation for each of the plurality of sensors and each of the plurality of emitters by implementing the coarse model on a second set of ultrasound signals received by the imaging device; generating an updated model by performing full wave inversion (FWI) based on the determination of the new positions and the new orientations of the plurality of sensors and the plurality of emitters; instructions for implementing at least the updated model to determine an improved orientation of each of the plurality of sensors and each of the plurality of emitters.

13. The system of claim 12 , wherein at least one outlier sensor-emitter pair is filtered based on signal strength, and each outlier sensor-emitter pair has a signal strength below a threshold.

14. 13. The system of claim 12, wherein at least one outlier sensor-emitter pair is filtered based on characteristics of a signal passing through non-soft tissue that is greater than a predetermined threshold size.

15. The system further comprises: The system of claim 12 configured to perform gradient descent.

16. 16. The system of claim 15, wherein the gradient descent is based on one of stochastic gradient descent (SGD), Broyden-Fletcher-Goldfarb-Shanno, limited-memory Broyden-Fletcher-Goldfarb-Shanno, adaptive moment estimation (ADAM), ADAM-W, multi-stage stochastic variational approximation gradient (M-SVAG), and ADAbelief.

17. The system further comprises: The system of claim 12 , configured to filter out at least one outlier sensor-emitter pair based on the estimated initial positions of the sensors.

18. The system further comprises: sorting the plurality of sensors based on a loss value determined for each sensor using the loss function; selecting at least one sensor of the plurality of sensors for which the determined loss value exceeds a predetermined threshold; changing the orientation of each of the selected at least one sensor to a direction of maximum intensity; 20. The system of claim 17, configured to repeat the process of sorting the plurality of sensors, selecting at least one sensor, and changing the orientation of each selected sensor until a loss value for each sensor falls below the predetermined threshold.

19. The system further comprises: The system of claim 12 configured to use inverse tomography.

20. The system of claim 12 , wherein the FWI is used to iteratively determine the orientation until a model generated based on the orientation converges.

21. The system further comprises:

20. The system of claim 18, configured to determine the loss value for each sensor based on a full amplitude comparison across all frequencies between the observed measurements and the calculated signal of the sensor.

22. The system further comprises:

20. The system of claim 18, configured to determine the loss value of each sensor based on a ratio of amplitudes at different frequencies.

Images