A method for estimating the flow profile in a shadowed area from ultrasound signal data.

Abstract

This disclosure relates in particular to a computer-implemented method for estimating a fluid flow profile within a pipe, comprising the following steps: - Acquiring a series of ultrasonic signal data containing information about the fluid flow within the pipe. - Identifying shaded regions in the ultrasonic signal data where no information about the fluid flow within the pipe exists. - Determining a fluid flow profile from the acquired ultrasonic signal data for the pipe flow region outside the shaded regions. - Estimating the flow profile within the shaded regions using a physically-informed machine learning method based on the following information: - Flow data containing information about the flow profile determined outside the shaded regions. - Position data containing multiple locations inside and outside the shaded regions and their corresponding time information.

Description

Technical Field

[0001] The present invention relates to a method for estimating a flow profile of a fluid flow in a vessel, particularly a computer-implemented method.

Background Art

[0002] Ultrasonic flow imaging diagnosis has been established as a first-line tool for diagnosing various diseases such as vascular stenosis and congenital heart diseases. Its high frame rate (over dozens of Hz), which cannot be easily achieved by other imaging diagnostic methods such as magnetic resonance imaging (MRI) and computed tomography (CT), provides a unique advantage of accurately capturing complex flow patterns that can be clinically useful.

[0003] On the other hand, ultrasonic flow imaging diagnosis has specific problems in difficult situations such as acoustic shadowing that hinders visualization of the flow field and the corresponding vascular morphology.

[0004] Stroke is one of the major causes of mortality and disability worldwide, along with cancer and ischemic heart disease. Atherosclerosis with plaques as the main symptom is the cause of many stroke cases. In response to microscopic damage in the arterial wall, biochemically induced signals by the endothelium induce repair, inflammatory cells and cholesterol, fat and other substances accumulate at the damaged site, and a fatty plaque protruding into the vascular lumen is formed. As a result, the diameter of the vascular lumen decreases and stenosis finally occurs.

[0005] Ultrasonic imaging diagnosis is a major means for non-invasively diagnosing vascular stenosis and has the advantages of being non-radiative, available at the bedside, and low-cost compared to other imaging diagnostic means. Morphological analysis (ratio-percent method) and the value of peak systolic velocity (PSV) are two common approaches adopted to quantify the degree of stenosis using ultrasound. The corresponding threshold values are used to evaluate the severity and then determine the subsequent diagnosis and treatment strategy.

[0006] However, many plaques are calcified, and calcified plaque produces acoustic shadowing, obscuring the vascular lumen and consequently hindering the examiner's ability to perform ultrasound measurements. For example, in a study involving 400 consecutive carotid artery ultrasound examinations, acoustic shadowing was observed in 14.7% of participants. Acoustic shadowing occurs when highly reflective plaque on the vessel wall is illuminated by the ultrasound beam. This artifact is projected downwards and returns only low-intensity echoes, posing significant problems for morphological and hemodynamic assessment of stenosis. [Overview of the project] [Problems that the invention aims to solve]

[0007] Because it is not possible to determine the flow profile within the shaded area, there is a strong need to address the problem of acoustic shading. [Means for solving the problem]

[0008] The present invention is disclosed by claim 1. Advantageous embodiments are disclosed in the dependent claims.

[0009] A first aspect of the present invention is a method for estimating the flow profile of fluid flow in a pipe, particularly a computer-implemented method, comprising at least the following steps: - A process of acquiring a series of ultrasonic signal data that includes information about the fluid flow inside the pipe. - A step of identifying shadowed regions in ultrasonic signal data that lack information about fluid flow within the pipe. - A process to determine the fluid flow profile from ultrasonic signal data for the flow region of the pipe located outside the shaded area. - A process of estimating the flow profile within a shaded region by a physically informative machine learning method learned using flow data, which includes information about the flow profile determined for the flow region outside the shaded region, and position data, which includes multiple locations inside and / or outside the shaded region and their corresponding time points.

[0010] Using physically-enhanced machine learning, it is possible to more accurately predict the flow in shaded areas within a pipe based on the physical laws and characteristics considered in the physically-enhanced machine learning method.

[0011] The results of the first method more accurately reflect the fluid flow within the pipe than other machine learning techniques trained to predict fluid flow in shaded areas. This is because other methods may not be based on the physical laws and properties of the fluid or the shape of the pipe.

[0012] In particular, it is possible to predict the temporal changes in fluid flow.

[0013] According to another embodiment of the present invention, a flow profile is determined for at least two flow regions. Here, of the at least two flow regions, the first flow region is located upstream of the shaded region, and the second flow region is located downstream of the shaded region. In particular, the fluid flow flows from the upstream region to the downstream region of the shaded region.

[0014] Therefore, the upstream and downstream regions are defined based on the net flow direction. The upstream and / or downstream regions may include one or more pipe sections.

[0015] According to another embodiment of the present invention, the flow region is located adjacent to the shaded region.

[0016] According to another embodiment of the present invention, the shaded area is automatically detected by this method.

[0017] According to another embodiment of the present invention, the shaded region may extend along the net flow direction to 5 mm, 10 mm, 20 mm, 50 mm, 100 mm, 150 mm, or 300 mm. In particular, the diameter of the shaded region, or the tube portion within the shaded region, may be in the range of 0.5 mm to 50 mm, and especially 1 mm to 30 mm.

[0018] According to another embodiment of the present invention, a fully connected deep neural network is used to implement a machine learning method for imparting physical information. The fully connected deep neural network may take spatial and temporal coordinates, i.e., position and time, as input and predict multidimensional velocity components and pressure. For a two-dimensional fluid profile, the inputs are spatial coordinates x, y and the corresponding temporal coordinate t, and the outputs are velocity components u, v and pressure p. For a three-dimensional fluid profile, there is an additional input of a third spatial coordinate z and an additional output of a third velocity component w. The fully connected deep neural network includes multiple hidden layers, each of which may have multiple neurons. A sinusoidal activation function may be used for each neuron. Other activation functions may be used as long as they have a meaningful second derivative for determining the physical loss, as described later.

[0019] According to another embodiment of the present invention, flow data is used to train a physically informative machine learning method to reconstruct velocities at multiple locations and points in time based on a flow data loss function and match them to the velocities included in the flow profile. In particular, the flow data loss function represents the deviation between the reconstructed velocities and the velocities included in the flow profile.

[0020] According to another embodiment of the present invention, the position data can be sampled inside and outside the shadow region, and the position data is used to train a physics-informed machine learning method to reconstruct velocity and pressure for a plurality of positions and time points in the shadow region by applying a physical loss function. By incorporating physical laws in the form of a loss function into the training process, the machine learning method is constrained to generate solutions that respect the physics underlying the flow even in the shadow region.

[0021] According to another embodiment of the present invention, the training of the physics-informed machine learning method is facilitated by minimizing a composite error function that includes a flow data loss function and a physical loss function. The flow data sampled outside the shadow region is used as boundary conditions or initial conditions for deriving a solution. In conventional machine learning methods, the lack of training data may result in non-physical flow profiles in the shadow region. By enforcing physical laws as a form of regularization through the loss function, penalties are imposed on solutions that deviate from these laws. This physics-informed machine learning method can be trained to satisfy the governing equations across the entire region. Thus, it can be generalized to shadow regions where direct measurements do not exist, providing a structured method for reconstructing a consistent and physically plausible high-density flow profile across the entire target region including the shadow region.

[0022] According to another embodiment of the present invention, the composite loss function and its corresponding composite loss value are minimized by the gradient descent method.

[0023] According to another embodiment of the present invention, flow data is generated, particularly sampled from a determined flow profile for a flow region, and the flow data includes information regarding the velocity of the fluid at a plurality of positions at a plurality of corresponding time points.

[0024] Thus, the flow data represents a spatially resolved hydrodynamics or flow profile in an unshaded pipe portion, e.g., in the flow region.

[0025] Generating flow data from ultrasonic signal data is known to those skilled in the art.

[0026] Various methods for determining flow from signal data are available. For example, it is possible to determine a flow profile by the speckle tracking method and the vector Doppler method.

[0027] In particular, the flow data is limited to positions included in the flow region. The flow data may include information regarding the velocity at one or more positions in the flow region at one or more points in time.

[0028] According to another embodiment of the present invention, the learning of the physics-informed machine learning method includes providing information regarding positions and points in time from the flow data as an input to the physics-informed machine learning method, the physics-informed machine learning method determines an estimated velocity for each position and point in time, and the flow data loss function determines a flow data loss value indicating a deviation of the velocity included in the flow data from the velocity estimated by the physics-informed machine learning method.

[0029] Note that in the context of this specification, it should be noted that the terms "flow data loss" and "data loss" may be used synonymously. Since the flow data including information regarding the flow profile can be physically modeled by the dynamic information regarding the flow profile of the fluid, it is particularly suitable for the physics-informed machine learning method.

[0030] This embodiment enables, for example, learning the physics-informed machine learning method to reconstruct the velocity in a region outside the shadow region. This makes it possible to match the velocities, and further contributes, for example, as a constraint to the physical loss function based on the Navier-Stokes equations.

[0031] According to another embodiment of the present invention, training a physical information-enhancing machine learning method includes providing at least some locations and their corresponding time points from location data as input to the physical information-enhancing machine learning method, which determines estimated fluid velocity and estimated pressure for each location and each time point, the estimated velocity and estimated pressure being provided to a physical loss function including Navier-Stokes equations, the physical loss function determining physical loss values ​​that represent the deviations of the velocity and pressure estimated by the physical information-enhancing machine learning method from the velocity and pressure solving the Navier-Stokes equations.

[0032] By assigning the Navier-Stokes equations to the physical loss function, an appropriate physically informed and physically forcibly generated loss function is provided, constraining machine learning estimations to a physically meaningful flow profile even in shaded areas.

[0033] According to another embodiment of the present invention, learning comprises a first, second, and third stage, in which a data loss value is determined in the first stage, a physical loss value is determined in the second stage, and a composite loss value is determined from a composite loss function that includes a data loss function and a physical loss function, in particular the composite loss function which includes the sum of the data loss function and the physical loss function, in particular a weighted sum, and the composite loss function is minimized during the learning episode.

[0034] This three-stage process allows for the modeling of a robust learning process in which the fluid data loss function and the physical loss function can be separated, and their contributions to the output estimation of the fluid profile can be adjusted by a combined loss function.

[0035] According to another embodiment of the present invention, the Navier-Stokes equations correspond to Navier-Stokes equations adjusted to fit incompressible fluids, momentum conservation, and mass conservation.

[0036] According to another embodiment of the present invention, the flow profile is determined in either three or two dimensions.

[0037] This makes it possible to apply this method to various ultrasonic recording systems designed to record two-dimensional or three-dimensional fluid data.

[0038] According to another embodiment of the present invention, when the flow profile is determined in three dimensions, the physical loss function includes physical residual functions f1, f2, f3, f4 based on terms of the Navier-Stokes equations.

number

[0039] According to another embodiment of the present invention, when the flow profile is determined in two dimensions, the physical loss function includes physical residual functions f1, f2, f3 based on terms of the Navier-Stokes equations.

number

[0040] According to another embodiment of the present invention, when the flow profile (100) is determined in three dimensions, the data loss function and data loss value L data This is determined according to the following formula.

number

[0041] According to another embodiment of the present invention, when the flow profile (100) is determined two-dimensionally, the data loss function and data loss value L data This is determined according to the following formula.

number

[0042] According to another embodiment of the present invention, the composite loss value L composite L composite =a|L data |+b|L physicsIt is determined according to |, where a and b are positive numbers for weighting the contributions of the physical loss value and the data loss value.

[0043] According to another embodiment of the present invention, the tube portion is a blood vessel in a living organism.

[0044] The shadowed areas can be caused by calcified portions of the blood vessel walls.

[0045] According to another embodiment of the present invention, the method is performed during ultrasound imaging of a patient, and the learning of the physical information-assigning machine learning method is performed at least once for each patient.

[0046] This makes it possible to predict patient-specific fluid profiles in shadowed areas.

[0047] It should be noted that sufficient training data can be acquired quickly because regions adjacent to the shaded areas provide a temporal flow profile that may be repeated after each heartbeat. Therefore, training data can be easily acquired with one or a few heartbeats.

[0048] The acquisition time can range from 0.5 seconds to 10 seconds.

[0049] According to another embodiment of the present invention, the pipe boundary is determined for at least the flow region, and the velocity of the determined flow profile is set to zero outside the boundary, the flow region, and the shaded region.

[0050] This enables stable estimation of flow profiles in shaded areas.

[0051] According to another embodiment of the present invention, the flow profile in the shaded region is determined by inputting corresponding spatial and temporal coordinates, such as positional data and corresponding time points, from the velocity estimated using a trained physical information-enhancing machine learning method.

[0052] According to another embodiment of the present invention, a machine learning method that assigns pre-trained physical information estimates the time-resolved velocity and / or pressure distribution for at least a portion of the shaded region, particularly the entire shaded region.

[0053] This makes it possible to evaluate and determine key parameters that characterize the flow conditions within the pipe, without requiring more complex or invasive measurement systems.

[0054] According to another embodiment of the present invention, the velocity and / or pressure estimated by a trained physical information-granting machine learning method are displayed, in particular the estimated velocity and / or estimated pressure are displayed in combination with flow regions and shaded regions.

[0055] According to another embodiment of the present invention, a sequence of velocity and / or pressure estimated by a trained physical information-assigning machine learning method is generated, particularly in the form of a video. In this way, a temporal change in velocity and / or pressure, particularly the temporal change in the velocity distribution and / or estimated pressure distribution, is generated. The sequence is displayed together with the flow profile, particularly with the velocity in the flow region, thereby showing a flow profile in which the flow region and shaded region change continuously over time, particularly in the form of a video.

[0056] This allows observers to evaluate the dynamic behavior within the tube even in shaded areas.

[0057] According to another embodiment of the present invention, the flow profile in the flow region is determined by tracking tracers contained in the fluid.

[0058] Such tracers may contain microbubbles.

[0059] According to another embodiment of the present invention, the method includes a segmentation step of the tubular section, in which at least the boundaries of the tubular section, more specifically the lumen of the tubular section, are determined. This step is performed together with, and in particular simultaneously with, a step of estimating the flow profile.

[0060] A second aspect of the present invention is a computer program that includes computer program code, which, when executed on a computer, causes the computer to perform the method described in any of the claims.

[0061] Computer programs can be stored in non-temporary storage media.

[0062] A third aspect of the present invention comprises the following components: - An ultrasonic transducer head system configured to generate and record ultrasonic signal data; - A computer connected to an ultrasonic transducer head system and configured to receive ultrasonic signal data. An ultrasonic measurement system comprising at least the following: This is a system in which a computer is configured to execute a method according to the first aspect of the present invention or a computer program according to the second aspect of the present invention.

[0063] By utilizing flow data available in the flow regions upstream and downstream of the acoustically shadowed area, this method aims to reconstruct missing flow profiles and pipe boundary information under acoustic shadowing, enabling immediate application to the analysis of vascular stenosis.

[0064] Basic operating principle / Concept of the invention The inventors have discovered a remarkable method for estimating flow profiles in shaded areas where flow data cannot be acquired.

[0065] The core of this invention is a physical information-enhancing machine learning method, which, by design, generates solutions that follow, or at least substantially follow, physical laws.

[0066] In relation to this invention, the underlying physical principles include the principles or laws of fluid dynamics. By penalizing the machine learning method during training in accordance with deviations from the underlying physical principles or laws, the machine learning method becomes biased toward physically meaningful estimations. That is, the estimation results fit to a high degree the physical principles underlying the observed natural phenomena, in this case, for example, fluid flow through blood vessels.

[0067] This inventive concept extends, in particular, to the use of one or more equations, which are described to function as error functions that penalize deviations from these equations, i.e., deviations from physically expected solutions.

[0068] The present invention aims to determine fluid flow in a given region. For this purpose, the machine learning method for imbuing physical information includes an error function that, at least in part, incorporates the laws of fluid dynamics. In the context of this specification, this part is referred to as the "physics loss function."

[0069] However, even with this knowledge, the inventors faced the problem of how to generate solutions that follow a physical loss function in the shaded region while preventing the generation of unrealistic outputs, such as solutions involving velocity and pressure.

[0070] The inventors have recognized that outside the shaded region, particularly in the flow region adjacent to the shaded region, there is information available to prevent the generation of unrealistic solutions that minimize the physical loss function but are nevertheless physically meaningless.

[0071] To bias the machine learning method for imparting physical information towards physically meaningful solutions, the inventors recognized that the machine learning method can be trained using an error function for the flow region outside the shaded region to recover / reconstruct the velocity in the flow region, i.e., the flow profile. This error function is called the data loss function. The data loss function does not necessarily have to be an equation that models a physical process, but can be any function suitable for training the machine learning method for imparting physical information to reconstruct a (known) flow profile. For example, it could be the chi-squared deviation between the estimated velocity at a certain location and time and the velocity determined from flow data at the same location and time.

[0072] Therefore, this method requires only position and time as input, and ultimately (after training) can estimate the velocity at that position and time. Clearly, the trained method can predict velocity for almost any position and any time in the flow region outside the shaded area.

[0073] This learning process can be performed based on acquired data and does not require a specific training dataset.

[0074] In this way, if we obtain the possibility of essentially generating velocity boundary conditions at the interface between the shaded region and the flowing region, the aforementioned physical loss function can be constrained to solutions that connect to these boundary conditions. Therefore, training a machine learning method for the physical loss function can also be performed by inputting position and time inside and outside the shaded region, in which case the velocity and pressure governed by the physical loss function are determined in such a way that the velocity smoothly connects to the flow profile in the flowing region.

[0075] For example, the physical loss function based on the Navier-Stokes equations is sufficiently detailed and does not require any additional conditions other than the velocity, which can be reconstructed by the boundary or initial conditions already present in the flow domain, i.e., the data loss function.

[0076] The following diagram description demonstrates that a machine learning method trained on these two error functions enables highly accurate estimation of the flow profile, velocity, and / or pressure in the shaded region of the pipe.

[0077] Furthermore, this method is designed to allow for the reconstruction of flow profiles, velocity, and pressure at points in time before or after dataset acquisition. To balance the effects of the data loss function and the physical loss function, it may be advantageous to integrate these two functions as a weighted composite function.

[0078] For example, each function may have the same weight, i.e., a 1:1 (50:50) weighting. Alternatively, it may be advantageous to use different weightings. A person skilled in the art can select weightings by experimenting with different weightings or based on performance parameters that indicate the optimal weighting.

[0079] Exemplary embodiments are described below with reference to drawings. The drawings are attached to the claims and accompanied by text describing the individual features of the embodiments and aspects of the invention shown. Each individual feature shown in the drawings and / or described in the text of the drawings may be incorporated (even individually) into a claim relating to an apparatus according to the present invention. [Brief explanation of the drawing]

[0080] [Figure 1] Figure 1 shows an exemplary model for data acquisition. [Figure 2] Figure 2 shows one frame of the acquired ultrasonic signal data. [Figure 3] Figure 3 shows the pulsating flow profiles of the liquid in the tube at three different time points. [Figure 4] Figure 4 demonstrates the ability of this method to estimate the flow profile in shaded areas. [Figure 5] Figure 5 shows the ability of this method to determine the area occupied by the pipe section. [Figure 6]Figure 6 shows another example of fluid profile reconstruction in a tubular section containing a constriction within the shaded area, using the method of the present invention. [Modes for carrying out the invention]

[0081] Figure 1A shows a model of the core of polylactic acid (PLA) tube 1, including branching for demonstration. The core of tube 1 is embedded in a material that provides an ultrasound signal similar to that of human tissue. In the embodiment described here, the PLA tube core is dissolved with chloroform, and then a wallless carotid artery branching fluid model is constructed using polyvinyl alcohol (PVA) cryogel. The diameters of the common carotid, internal carotid, and external carotid artery branches are 6 mm, 4.2 mm, and 3.5 mm, respectively, all of which are within the range of normal carotid artery sizes for adults. At the entrance of the internal carotid artery branch, 50% eccentric stenosis (according to the North America Symptomatic Carotid Endarterectomy Trial (NASCET) criteria) is added to construct a disease state with vascular stenosis.

[0082] Figure 1B shows an experimental apparatus for carrying out the method according to the present invention. After removing this core with chloroform, the wallless channel is connected to tube 3 via a hose connector to allow liquid to flow through it using a pump device 2.

[0083] Ultrasonic imaging is performed by repeatedly transmitting short pulses and beamforming the echoes received by the transducer head 4 for each transmission. With unfocused ultrasonic (plane wave / divergent wave) beams, images can be reconstructed using a single pulse transmission, resulting in high frame rates (approximately several kHz) useful for fluid imaging. With focused beams, the region of interest must be scanned sequentially line by line, resulting in lower frame rates (determined by the density of imaging lines). Note that the fluid and boundary reconstruction methods described here are not limited to the type of ultrasonic transducer and ultrasonic beam employed.

[0084] The experiment is conducted using a fabricated model and pulsating flow. The pulsating flow is 60 strokes / min, 3 mL / stroke, and is generated by a pulsating flow pump 2 (Model 1405, Harvard Apparatus, Massachusetts, United States). This flow rate is within the range of human carotid artery blood flow. SonoVue microbubbles diluted 1:2000 are used as tracers for flow estimation (this method is not limited to the use of microbubble contrast agents). For ultrasonic excitation generation and data acquisition 5, the Ultrasound Array Research Platform IIa (UARP IIa), developed by the Ultrasonics and Embedded Systems Group at the University of Leeds, is used. The UARP IIa system features a quintuple excitation method and harmonic reduction method. A Verasonics L11-4v linear array transducer (Verasonics, Inc., WA, USA) is connected to the UARP IIa and excited with a 2-cycle sinusoidal waveform with a center frequency of 7.55 MHz. 0-degree unfocused plane wave imaging is performed, scanning the branching flow model with a pulse repetition frequency of 6 kHz (corresponding to a 6 kHz frame rate) and a mechanical index of 0.1. Ultrasonic RF (Radio Frequency) data is acquired at a sampling frequency of 40 MHz. The acquired RF data is transferred to a local PC 5, where two-dimensional motion estimation (at each pixel) is performed by determining beamforming and inter-frame displacement. The results and control information can be displayed on the system display 6 shown in Figure 1B. Note that the system according to the present invention typically does not include a flow pump, tubing, and model.

[0085] Ultrasound images are reconstructed using a delay-and-sum beamforming algorithm. A singular value decomposition (SVD) filter is applied to the image sequence to remove tissue background signals. Subsequently, two-dimensional vector flow mapping is performed on the beamformed and SVD-filtered RF frames using a correlation-based motion estimation method. The resulting flow map has a frame rate of 6 kHz.

[0086] Figure 2A shows a single frame (#01131) of a recording of a wallless fluid model, i.e., an embedded tube 1, acquired by an ultrasonic measurement system. In this example, a tracer is included to enhance the contrast of the fluid. However, blood can be recorded without a tracer, as it can provide sufficiently high contrast in the image to estimate the fluid profile of the tube.

[0087] Figure 2B shows the filtered image, where the background signal from the tissue material, which is the embedding material for tube section 1, has been removed by a software filter.

[0088] This makes it possible to determine the pipe boundary in the flow region.

[0089] Figure 3 shows the flow profiles obtained from single-frame ultrasonic signal data for three different time points. Figure 3A is time point 814, Figure 3B is time point 1072, and Figure 3C is time point 1612. In particular, velocity vectors are shown for multiple locations within the pipe at each time point. The color coding indicates the magnitude of the velocity.

[0090] In Figure 4A, the data is prepared so that a shaded region is generated by deleting the information contained in the region located between the two flow regions. One flow region is located upstream of the shaded region, and the other flow region is located downstream of the shaded region.

[0091] This method determines flow data, including velocity at various time points and locations, from two flow regions. It can also determine the segmentation of the pipe boundary available outside the acoustic shadow. For the tissue background outside the acoustic shadow, the flow velocity value is assigned as zero (no flow). A physical information-enhancing neural network is constructed for flow determination based on the flow data, and missing flow and pipe boundary information is supplemented.

[0092] Determining flow characteristics based on flow data and constructing a physical information-adding neural network to supplement missing flow information and pipe boundary information. In one implementation, shaded regions are artificially generated by removing data (tissue background and flow) beneath the pipe boundary. This configuration makes ground truth available for evaluating the performance of the method according to the present invention. To implement the physically informative machine learning method, the inventors utilize the ability of deep neural networks as universal function approximators to identify nonlinear mappings between spatiotemporal coordinates and flow parameters. A deep neural network is defined and trained to match velocity data available over a period of time, while the solution is constrained to respect prior knowledge of computational fluid dynamics that explains the observed data. Under this setting, the time-dependent characteristics of the flow pattern are captured by the trained neural network and subsequently used to estimate the spatiotemporal flow field under acoustic shading. This physically informative network accepts spatial coordinates (x,y) and temporal coordinates (t) as input. In one embodiment, the physically informative neural network has 10 fully connected hidden layers, each consisting of 120 neurons, and outputs two-dimensional velocity components (u,v) and pressure (p). In other words, a physically information-assigned neural network maps the input (x,y,t) to the output (u,v,p). A hyperbolic tangent or sine function is used as the nonlinear activation function for each neuron. To train the neural network, flow data (pixel position [x,y]) and time information (t) corresponding to each learning point are provided as input. To penalize the neural network, the velocity mismatch between its output velocity component [u,v] and the measured reference value is used. The Navier-Stokes equations are valid as physical constraints within the flow domain. During the training phase, it is also proposed to enforce physical constraints across the entire domain of interest, including the observable flow and tissue background regions. For the tissue background region, a velocity of zero is assigned. Visible zero-velocity tissue background regions are also included in the calculation of the velocity data mismatch described above, and velocity predictions far from zero are penalized.With this setting, the trained neural network can be used for prediction by inputting arbitrary spatial coordinates, including those present within the tissue background under acoustic shading (the predicted velocity values ​​will approach zero). This implementation uses 200ms of data, including the peak velocity period. The neural network parameters are optimized over 20,000 iterations using the Adam optimization algorithm. The learning rate starts at 1e-3 and decreases by 25% every 1,000 iterations.

[0093] After the neural network has been trained, it is used to reconstruct the flow field, including dark acoustic shading regions, by inputting corresponding spatial and temporal coordinates.

[0094] Figure 4A shows an example of flow data 100 with an acoustic shadow region 103 (outside the dark acoustic shadow 103 and within the background region 105, the two-dimensional velocity component is set to zero). Figure 4B shows the ground truth for which all ultrasonic signals are available. Figure 4C shows the corresponding estimation results by the method according to the present invention.

[0095] Figure 4A shows flow data 200 representing a single recorded point in time of the fluid flow through the pipe. The pipe is branched, and shaded areas are located at the branching points.

[0096] The shaded region 103 is added by generating ground truth (see Figure 4B) and then artificially removing all flow information from this shaded region 103.

[0097] In this way, the ground truth becomes known and can be used for benchmark evaluation of the method according to the present invention.

[0098] Figure 4B shows the flow data 200 recorded without any shaded areas.

[0099] Figure 4C shows the solution generated by the method according to the present invention using the deep neural network and learning parameters described above (10 fully connected hidden layers, 120 neurons in each layer, 200 ms of data usage including a peak velocity period, 20,000 iterations using the Adam optimization algorithm, and a learning rate starting at 1e-3 and decreasing by 25% every 1,000 iterations).

[0100] Figure 4A shows that in the previously shaded region, a flow profile 104 is estimated that is very similar to the flow profile of the recorded flow data (Figure 4B). Although Figure 4 shows only the flow profile at one time point, it is clear that this method can be applied with similar results to all frames of the recorded flow data. The estimated velocity for the shaded region 103 is highly accurate. The mean error of the lateral velocity component in the shaded region is 2.43 ± 0.26 cm / s, and the mean error of the axial velocity component is 2.14 ± 0.35 cm / s. These values ​​correspond to an error of approximately 4% relative to the peak value of approximately 55 cm / s.

[0101] The grayscale indicates the recorded or estimated speed within pipe section 1.

[0102] An additional advantage of this method is that, based on the velocity map generated by this method, it is possible to visualize the vascular boundaries (subvascular boundaries) beneath the acoustically shaded region. This is useful in anatomical analysis.

[0103] Figure 5 shows an embodiment of the present invention that enables the determination of the boundary of pipe section 1 from the estimated flow profile. Figure 5A shows the estimated pipe section boundary determined by an expert.

[0104] In Figure 5B, the boundary of the same blood region is estimated by analyzing the flow profile predicted by mathematical operations such as the Otsu method using this method. The results are remarkably similar.

[0105] Please note that this method can be used to estimate the boundaries of tubes within shaded areas.

[0106] Regions where pipe section 1 exists are shown in black, and regions where pipe section 1 does not exist are shown in white. The boundary is at the transition point from black to white.

[0107] Figure 6 shows another example of flow profile reconstruction in a shaded region within a tube, similar to Figure 4. In this case, the tube is not branched, but the diameter decreases in the shaded region, indicating, for example, a constriction. The training of the physically informative machine learning method is essentially the same as in Figure 4 and the above description of the training procedure. The neural network used has the same structure, i.e., the same number of layers and neurons as used in Figure 4.

[0108] This example demonstrates that the method can be applied to various situations without prior knowledge of pipe geometry, etc. It is noteworthy that the reconstructed flow profile shows a high degree of agreement with the actual flow profile (the flow profile of the shaded region hidden to demonstrate the capabilities of the present invention), and furthermore, the prediction of the pipe geometry within the shaded region also matches.

[0109] Figure 6A shows an example of recorded flow data 100 that includes an acoustically shaded region 103 with a length of approximately 10 mm along the flow direction. The flow data was obtained from a blood vessel with stenosis in the shaded region. This dataset, like the dataset in Figure 4A, was created by modifying flow data obtained from ground truth and artificially applying a shaded region. Therefore, the flow regions 101 and 102 upstream and downstream of the shaded region 103 are identical to the corresponding regions in ground truth and correspond to realistic measurement conditions.

[0110] Figure 6B shows the ground truth for which all ultrasonic signals are available.

[0111] Figure 6C shows velocity 104 reconstructed within the shaded region by the method according to the present invention.

[0112] Figure 6A shows flow data 200 representing a single recorded point in time of the fluid flow through the pipe. In contrast to Figure 4, this pipe does not branch and has a constriction in the central region.

[0113] The shaded region 103 is generated by artificially removing all flow information from the recorded data in Figure 6B.

[0114] In this way, the ground truth becomes known and can be used to evaluate the performance of the method according to the present invention.

[0115] Figure 6C shows the solution generated by the method according to the present invention.

[0116] Figure 6A shows that in the previously shaded region, a flow profile 104 very similar to the flow profile of the recorded flow data (Figure 6B) has been estimated. Although Figure 6 shows only one time point, this method is applicable to all frames of the recorded flow data and similar results can be obtained.

[0117] The specific experimental conditions for the data shown in Figure 6 are as follows:

[0118] The experiment in Figure 6 was performed using a walled, constricted PVA straight tube model with an inner diameter of 6 mm. The pulsating flow was 60 strokes / min and 2 mL / stroke. Coherent plane-wave imaging was performed at three steering angles of -3 degrees, 0 degrees, and 3 degrees, with a pulse repetition frequency of 6 kHz and an effective frame rate of 2 kHz. Other experimental equipment, beamforming, and signal processing procedures were the same as those for the branched flow experiment in Figure 4. Motion estimation was performed using the corresponding low-resolution beamforming image pair before compounding. 200 ms of flow data was processed by a fully connected deep neural network with the same structure and training settings as those used for the branched flow data in Figure 4. The accuracy of flow profile reconstruction in the shaded region was as follows: lateral mean velocity error (time average) 1.71 ± 0.16 cm / s, axial mean velocity error (time average) 0.54 ± 0.14 cm / s.

[0119] These results demonstrate the high accuracy and reliability of the method according to the present invention.

Claims

1. A method for estimating the flow profile (100) of fluid flow within a pipe (1), particularly a computer implementation method, comprising the following steps: - A step of acquiring a series of ultrasonic signal data (200) that includes information about the fluid flow inside the pipe section (1), - A step of identifying shadowed regions (103) in the ultrasonic signal data that lack information regarding the fluid flow within the pipe (1), - A step of determining the fluid flow profile (100) from ultrasonic signal data for the flow regions (101, 102) of the pipe section (1) located outside the shadowed region (103), - A step of estimating the flow profile (104) within the shaded region (103) by a physically-informed machine learning method learned using flow data including information on the flow profile (100) determined for the flow regions (101, 102) outside the shaded region (103), and position data including multiple locations inside and / or outside the shaded region (103) and corresponding time points.

2. The method according to claim 1, wherein a flow profile (100) is determined for at least two flow regions (101, 102), of which the first flow region (101) is located upstream of the shaded region (103), and the second flow region (102) is located downstream of the shaded region (103), and in particular the fluid flow flows from the upstream region (101) to the downstream region (102) of the shaded region (103).

3. The method according to any one of claims 1 to 2, wherein flow data (200) is generated from a flow profile determined for a flow region, and the flow data (200) includes information about the velocity of the fluid at multiple locations at multiple corresponding time points.

4. The method according to any one of claims 1 to 3, wherein the training of the physically information-enhanced machine learning method includes providing location and time information from flow data (200) as input to the physically information-enhanced machine learning method, the physically information-enhanced machine learning method determines an estimated velocity for each location and each time, and the flow data loss function determines a flow data loss value that indicates the deviation of the velocity contained in the flow data (200) from the velocity estimated by the physically information-enhanced machine learning method.

5. The method according to any one of claims 1 to 4, wherein the training of a physically identifiable machine learning method includes providing at least some locations and their corresponding time points from location data as input to the physically identifiable machine learning method, the physically identifiable machine learning method determines estimated fluid velocity and estimated pressure for each location and each time point, the estimated velocity and estimated pressure are provided to a physical loss function including Navier-Stokes equations, the physical loss function determines physical loss values ​​that represent the deviation of the velocity and pressure estimated by the physically identifiable machine learning method from the velocity and pressure obtained by solving the Navier-Stokes equations.

6. The method according to claim 4 or 5, wherein the learning comprises a first stage, a second stage and a third stage, in which a data loss value is determined; in the second stage, a physical loss value is determined; and in the third stage, a composite loss value is determined from a composite loss function comprising a data loss function and a physical loss function, in particular the composite loss function comprising the sum of the data loss function and the physical loss function, in particular a weighted sum, and the composite loss function is minimized during the learning episode.

7. The method according to any one of claims 1 to 6, wherein the flow profile (100) is determined in three or two dimensions.

8. When the flow profile is determined in three dimensions, the physical loss function is the physical residual function f based on the Navier-Stokes equations. １ , f ２ , f ３ , f ４ : [Math 1] The method according to claim 7, including the method described in claim 7.

9. When the flow profile (100) is determined in two dimensions, the physical loss function is the physical residual function f based on the Navier-Stokes equations. １ , f ２ , f ３ : [Math 2] The method according to claim 7, including the method described in claim 7.

10. When the flow profile (100) is determined in three dimensions, the data loss function and data loss value L data The formula is as follows: [Math 3] The method according to claim 7 or 8, determined according to the following.

11. When the flow profile (100) is determined two-dimensionally, the data loss function and the data loss value L data are given by the following equation: [Math 4] The method according to claim 7 or 9, as determined accordingly.

12. The method according to any one of claims 1 to 11, wherein the method is performed during ultrasound imaging of a patient, and the learning of the physical information-assigning machine learning method is performed at least once for each patient.

13. The method according to any one of claims 1 to 12, wherein the time-resolved velocity and / or pressure distribution are estimated by a trained physical information-assigning machine learning method for at least a portion of the shaded region (103), particularly the entire shaded region.

14. A computer program that, when executed on a computer, includes computer program code that causes a computer to perform the method described in any one of claims 1 to 13.

15. The following components: - An ultrasonic transducer head system configured to generate and record ultrasonic signal data; - A computer connected to an ultrasonic transducer head system and configured to receive ultrasonic signal data. An ultrasonic measurement system comprising at least the following: The system wherein the computer is configured to execute the method according to any one of claims 1 to 13 or the computer program described in claim 14.

Images