Method and apparatus for predicting physical parameters based on input physical information

Abstract

The present disclosure relates to a method and apparatus for predicting a physical parameter based on input physical information. The method can include predicting, by a processor, an intermediate variable using an intermediate sub-model based on input physical information, the intermediate sub-model combining prior information of the physical parameter with a constraint condition on the intermediate variable. The method can also include converting, by the processor, the intermediate variable predicted by the intermediate sub-model into the physical parameter using a conversion intermediate sub-model.

Description

[0001] Cross-references to related applications

[0002] This application is based on and claims priority to U.S. Provisional Application No. 63 / 081,279, filed September 21, 2020, which is incorporated herein by reference in its entirety. Technical Field

[0003] This disclosure relates to the prediction of physical parameters using machine learning, and more specifically, to methods and apparatus for predicting physical parameters, such as the fractional flow reserve (FFR) value of a blood vessel, using prior information about physical information as constraints. Background Technology

[0004] Machine learning has been used as an indispensable tool in many fields, such as insurance (premium prediction), healthcare (medical diagnosis, development, and growth), and agriculture (plant growth), for modeling complex functions. As the complexity of the learning model increases, the predictive power for various complex problems in practical applications can be improved. However, because the learning model is primarily configured to derive a mapping function from input physical information to output physical parameters based on training data (as a black box), the prediction results may not conform to the fundamental rules governing physical parameters. For example, the premium predicted by the learning model may decrease with age (contradicting the fundamental rule that premiums increase with age). As another example, the height of a child predicted by the learning model may decrease as the child grows (contradicting the fundamental rule that a child's height should increase). As yet another example, the blood flow pressure predicted by the learning model may increase from upstream to downstream in a vascular tree (contradicting the fundamental rule that blood flow pressure decreases from upstream to downstream in a vascular tree).

[0005] To compensate for the fact that learning models often ignore the fundamental rules governing the physical parameters to be predicted, some traditional methods consider information related to these fundamental rules through post-processing steps. However, these methods require additional steps, which can degrade the performance of the learning model. Other methods may use an additional loss term in the loss function to penalize predictions that contradict the fundamental rules during the training phase. Taking the monotonic curves of physical parameters in a sequence as an example, an additional loss term designed to penalize non-monotonic predictions is applied to the loss function during the training phase. However, low non-monotonic loss in the training data does not necessarily indicate low non-monotonic loss on all test data, especially when the model overfits relative to the training data. More importantly, strict monotonicity of the predictions is not guaranteed.

[0006] There is still room for improvement in learning models, especially for those that want to use prior information to model complex functions. Summary of the Invention

[0007] This disclosure addresses the aforementioned problems in the prior art. It provides methods, apparatus, and computer-readable media for predicting physical parameters based on input physical information using a learning model, which can enforce prior information about the physical parameters as constraint functions into the architecture of the learning model without additional loss terms or post-processing steps. Therefore, it can force the prediction results to substantially conform to basic rules, thereby improving model performance.

[0008] According to a first aspect of this disclosure, a method for predicting physical parameters based on input physical information is provided. The method may include a processor predicting intermediate variables using an intermediate sub-model based on the input physical information, the intermediate sub-model incorporating prior information about the physical parameters and constraints on the intermediate variables. The method may further include the processor converting the intermediate variables predicted by the intermediate sub-model into the physical parameters using a transformation sub-model.

[0009] According to a second aspect of this disclosure, an apparatus for predicting physical parameters based on input physical information is provided. The apparatus may include a memory and a processor. The memory may be configured to load or store intermediate sub-models and transformation sub-models. The processor may be configured to: predict intermediate variables based on the input physical information using the intermediate sub-models, the intermediate sub-models combining prior information about the physical parameters with constraints on the intermediate variables; and convert the intermediate variables predicted by the intermediate sub-models into the physical parameters using the transformation sub-models.

[0010] According to a third aspect of this disclosure, a non-transitory computer-readable medium storing computer-executable instructions is provided. When the computer-executable instructions are executed by a processor, a method for predicting physical parameters based on input physical information can be implemented. The method includes predicting intermediate variables based on the input physical information using an intermediate sub-model, the intermediate sub-model incorporating prior information about the physical parameters and constraints on the intermediate variables. The method may further include converting the intermediate variables predicted by the intermediate sub-model into the physical parameters using a transformation sub-model.

[0011] The aforementioned methods, apparatus, and media can force the prior information of physical parameters into the architecture of the learning model as a constraint function without additional loss terms or post-processing steps, thereby ensuring that the prediction results basically conform to the basic rules and improving model performance.

[0012] The above general description and the following detailed description are exemplary and illustrative only and are not intended to limit the claimed invention. Attached Figure Description

[0013] In drawings that are not necessarily drawn to scale, the same reference numerals may describe similar parts in different views. Similar reference numerals with different letter suffixes may indicate different examples of similar components. The drawings generally illustrate various embodiments by way of example rather than limitation, and are used together with the specification and claims to illustrate the disclosed embodiments. Such embodiments are illustrative and exemplary, and are not intended to be exhaustive or exclusive embodiments of the method, apparatus, system, or non-transitory computer-readable medium having instructions for implementing the method.

[0014] Figure 1 A schematic diagram illustrating an exemplary framework of a physical parameter prediction model according to an embodiment of the present disclosure is shown.

[0015] Figure 2 A flowchart illustrating an exemplary method for predicting physical parameters based on input physical information according to an embodiment of the present disclosure is shown.

[0016] Figure 3 A flowchart illustrating an exemplary method for training a physical parameter prediction model according to an embodiment of the present disclosure is shown.

[0017] Figure 4 A schematic diagram of an exemplary physical parameter prediction model according to an embodiment of the present disclosure is shown.

[0018] Figure 5 A schematic diagram of another exemplary physical parameter prediction model according to an embodiment of the present disclosure is shown.

[0019] Figure 6 A schematic block diagram of an exemplary apparatus for predicting physical parameters based on input physical information according to an embodiment of the present disclosure is shown.

[0020] Figure 7 This is a schematic block diagram of an exemplary system for predicting physical parameters based on input physical information according to embodiments of the present disclosure. Detailed Implementation

[0021] Reference will now be made in detail to exemplary embodiments, examples of which are shown in the accompanying drawings.

[0022] In this disclosure, "physical information" can be any information collected or acquired in various technical fields governed by certain physical rules. Physical information can be acquired in various formats, such as, but not limited to, sequences of data, vectors, image patches, lists, etc. Accordingly, the "physical parameter" to be predicted can be a physical parameter relating to physical information in the corresponding technical field. For example, in the technical field of insurance, the age and health information of the insured can be used as physical information, and the insured's premium can be set as the physical parameter to be predicted. As another example, in the technical field of healthcare, such as the diagnosis of coronary artery stenosis, a sequence of image patches in a coronary tree can be used as physical information, and a sequence of fractional flow reserve (FFR) or instantaneous waveless rate (iFR) in the coronary tree can be set as the physical parameter to be predicted. In this disclosure, "prior information about the physical parameter" can include known or determined knowledge about the predicted physical parameter, such as fundamental rules governing the physical parameter or its transformation parameters according to physical principles or theories. In the exemplary technical field of insurance, an example of prior information could be that the premium must increase with the insured's age and the deterioration of their medical condition. In exemplary technical fields for diagnosing coronary artery stenosis, an example of prior information might be that the FFR value from downstream of the coronary tree should not be higher than the value from upstream of the coronary tree.

[0023] Figure 1 A schematic diagram illustrating an exemplary framework of a physical parameter prediction model 100 according to an embodiment of the present disclosure is shown. The physical parameter prediction model 100 can model a predetermined relationship between physical parameters and input physical information; for example, the physical parameters are an objective function of the physical information. Figure 1 As shown, instead of directly modeling the objective function, the physical parameter prediction model 100 can generally be divided into two sub-models: a constrained intermediate sub-model 103 and a transformation sub-model 104. The constrained intermediate sub-model 103 can be configured to receive physical information as input 101, where the physical information can be obtained from a specific technical field. When the constrained intermediate sub-model 103 is applied by the processor, it can be configured to predict intermediate variables based on the received physical information, and this prediction can be adjusted by constraints that conform to prior information governing the technical field from which the physical information was obtained. The transformation sub-model 104 then maps the intermediate variables to physical parameters. As a result, the physical parameter prediction model 100 can be applied to predict physical parameters from the input physical information, taking into account prior information.

[0024] As in Figure 1As shown, the constrained intermediate sub-model 103 may include an unconstrained intermediate sub-model 103a and a constraint function 103b. The constrained intermediate sub-model 103 may be based on prior information about physical parameters, combined with constraints on intermediate variables, such as constraint function 103b.

[0025] In some embodiments, prior information about physical parameters may include curve trends (particularly physical parameters as a sequence) and / or boundary ranges (e.g., positive, negative, or within a range defined by a lower and / or upper limit) in the time and spatial domains. In some embodiments, for a sequence of physical parameters, curve trends may include any of the following: the monotonicity of curve changes (e.g., increasing, decreasing, non-increasing, or non-decreasing), the periodicity of curve changes, the convexity of the curve, and the concaveness of the curve.

[0026] In some embodiments, intermediate variables can be determined based on prior information about the physical parameters, such that the prior information can be mathematically represented as constraint function 103b through the intermediate variables. Based on the prior information about the physical parameters, intermediate variables can be predefined to model intermediate functions of the input physical information, and the transformation sub-model 104 can be a function constructed based on the intermediate function and the objective function, such that they collectively model the objective function. As an example, when the prior information is a monotonically increasing curve of the sequence of physical parameters, the derivatives of the physical parameters can be set as intermediate variables, and a function that maps the derivatives of the physical parameters to positive values ​​(such as, but not limited to, ReLU) can be applied as constraint function 103b as part of the intermediate sub-model 103. Therefore, the transformation sub-model 104 can be set as an integral function (or based on an integral function).

[0027] In some embodiments of this disclosure, for each prediction of physical parameters, the intermediate variables of the physical parameters are first predicted without constraints, and then directly processed by constraint function 103b to satisfy prior information. Afterwards, the predicted intermediate variables can be subjected to the inverse operation relative to the operation obtained from the physical parameters, as a transformation sub-model 104. As a result, it is possible to ensure, for example, that the predicted physical parameters of output 102 are consistent with the prior information. The resulting physical parameter prediction model 100 can achieve accurate prediction performance for physical parameters in an end-to-end manner (i.e., without post-processing steps), while effectively suppressing unrealistic data (that contradicts prior information) and preventing overfitting of the training data.

[0028] In some embodiments, prior information can govern the entire sequence, a portion of the sequence, or scattered locations / points within the sequence, or samples in a multi-scale (scalar) prediction problem, for a sequence of physical parameters.

[0029] In some embodiments, the unconstrained intermediate sub-model 103a can be generated in various ways, including but not limited to linear models, curve models (such as multinomial models), learning models (such as machine learning models or deep learning models), etc. In some embodiments, the unconstrained intermediate sub-model 103a can be configured as a learning model such as a decision tree, support vector machine, Bayesian prediction model, CNN, or MLP to model the implicit and complex mapping function between physical information (e.g., input 101) and intermediate variables.

[0030] Generally, this disclosure involves two phases: a prediction phase and a training phase. The training phase can be performed to train a physical parameter prediction model 100, and the prediction phase can be performed to apply the trained physical parameter prediction model 100 to make predictions of physical parameters based on the input physical information. Each of the prediction and training phases can be performed online (e.g., in real time) or offline (e.g., in advance). In some embodiments, the training phase can be performed offline, and the prediction phase can be performed online.

[0031] Figure 2 A flowchart illustrating an exemplary method for predicting physical parameters based on input physical information according to an embodiment of the present disclosure is shown.

[0032] As in Figure 2 As shown, the method begins with step 200: receiving physical information. The physical information may be obtained in a specific technical field. In step 201, the method may include a processor predicting intermediate variables using an intermediate sub-model based on the input physical information, the intermediate sub-model incorporating prior information about the physical parameters and constraints regarding the intermediate variables. In step 202, the method may further include a processor converting the intermediate variables predicted by the intermediate sub-model into physical parameters using a transformation sub-model.

[0033] For example, the technical field can be the medical field, and the physical information can be medical information, such as clinical information from medical history, images (or blocks), and / or feature vectors extracted from them (explicitly defined or implicit feature information). Therefore, physical parameters can be medical parameters. For example, medical parameters can include medical indicators, physiological state parameters, disease types, etc. Medical images can be acquired via any of the following image modalities: functional MRI (e.g., fMRI, DCE-MRI, and diffusion MRI), cone-beam computed tomography (CBCT), spiral CT, positron emission tomography (PET), single-photon emission computed tomography (SPECT), X-ray, optical tomography, fluorescence imaging, ultrasound imaging, radiotherapy field imaging, etc., or combinations thereof.

[0034] Already referred to Figure 1The details of the intermediate sub-model, constraint functions, and transformation sub-model are described in detail, and therefore will not be repeated here.

[0035] In some embodiments, the constrained intermediate sub-model 103 can be a learning model (e.g., a machine learning model or a deep learning model), and the transformation sub-model 104 can be a preset function. In some embodiments, the constrained intermediate sub-model 103 and the transformation sub-model 104 can be jointly trained using a training set of physical information labeled with physical parameters. In this way, the lack of ground truth labels for intermediate variables can be overcome; instead, the physical parameter prediction model 100 can be trained holistically using rich ground truth labels for physical parameters. The training of the physical parameter prediction model 100 effectively trains the constrained intermediate sub-model 103 as a learning model.

[0036] For physical parameter prediction models, which have predefined configurations (i.e., predefined intermediate variables, transformation sub-models, and constraint functions), and predetermined configurations of unconstrained intermediate sub-models (such as CNNs), they can perform operations such as... Figure 3 The training process is shown in the figure.

[0037] The training process can begin with step 301, where the training data includes physical information and ground truth labels corresponding to the physical parameters. The training data is then input into a physical parameter prediction model (with features such as...). Figure 1 (as shown in the predefined framework). In some embodiments, model parameters (such as weights) of unconstrained intermediate sub-models in the physical parameter prediction model can be initialized. For example, model parameters can be initialized to all 0s or 1s, or to a set of values ​​used in previously trained intermediate sub-models (for the same or different technical fields).

[0038] In step 302, from the physical information in the training data, the constrained intermediate sub-model can use the current model parameters to predict intermediate variables. In step 303, the predicted intermediate variables are then converted into predictions of physical parameters by a transformation sub-model. In step 304, a loss function can be calculated by comparing the predictions of physical parameters with their ground truth labels. In step 305, the calculated loss is compared with a stopping criterion (e.g., a nominal threshold). If the calculated loss is lower than the stopping criterion (step 305: Yes), the current model parameters are sufficiently optimized and no further iterations are needed. Therefore, the method proceeds to step 306, outputting a physical parameter prediction model with the current model parameters of the unconstrained intermediate sub-model. Otherwise (step 305: No), further optimization is required. In step 307, the model parameters of the unconstrained intermediate sub-model can be optimized based on the calculated loss function. Then, the method iterates steps 302-305 based on the updated unconstrained intermediate sub-model with the current model parameters until the loss is less than the stopping criterion.

[0039] In some embodiments, the optimization of model parameters can be performed using various algorithms, such as, but not limited to, stochastic gradient descent, Newton's method, conjugate gradient method, quasi-Newton method, and Levenberg-Marquardt algorithm.

[0040] Because prior information is explicitly enforced by imposing constraints on intermediate variables, the physical parameter prediction model requires no additional loss term relative to the prior information during training. Furthermore, the training process ensures that the prediction results match the prior information, and its workload is comparable to that of other physical parameter prediction models that attempt to effectively avoid overfitting without enforcing prior information.

[0041] In some embodiments, the sequence of physical parameters may include vascular parameters at a series of locations in a vascular structure, such as a vascular tree or vascular path.

[0042] In the following text, fractional flow reserve (FFR) is described as an example of a physical parameter. Two examples of prior information—the monotonicity of the curvilinear changes of a sequence of physical parameters and the boundary range of a single physical parameter—are used to illustrate how various prior information can be explicitly forced into a physical parameter prediction model. However, these exemplary methods for predicting FFR can be applied or adapted to predicting other medical or physiological parameters in the medical field, or physical parameters in other technical fields. Furthermore, these methods can also be applied to other types of prior information.

[0043] Fractional flow reserve (FFR) is considered a reliable indicator for assessing cardiac ischemia, and learning models have been used to predict FFR values ​​in the coronary tree. FFR is defined as the ratio between the pressure following stenosis (or pressure at any location within the coronary tree) and the pressure at the portal (the entrance to the coronary tree). According to the principles of physics, in a sequence of FFR values ​​within the coronary tree, the downstream FFR value should not be higher than the upstream FFR value.

[0044] In some embodiments, instead of directly predicting FFR values, the methods and apparatus of this disclosure can be used to model the decrease in FFR at a current point relative to its adjacent upstream point. The decrease in FFR value can be defined as the derivative of the FFR along the sequence. Based on the monotonicity of the curvilinear change of the FFR value sequence along the vascular structure, intermediate variables can be defined based on the derivatives of the FFR value sequence (such as the derivative of an upstream FFR value relative to its adjacent downstream FFR value), and correspondingly, the constraint function can be defined as mapping to a non-negative range. A transformation sub-model can be defined based on an integral function to derive the sequence of FFR values ​​from the non-negative derivatives of the FFR value sequence. Similarly, for other physical parameters with prior information including the monotonicity of the curvilinear change of the physical parameters, intermediate variables can be defined based on the derivatives of the physical parameters.

[0045] like Figure 4 As shown, the FFR prediction model can receive image patches or feature vectors along the coronary artery tree or path as input 401x(t). The FFR prediction model may include a constrained derivative sub-model 403 and a transformation sub-model 404.

[0046] The constrained derivative sub-model 403 aims to model the derivative of the FFR value sequence. Based on the derivative of the predicted FFR value sequence, the transformation sub-model 404 can map the constrained derivative to the FFR values ​​in the target domain.

[0047] like Figure 4As shown, the constrained derivative submodel 403 may include an unconstrained derivative unit 403a and a constraint function 403b, and may be based on a learning model (particularly for the unconstrained derivative unit 403a). Specifically, the unconstrained derivative unit 403a may be constructed as a convolutional neural network (CNN), a multilayer perceptron (MLP), a fully convolutional neural network (FCN), etc. The constraint function 403b may be implemented by an activation function at the end of the learning model for the unconstrained derivative unit 403a. In some embodiments, a ReLU activation function may be employed to force the decrease in upstream FFR relative to downstream FFR to be non-negative, incorporating non-increasing FFR prior information into the FFR prediction model. It is expected that ReLU is only an example of an activation function, and other examples of activation functions, such as sigmoid, which can map the derivative to a non-negative range, may also be appropriately employed.

[0048] The final predicted FFR value y(t) can be calculated recursively using the transformation submodel 404 from the output of the activation function, i.e., the non-negative derivative of the sequence of FFR values, which is essentially a non-negative descent of the FFR value sequence along the vascular tree / path. Then, the final predicted FFR value y(t) can be provided as output 402, as... Figure 4 As shown in the image.

[0049] As a result, no additional loss term is needed to penalize non-monotonic predictions, as it can be explicitly enforced in the FFR prediction model.

[0050] In some embodiments, the FFR prediction model is designed to model the objective function, i.e., the true deep function F(x(t)). For example, the FFR prediction model can be represented as the function Ф(x(t)). Ф(x(t)) is constructed using an intermediate function f(x(t)) (corresponding to the trained unconstrained derivative unit 403a) to model the objective function F(x(t)). For example, the intermediate function f(x(t)) can be the derivative function of F(x(t)), where t represents the position or index in the sequence, which can move downstream as t increases. As an example, the intermediate function f(x(t)) can be defined as the following formula (1):

[0051]

[0052] Or some other conversion function.

[0053] Based on the intermediate function f(x(t)), a construction function Ф(x(t)) (corresponding to the trained FFR prediction model) can be established, which attempts to model and approximate the true deep function F(x(t)).

[0054] like Figure 4As shown, the input x(t)401 can first be fed into the constrained derivative submodel 403. Parameterized by θ. Constrained derivative submodel 403 It can model the intermediate function f(x(t)) rather than the deep function F(x(t)). It can be easily used to enforce prior information, that is, by The predicted constrained intermediate values ​​can be further fed into the transformation sub-model 404 to produce the final prediction of FFR, y(t). Specifically, the input x(t) 401 can first be fed into the unconstrained derivative unit 403a to predict the 'raw' (without validation from non-decreasing monotonic prior information) FFR derivatives within the vascular tree (upstream position relative to adjacent downstream position). The predicted 'raw' FFR derivatives are then fed into the constraint function 403b, for example, a ReLU activation function connected to the end of the unconstrained derivative unit 403a. The constraint function 403b maps the 'raw' FFR derivatives to constrained (non-negative) FFR derivatives to conform to the non-decreasing monotonic prior information from downstream to upstream. The non-negative FFR derivative can be output by the constraint function 403b and fed into the transformation sub-model 404 to generate and output the final prediction result y(t)402 of FFR, which is forced to conform to the prior information of the non-decreasing monotonicity of FFR along the vascular tree from downstream to upstream through the constraint function 403b in the constrained derivative sub-model 403.

[0055] The loss function L can be computed by comparing the generated prediction y(t) with the ground truth value of the FFR. For the training set D, the parameter θ can be optimized by minimizing the loss function L. Optimization can be performed using methods such as stochastic gradient descent.

[0056] Without limiting the scope of this disclosure, a type of prior information about FFR, namely non-decreasing monotonicity, can be used as an example of the entire description. For example, the function Ф(x(t)) could be a monotonic function using the derivative as an intermediate variable along with a non-negativity constraint function 403b, which maps the input x(t)401 to the output y(t)402 such that for any t2>t1, y(t1)>y(t2). For different prediction problems, the input x(t) could be an image or a feature vector. Constrained derivative submodel 403 The derivative function defined by formula (1) can be used to replace the deep function F(x(t)). This is achieved by forcing a constrained derivative sub-model. To ensure non-negativity (i.e., to guarantee that the predicted FFR value is not decreasing from downstream to upstream), Ф(x(t)) can be easily constrained to monotonicity. In some embodiments, if prior information requires the predicted value to be non-increasing, the constrained derivative submodel... It can be forced to be non-positive; if the prior information only requires the prediction to increase, the constrained derivative sub-model... It can be forced to be positive; if the prior information only requires a decrease in the predicted value, the constrained derivative submodel... It can be forced to be negative. The predicted constrained derivative can be fed into the transformation sub-model 404 to obtain the final prediction result y(t), for example, according to the following formula (2):

[0057]

[0058] If the prediction result y(t0) at position t0 is given (either predefined or determined by a machine learning model), y(t0) = y0, then the prediction result y(t) can be calculated using the following formula (3):

[0059]

[0060] Finally, the value of the loss function L can be calculated by comparing the generated prediction y(t) with the ground truth FFR value. In some embodiments, the loss function L can be the difference between the generated prediction y(t) and the ground truth FFR value (e.g., L⁻¹, L⁻², etc.).

[0061] In some embodiments, for predicting FFR, the input x(t) can be features of an image, image patch, mask, or point along the coronary artery tree. In some embodiments, the input information can be encoded by various learning models such as CNN, FCN, MLP, or other methods applied by the unconstrained derivative unit 403a. In some embodiments, the intermediate variable can be defined as the derivative function of FFR, or simply the decrease in FFR relative to a previous upstream position along the vessel tree.

[0062] Figure 5 A schematic diagram of another example of an FFR prediction model according to embodiments of the present disclosure is shown. In some embodiments, the physical parameter to be predicted by the FFR prediction model is a single physical parameter, i.e., a single FFR at an independent location along the vascular tree, and prior information about the boundary range of the physical parameter is taken into account. In particular, the boundary range of a single FFR has a lower limit of 0 and an upper limit of 1.

[0063] like Figure 5 As shown, the FFR prediction model can contain two parallel modeling branches: a left branch defining the lower bound of the boundary range, and a right branch defining the upper bound of the boundary range. For the left branch, a first intermediate variable can be defined based on subtracting the lower bound of the boundary range from the FFR; for the right branch, a second intermediate variable can be defined by subtracting the FFR from the upper bound of the boundary range.

[0064] In some embodiments, the input x(t) 501, which may be an image patch, feature vector, etc., can be input into a first constrained subtraction submodel 503a and a second constrained subtraction submodel 503b. In some embodiments, the first constrained subtraction submodel 503a may include a first unconstrained subtraction unit 503a1 and a ReLU 503a2 as a corresponding constraint function (also used as an activation function at the end of the learning model). The first unconstrained subtraction unit 503a1 may be constructed based on any of CNN, MLP, etc., and may be configured to model and determine the difference between the FFR value and the lower bound (e.g., 0). The difference may then be mapped to a non-negative range by ReLU 503a2 to enforce prior information associated with the lower bound. ReLU 503a2 may output the non-negative difference between the FFR value and the lower bound and feed it into a first transformation submodel 504a. The first transformation sub-model 504a can be constructed based on subtraction, for example, by performing the inverse operation of the operation by the first unconstrained subtraction unit 503a1 to obtain the FFR value as the first output y1(t) 502a.

[0065] Similarly, in the right branch targeting the upper bound, the second constrained subtraction submodel 503b may include a second unconstrained subtraction unit 503b1 and a ReLU 503b2 as the corresponding constraint function (also used as the activation function at the end of the learned model). The second unconstrained subtraction unit 503b1 may be constructed based on any of CNN, MLP, etc., and may be configured to model and determine the difference between the upper bound (e.g., 1) and the FFR value. The difference may then be mapped to a non-negative range by ReLU 503b2 to enforce the prior information associated with the upper bound. ReLU 503b2 may output the non-negative difference between the upper bound and the FFR value and feed it to the second transformation submodel 504b. Similar to the first transformation submodel 504a, the second transformation submodel 504b may be constructed based on subtraction, for example, by performing the inverse operation of the operation by the second unconstrained subtraction unit 503a1 to obtain the FFR value as the second output y2(t) 502b.

[0066] The final output y(t) 502c, which is the final predicted FFR value, can be obtained using both the first output y1(t) 502a and the second output y2(t) 502b. As an example, the final output y(t) 502c can be obtained by averaging the first output y1(t) 502a and the second output y2(t) 502b using the averaging unit 502d. In some embodiments, other operations, such as minimization, can be used to simultaneously consider the first output y1(t) 502a and the second output y2(t) 502b to obtain the final predicted FFR value.

[0067] Although Figure 5 A parallel framework is shown, comprising one branch about the lower bound and another about the upper bound, but this is merely an example. In some embodiments, either of the two branches can operate independently. Furthermore, although... Figure 4 and Figure 5 The transformation sub-models are shown to exist outside their respective constrained derivative sub-models, but these sub-models can be combined into a single model. Furthermore, in some embodiments, prior information may include non-decreasing monotonicity and boundary ranges, both of which can be applied as constraints. For example, as... Figure 4 The constrained derivative submodel 403 shown in the figure, and Figure 5 The first and second constrained subtraction submodels 503a and 503b shown can be combined into an FFR prediction model.

[0068] In some embodiments, prior information about the convexity of the curve of the sequence of physical parameters can be employed and enforced in the learning model. Therefore, intermediate variables can be defined based on second derivatives, activation functions (such as, but not limited to, ReLU) can be employed at the end of the learning model, and the transformation function can be based on indefinite integrals to recover the physical parameters to be predicted from the output of the intermediate sub-model (i.e., the second derivative of the predicted sequence of physical parameters).

[0069] In the above embodiments, the coronary artery is used as an example of a blood vessel; however, it is expected that the blood vessel can be any one of the coronary artery, carotid artery, abdominal aorta, cerebral blood vessel, ophthalmic blood vessel, and femoral artery.

[0070] Figure 6 A schematic block diagram of a physical parameter prediction device 600 for predicting physical parameters based on input physical information, according to an embodiment of the present disclosure, is shown. Figure 6 As shown, the physical parameter prediction device 600 may include a communication interface 603, a processor 602, a memory 601', a memory 601, and a bus 604, and may also include a display. The communication interface 603, the processor 602, the memory 601', and the memory 601 may be connected to the bus 604 and may communicate with each other via the bus 604.

[0071] Memory 601 may be configured to load or store intermediate sub-models according to any one or more embodiments of the present disclosure, including, for example, constrained intermediate sub-models and transformation sub-models. Processor 602 may be configured to use the intermediate sub-models to predict intermediate variables based on input physical information; and to use the transformation sub-models to convert the intermediate variables predicted by the intermediate sub-models into physical parameters.

[0072] In some embodiments, processor 602 may be a processing device including one or more general-purpose processing devices, such as a microprocessor, a central processing unit (CPU), a graphics processing unit (GPU), etc. More specifically, the processor may be a Complex Instruction Set Computing (CISC) microprocessor, a Reduced Instruction Set Computing (RISC) microprocessor, a Very Long Instruction Word (VLIW) microprocessor, a processor running other instruction sets, or a processor running a combination of instruction sets. The processor may also be one or more special-purpose processing devices, such as application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), digital signal processors (DSPs), system-on-a-chip (SoCs), etc.

[0073] Memory 601 may be a non-transitory computer-readable medium, such as read-only memory (ROM), random access memory (RAM), phase-change random access memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), electrically erasable programmable read-only memory (EEPROM), other types of random access memory (RAM), flash memory or other forms of flash memory, cache, registers, static memory, optical disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical memory, magnetic tape cassette or other magnetic storage devices, or any other possible non-transitory medium for storing accessible information or instructions such as those of a computer device. When the processor 602 executes the instructions stored on memory 601, a method for predicting physical parameters based on input physical information according to any embodiment of the present disclosure may be performed. In some embodiments, the physical parameter prediction device 600 may also perform model training functions, and therefore, memory 601 may be configured as a training dataset labeled with physical information of physical parameters, and processor 602 may be configured to jointly train intermediate sub-models and transformation sub-models based on the loaded training dataset.

[0074] In some embodiments, the physical parameter prediction apparatus 600 may further include a memory 601', configured to load intermediate sub-models according to any one or more embodiments of the present disclosure. A processor 602 may be communicatively attached to the memory 601' and configured to execute executable instructions stored thereon to perform a method for predicting physical parameters based on input physical information according to any embodiment of the present disclosure.

[0075] In some embodiments, memory 601' may be a non-transitory computer-readable medium, such as read-only memory (ROM), random access memory (RAM), phase-change random access memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), electrically erasable programmable read-only memory (EEPROM), other types of random access memory (RAM), flash disks or other forms of flash memory, cache, registers, static memory, or any other possible medium, for storing information or instructions that can be accessed and executed by a computer device, etc.

[0076] In some embodiments, the physical parameter prediction device 600 may further include a communication interface 603. In some embodiments, the communication interface 603 may include any of the following: a network adapter, a cable connector, a serial connector, a USB connector, a parallel connector, a high-speed data transmission adapter (such as fiber optic, USB 3.0, Thunderbolt interface, etc.), a wireless network adapter (such as a WiFi adapter), a telecommunications (3G, 4G / LTE, 5G, etc.) adapter, etc.

[0077] Figure 7 A schematic block diagram of a system for predicting physical parameters based on input physical information according to an embodiment of the present disclosure is shown. As shown, the system may include a physical parameter prediction device 600, a model training device 700, and an image acquisition device 701. Details of the physical parameter prediction device 600 have already been mentioned above, and therefore will not be repeated here.

[0078] Specifically, the image acquisition device 701 may include any one of conventional CT, conventional MRI, functional magnetic resonance imaging (such as fMRI, DCE-MRI and diffusion MRI), cone-beam computed tomography (CBCT), positron emission tomography (PET), single-photon emission computed tomography (SPECT), X-ray imaging, optical tomography, fluorescence imaging, ultrasound imaging, radiotherapy field imaging, etc.

[0079] In some embodiments, the model training device 700 may be configured to train a physics parameter prediction model (e.g., an unconstrained intermediate sub-model) and send the trained physics parameter prediction model to the physics parameter prediction device 600 so as to predict physics parameters based on input physics information according to any embodiment of the present disclosure by using the trained physics parameter prediction model. In some embodiments, the model training device 700 and the physics parameter prediction device 600 may be implemented by a single computer or processor.

[0080] In some embodiments, the physical parameter prediction device 600 may be a dedicated computer or a general-purpose computer. For example, the physical parameter prediction device 600 may be a computer customized for hospitals to perform image acquisition and image processing tasks, or it may be a server in the cloud.

[0081] The physical parameter prediction device 600 can be connected to the model training device 700, the image acquisition device 701, and other components via a communication interface 603. In some embodiments, the communication interface 603 can be configured to receive a trained physical parameter prediction model from the model training device 700, and can also be configured to receive medical images, such as a set of images of blood vessels, from the image acquisition device 701.

[0082] In some embodiments, memory 601 may store trained models, prediction results of physical parameters, or intermediate information generated during the training or prediction phase, such as feature information generated while executing a computer program. In some embodiments, memory 601' may store computer-executable instructions, such as one or more image processing (e.g., physical parameter prediction) programs. In some embodiments, various units, functions, sub-models, and models may be implemented as applications stored in memory 601, and these applications can be loaded into memory 601' and then executed by processor 602 to perform corresponding processing.

[0083] In some embodiments, the model training apparatus 700 may be implemented using hardware specifically programmed by software that performs training processing. For example, the model training apparatus 700 may include a processor and a non-transitory computer-readable medium similar to the physical parameter prediction apparatus 600. The processor performs training by executing executable instructions for the training process stored in the computer-readable medium. The model training apparatus 700 may also include input and output interfaces for communicating with a training database, a network, and / or a user interface. The user interface can be used to select a training dataset, adjust one or more parameters during training processing, select or modify the framework of the learned model, etc.

[0084] Another aspect of this disclosure aims to provide a non-transitory computer-readable medium storing instructions that, when executed, cause one or more processors to perform the methods described above. The computer-readable medium may include volatile or non-volatile, magnetic, semiconductor-based, magnetic tape-based, optical, removable, non-removable, or other types of computer-readable media or computer-readable storage devices. For example, the computer-readable medium may be a storage device or storage module in which computer instructions are stored, as disclosed. In some embodiments, the computer-readable medium may be a magnetic disk or flash drive on which computer instructions are stored.

[0085] Various modifications and alterations can be made to the methods, apparatus, and systems disclosed herein. In view of the description and practice of the disclosed systems and related methods, other embodiments can be derived by those skilled in the art. Each claim of this disclosure is to be understood as an independent embodiment, and any combination thereof is also used as an embodiment of this disclosure, and such embodiments are considered to be included in this disclosure.

[0086] The descriptions and examples are to be considered exemplary only, and the true scope is indicated by the appended claims and their equivalents.

Claims

1. A method for predicting physical parameters based on input physical information, comprising: The processor predicts intermediate variables based on the input physical information using an intermediate sub-model. The intermediate sub-model includes an unconstrained intermediate sub-model and a constraint function. The intermediate variables are initially predicted without constraints, and then directly processed by the constraint function to satisfy prior information. Here, the constraints imposed on the intermediate variables explicitly enforce the prior information. During the training process, the physical parameter prediction model does not require an additional loss term relative to the prior information. Here, the physical information is an image / image patch and / or a sequence of feature vectors extracted from it. The processor uses a transformation sub-model to convert the intermediate variables predicted by the intermediate sub-model into the physical parameters; the transformation sub-model is a preset function. The physical parameter is a single physical parameter or a sequence of physical parameters, the sequence of physical parameters including vascular parameters at a series of locations in the blood vessel, the blood vessel having a vascular tree structure or vascular path; the prior information includes the curve trend of the physical parameter curve, or the boundary range of the physical parameter in the time domain or spatial domain.

2. The method according to claim 1, wherein the intermediate sub-model is based on a learning model, and the intermediate sub-model and the transformation sub-model are jointly trained using a training dataset, wherein the training dataset includes sample physical information labeled with corresponding ground truth physical parameters.

3. The method according to claim 1, wherein the curve trend includes any one of the following: monotonicity of curve change, periodicity of curve change, convexity of the curve, and concaveness of the curve.

4. The method of claim 1, wherein the intermediate sub-model is based on a learning model, and the constraints include an activation function.

5. The method of claim 1, wherein the physical parameter to be predicted comprises a sequence of physical parameters, the prior information of the physical parameters is the monotonicity of the curve variation of the sequence of physical parameters, the intermediate variable is the derivative of the sequence of physical parameters, the constraint condition comprises an activation function, and the transformation function is an integral function.

6. The method of claim 1, wherein the physical parameter to be predicted is a single physical parameter, the prior information of the physical parameter is the boundary range of the physical parameter, the intermediate variable is determined by subtracting the lower limit of the boundary range from the physical parameter or subtracting the physical parameter from the upper limit of the boundary range, the constraint is an activation function, and the transformation function is subtraction.

7. The method of claim 1, wherein the physical parameter to be predicted comprises a sequence of physical parameters, the prior information of the physical parameters is the convexity of a curve of the sequence of physical parameters, the intermediate variable is the second derivative of the sequence of physical parameters, the constraint is an activation function, and the transformation function is an indefinite integral.

8. An apparatus for predicting physical parameters based on input physical information, comprising: A memory configured to load or store intermediate sub-models and transformation sub-models; as well as The processor is configured as follows: Based on the input physical information, the intermediate sub-model predicts intermediate variables. The intermediate sub-model includes an unconstrained intermediate sub-model and a constraint function. The intermediate variables are initially predicted without constraints, and then directly processed by the constraint function to satisfy prior information. Specifically, by imposing constraints on the intermediate variables, the prior information is explicitly enforced. During training, the physical parameter prediction model requires no additional loss term relative to the prior information. The physical information is an image / image patch and / or a sequence of feature vectors extracted from it; The intermediate variables predicted by the intermediate sub-model are converted into the physical parameters using a transformation sub-model; the transformation sub-model is a preset function. The physical parameter is a single physical parameter or a sequence of physical parameters, the sequence of physical parameters including vascular parameters at a series of locations in the blood vessel, the blood vessel having a vascular tree structure or vascular path; the prior information includes the curve trend of the physical parameter curve, or the boundary range of the physical parameter in the time domain or spatial domain.

9. The apparatus of claim 8, wherein the intermediate sub-model is based on a learning model, and the intermediate sub-model and the transformation sub-model are jointly trained using a training dataset, the training dataset comprising sample physical information labeled with corresponding ground truth physical parameters.

10. The apparatus of claim 8, wherein the curve trend includes any one of the following: monotonicity of curve change, periodicity of curve change, convexity of the curve, and concaveness of the curve.

11. The apparatus of claim 8, wherein the intermediate sub-model is based on a learning model, and the constraints include an activation function.

12. The apparatus of claim 8, wherein the physical parameter to be predicted comprises a sequence of physical parameters, the prior information of the physical parameters is the monotonicity of the curve variation of the sequence of physical parameters, the intermediate variable is the derivative of the sequence of physical parameters, the constraint includes an activation function, and the transformation function is an integral function.

13. A non-transitory computer-readable storage medium storing computer-executable instructions thereon, wherein when the computer-executable instructions are executed by a processor, a method for predicting physical parameters based on input physical information is performed, the method comprising: Based on the input physical information, intermediate variables are predicted using an intermediate sub-model. This intermediate sub-model consists of an unconstrained intermediate sub-model and a constraint function. The intermediate variables are initially predicted without constraints, and then directly processed by the constraint function to satisfy prior information. By imposing constraints on the intermediate variables, the prior information is explicitly enforced. During training, the physical parameter prediction model requires no additional loss term relative to the prior information. The physical information is an image / image patch and / or a sequence of feature vectors extracted from it; The intermediate variables predicted by the intermediate sub-model are converted into the physical parameters using a transformation sub-model; the transformation sub-model is a preset function. The physical parameter is a single physical parameter or a sequence of physical parameters, the sequence of physical parameters including vascular parameters at a series of locations in the blood vessel, the blood vessel having a vascular tree structure or vascular path; the prior information includes the curve trend of the physical parameter curve, or the boundary range of the physical parameter in the time domain or spatial domain.

Images