Method for generating a spatio-temporal model of a part of a patient's heart

Abstract

The invention relates to a method for generating a spatio-temporal model (4DLVM) of a portion of a patient's heart, comprising the following steps: (E1) Receiving a plurality of ultrasound images (ECi) of a patient's heart; (E2) Generating a spatial model (3DLVM) of at least one given portion of the heart from the plurality of received ultrasound images; Receiving a photoplethysmographic signal (PPG) of said patient; Determining a heart volume change function (LVVC), during all or part of a cardiac cycle of the patient's heart, from the photoplethysmographic signal; Generating a spatio-temporal model (3DLVM) of said portion of the heart using the spatial model of said portion of the heart and said heart volume change function. Figure to be published with the abstract: Fig. 1

Description

Title of the invention: Method for generating a spatio-temporal model of a part of a patient's heart

[0001] The invention relates to the field of cardiology and medical imaging. More specifically, the invention relates to a method for generating a spatio-temporal model of a part of a patient's heart.

[0002] In the context of preparing a medical intervention targeting a patient's heart, it is common to carry out a simulation of the intervention beforehand, in particular for the purpose of reducing the risk of the intervention. This type of simulation requires the modeling of the patient's heart, both spatially and temporally, in order to obtain a model of the heart representative of the evolution of the shape of the heart during the patient's cardiac cycle, in particular during the systole and diastole phases.

[0003] Conventionally, a spatio-temporal model of the patient's heart, or of a part of the patient's heart, also called a 4D model, is generated from 3D images of the heart obtained by a medical imaging method such as angiography, fluoroscopy or magnetic resonance imaging.

[0004] These 3D images are composed of stacks of 2D images of the heart, each representing a section or slice of the heart through a given plane. Each element of the 3D image thus represents a unit of volume of the patient's heart. When a sequence of 3D images is acquired, for example during a period corresponding to at least one cardiac cycle of the patient, both spatial and temporal information is then available regarding the shape of the heart, so that it is possible to derive a 4D model therefrom.

[0005] This solution, however, has various drawbacks.

[0006] On the one hand, medical imaging equipment such as angiography, fluoroscopy or magnetic resonance imaging is particularly expensive and complex to handle. In addition, this equipment is incompatible with the presence of certain materials in the patient's body, in particular metals, or with certain pathologies likely to prevent the acquisition of 3D images. Finally, these medical imaging methods may be harmful or contraindicated for the patient, insofar as they require the emission of X-rays and / or the injection of a contrast agent. These drawbacks constitute obstacles to the systematic implementation of a simulation of a cardiac intervention based on a 4D model, regardless of the patient.

[0007] On the other hand, each 3D image of a sequence of 3D images is capable of form a 3D model of the heart, corresponding to the shape of the heart at a given moment in the cardiac cycle. The different 3D models thus have different volumes, or even different orientations. It is therefore necessary to carry out complex geometric or morphological transformations on each of these 3D models and / or to recalibrate them in a common reference frame in order to generate a 4D model.

[0008] Finally, given the drawbacks mentioned above, this solution does not make it possible to obtain a 4D model of the heart in real time, or at least in a context of monitoring cardiac mechanical activity. Indeed, this type of monitoring requires the acquisition of cardiac data for long continuous periods, which is incompatible with acquisition methods such as angiography, fluoroscopy or magnetic resonance imaging.

[0009] There is thus a need for a method for generating a spatio-temporal model of a part of a patient's heart, which is inexpensive, reliable and which can be implemented easily, quickly or in real time, and regardless of the patient concerned.

[0010] The present invention is placed in this context and aims to meet this need.

[0011] For these purposes, the invention relates to a method for generating a spatio- temporal of a portion of a patient's heart, comprising the steps of: a. Receiving a plurality of ultrasound images of a patient's heart; b. Generating a spatial model of at least a given portion of the heart from the received plurality of ultrasound images; c. Receiving a photoplethysmographic signal from said patient; d. Determination of a heart volume change function, during all or part of a cardiac cycle of the patient's heart, from the photoplethysmographic signal; e. Generating a spatio-temporal model of said portion of the heart using the spatial model of said portion of the heart and said heart volume change function.

[0012] The invention thus proposes, on the one hand, to spatially model all or part of the patient's heart by means of images acquired by an echocardiography device. This medical imaging modality uses ultrasound waves to obtain 2D or 3D images of the patient's heart. It thus has various advantages, such as in particular the fact of being risk-free for the patient's health, of maintaining an acceptable imaging quality and finally of being able to be implemented by means of inexpensive and easy-to-use equipment. It is in particular possible to connect an acquisition device to a portable device, such as a portable ultrasound scanner, or even a smartphone, so that the patient can perform his own echocardiography. It is thus possible to generate a spatial, or 3D, model of the heart or of a part of the heart, such as a right or left ventricle, a right or left atrium, a heart valve, or a blood vessel, such as an aorta.

[0013] The invention also proposes to model a function of change in volume of the patient's heart from a photoplethysmographic signal. Photoplethysmography consists of measuring, by means of an optical sensor or a photodetector, the variations in the light emitted by a light source towards a blood vessel and absorbed or reflected by this blood vessel. A photoplethysmographic signal resulting from this measurement thus represents the evolution or variations in the blood volume in a blood vessel. It has thus been demonstrated that this photoplethysmographic signal can be correlated with the variations in volume of the heart during the cardiac cycle and in particular during the systole and diastole phases. It is therefore possible to reconstruct, from this signal, a function of change in volume of the patient's heart.Since this modality only requires an optical sensor and a light source, it can be easily integrated into an acquisition device that can be worn by the patient, so that the volume change function can be generated in real time and in a simple, fast and inexpensive way.

[0014] According to the invention, it is therefore possible to combine the spatial information resulting from the 3D model of the heart, obtained through the ultrasound images, with the temporal information resulting from the volume change function of the patient's heart, obtained through the photoplethysmographic signal, to generate a spatio-temporal, or 4D, model of the patient's heart. Given the advantages mentioned above, this 4D spatio-temporal model can be generated in real time or in a short time, to meet a need for monitoring the patient's cardiac mechanical activity, and regardless of the type of patient. In particular, it will be possible to design the spatial model to be generated beforehand, and the spatio-temporal model to be regenerated in real time or with each new acquisition of a photoplethysmographic signal.The spatial model can thus be generated once by a prior ultrasound, or according to an acquisition frequency significantly lower than that of the photoplethysmographic signal.

[0015] The method according to the invention is advantageously implemented by a computer system. In the context of the present invention, a "computer system" may refer to, but is not limited to, a desktop computer, a laptop computer, a digital tablet, a smartphone, a test equipment, a network device, a controller, a digital signal processor, a computing engine in an appliance, a consumer electronics device, a portable computing device and / or another electronic device, and / or any combination thereof suitable for implementing the method according to the invention. Furthermore, the steps of the method according to the invention may all be executed on a single device. Alternatively, the steps of the method according to the invention can be executed on several devices, connected by wires or by a wireless connection.

[0016] In the context of the present invention, the term "ultrasound images of a patient's heart", or "echocardiogram", is understood to mean a two-dimensional, 2D, or three-dimensional, 3D, image of all or part of the patient's heart, acquired by means of a probe comprising an ultrasonic wave transmitter and one or more sensors, in particular arranged in a linear or matrix manner. These may be B-type or Doppler-type ultrasound images. The probe may in particular be connected, wired or wirelessly, to a portable unit, arranged to convert the ultrasound signals acquired by the sensors of the probe into ultrasound images.

[0017] In the context of the present invention, the term "photoplethysmo-graphic signal", or PPG signal, is understood to mean an alternating signal representing variations in the blood volume in a blood vessel, obtained by means of a photoplethysmographic device, comprising a light source, visible or infrared, and an optical sensor or a photodetector. It may in particular be an alternating component of a signal acquired by a photoplethysmographic device. This device may be integrated into equipment intended to be worn by the patient, such as a smartphone, a smart watch, a bracelet, a ring, a headset or an earpiece.

[0018] In the context of the present invention, the term "heart volume change function" means a mathematical function representing the change in the volume of all or part of the heart, during all or part of a cardiac cycle of the patient's heart, in particular during the systole and diastole phases. It may in particular be composed of several sub-functions, in particular a first sub-function representing the ejection fraction of the left or right ventricle during a systole phase and a second sub-function representing the filling fraction of the left ventricle during a diastole phase.

[0019] In the context of the present invention, the term "spatio-temporal model of a part of the heart" means a model representing a part of the heart during a given period of time, for example during a cardiac cycle. It may be a sequence of spatial models representing said part of the heart at different times of the cardiac cycle or a spatial model each element of which is associated with a displacement function so that it is possible to generate a spatial model representing said part of the heart at any time of the cardiac cycle. The spatial model may comprise a structured set of volume elements, or voxels, or an unstructured point cloud, and may represent an internal wall and / or an external wall and / or an approximation of the shape of said part of the heart. Where appropriate, anatomical regions of said part of the heart may be identified. in the spatio-temporal model, for example by labeling the points or voxels defining these regions or by positioning markers associated with these regions on the spatio-temporal model.

[0020] In one embodiment of the invention, the step of generating said spatial model of said part of the heart comprises a sub-step of transforming a predetermined initial model of a heart to generate a spatial model of said part of the heart corresponding to the plurality of received ultrasound images. This type of modeling, also called model-based segmentation, has the advantage of being robust and automatic.

[0021] Advantageously, said generation step may comprise a sub-step of determining a transformation which, when applied to the predetermined initial model, minimizes an error function between the transformed initial model and at least one of the received ultrasound images. Said transformation may comprise one or more affine transformations, one or more elastic transformations, or a combination of affine and / or elastic transformations.

[0022] This sub-step may be based on 2D or 3D ultrasound images. For example, said sub-step of determining a transformation may be a step of determining a transformation which, when applied to the predetermined initial model, minimizes the distance between the contour of a section of the transformed initial model, by a plane corresponding to one of the received ultrasound images, and the contour of the heart in said received ultrasound image. It may be envisaged as a variant that the transformation is determined by correspondence between the transformed initial model and a 3D ultrasound image.Where appropriate, said transformation may be determined by means of one or more machine learning algorithms, trained to determine from an ultrasound image, a transformation which, when applied to the predetermined initial model, minimizes an error function between the transformed initial model and this received ultrasound image.

[0023] If desired, said step of generating said spatial model of said part of the heart may comprise a sub-step of transforming a predetermined initial model of a heart to generate a spatial model of the patient's heart, corresponding to the plurality of received ultrasound images and a sub-step of segmenting said spatial model to generate said spatial model of said part of the heart. For example, said spatial model of the heart may comprise a plurality of regions of interest each labeled with an identifier of an anatomical region of the heart, said segmentation sub-step comprising selecting one or more regions of interest corresponding to said part of the heart.

[0024] Advantageously, the method according to the invention comprises a prior step of ge generation of a predetermined initial model of a heart, said step comprising the following sub-steps: a. Receiving a plurality of spatial models of the heart of a plurality of patients; b. Alignment of each of the spatial models in a reference frame aligned with a common reference frame; c. Determination of an average model from all the recalibrated spatial models, the predetermined initial model being generated from said average model.

[0025] This type of initial model is also called a statistical shape model or SSM (from the English “Statistical Shape Model”) and makes it possible to obtain a segmentation of a new model in a robust and reliable manner.

[0026] According to an exemplary embodiment of the invention, each of the models of said plurality of spatial models is a training model generated beforehand from ultrasound images of a heart of one of the patients. Each training model is then recalibrated, by an affine transformation, to a reference frame aligned with a common reference frame, for example the reference frame of an atlas of a heart. Each spatial model is thus moved to a coordinate system whose axes are aligned with those of the common reference frame. The coordinates of the elements of the different recalibrated spatial models are then averaged to obtain a set of average elements together defining said average model.

[0027] It may advantageously be provided to segment the average model into regions of interest from landmarks and / or regions of interest of the atlas of the heart corresponding to given anatomical zones of the heart, such as the ventricles, the atria, the aortas and / or the heart valves.

[0028] Advantageously, the common reference system comprises a plurality of predetermined reference points, the prior step of generating the predetermined initial model comprising the following sub-steps: a. Positioning of each of the predetermined markers in each recalibrated spatial model; b. Statistical analysis of the coordinates of the reference points of the set of recalibrated spatial models to generate a statistical representation of the distribution of the coordinates of these reference points among the recalibrated models, the predetermined initial model being generated from said average model and said statistical representation.

[0029] According to these characteristics, the predetermined initial model can have a variable shape, defined by the statistical representation, centered around the average model.

[0030] According to an exemplary embodiment of the invention, the common reference system is that of a atlas of a heart comprising a plurality of anatomical landmarks, for example defining the position of given anatomical structures, and / or geometric landmarks, for example defining borders of the heart, and / or reference landmarks, for example defining vessel junction points. Where appropriate, for each of the recalibrated spatial models, the sub-step of positioning each of the predetermined landmarks comprises the recalibration of said recalibrated spatial model in the common reference frame of the atlas, for example by means of an elastic transformation of said recalibrated spatial model, the positioning of each of said atlas landmarks on the recalibrated spatial model in the common reference frame of the atlas, then the recalibration of said recalibrated spatial model from the common reference frame into its initial reference frame aligned with the common reference frame, for example by means of an inverse elastic transformation.

[0031] For example, the statistical analysis of the coordinates of the reference points of the set of recalibrated spatial models may include a principal component analysis making it possible to obtain, for example, a covariance matrix of said coordinates. Alternatively or cumulatively, the statistical analysis of the coordinates of the reference points of the set of recalibrated spatial models may include the implementation of other data analysis methods, linear or non-linear.

[0032] Preferably, the predetermined initial model can be obtained by a linear combination of the average model and said statistical representation.

[0033] Advantageously, the step of generating said spatial model of said part of the heart comprises, for each image of the plurality of received ultrasound images, a sub-step of transforming the predetermined initial model of a heart to generate a spatial model of said part of the heart corresponding to said image, and a sub-step of averaging the spatial models to generate the spatial model of said part of the heart. In this example, each of the ultrasound images makes it possible to obtain, from the same initial model, a spatial model corresponding to this image and therefore to the shape of the patient's heart at a given time. All of the spatial models are then averaged to obtain a model corresponding overall to all of the ultrasound images.

[0034] In one embodiment of the invention, the method comprises a step of generating, from the spatial model of said part of the heart, a motion map of said spatial model, each element of the map being associated with a zone of the spatial model and indicating the capacity of said zone to expand or contract during a change in volume of the heart. Where appropriate, said spatio-temporal model of said part of the heart is generated by means of the spatial model of said part of the heart, said volume change function of the heart and the motion map of said spatial model. In this embodiment, it is assumed that each region of the heart, or of a wall of the heart, moves, during the cardiac cycle, in a different way from other regions, in particular if it is a healthy region or a region likely to be affected by heart disease or diabetes. Depending on these characteristics, it is possible to generate, from the spatial model and / or the ultrasound images, a motion map, indicating, for each point or voxel or for groups of points or voxels of the spatial model, the capacity of a region of the heart or of a wall of the heart, at a position corresponding to said point or voxel or said group of points or voxels, to move. The combination of the motion map and the volume change function thus makes it possible to model, with great precision, the shape of the patient's heart at a given moment of his cardiac cycle.

[0035] Advantageously, said plurality of ultrasound images comprises at least one sequence of ultrasound images acquired during a complete cardiac cycle of the patient's heart. Where appropriate, the step of generating the motion map of said spatial model comprises the following sub-steps: a. Splitting the spatial model of said portion of the heart into a plurality of surface segments; b. For each surface segment, determination, from said sequence of images, of at least one value and / or function representative of a geometric change of said segment during the complete cardiac cycle; said value and / or function forming an element of the motion map associated with said surface segment of the spatial model.

[0036] For example, said function representative of a geometric change of a segment can be determined from the evolution of the ratio between the surface area of ​​the segment at an instant of the cycle and the surface area of ​​the segment at a predetermined instant of the cycle, such as at the end of the diastole phase.

[0037] It will be possible in particular to consider estimating the duration of the cardiac cycle using the patient's photoplethysmographic signal.

[0038] In one embodiment of the invention, the spatio-temporal model of said part of the heart comprises a morphological function making it possible to generate an instance of a spatial model of said part of the heart at a given instant, said instance being determined by an affine transformation of the spatial model of said part of the heart by a weighting element calculated from a value of said volume change function of the heart at said given instant and the motion map of said spatial model.

[0039] For example, said weighting element may be determined from the ratio between the volume of the spatial model of said part of the heart and a volume estimated from said volume change function at said given instant.

[0040] Advantageously, the generation of said instance may be implemented recursively until the error between the volume of said portion of the heart determined from said value of said heart volume change function and the volume of the instance is less than a predetermined threshold value.

[0041] In one embodiment of the invention, at least a portion of the heart volume change function is identical to the relative value of a portion of the photoplethysmographic signal.

[0042] In another embodiment, parameters and / or coefficients of an equation for determining the volume of all or part of the heart, and in particular of a cavity of the heart, are determined from the photoplethysmographic signal, said heart volume change function being determined from said parameters and / or coefficients. For example, said parameters and / or coefficients may be estimated by one or more machine learning algorithms, trained to estimate values ​​of said parameters and / or coefficients from data of a photoplethysmographic signal.

[0043] According to an exemplary embodiment of the invention, the method comprises a step of estimating the coefficients a1 and a2 of the equation of the biplane surface / length method, from the values ​​of the photoplethysmographic signal, the coefficient a1 being determined by means of a first machine learning algorithm trained to estimate the value of the coefficient a1 from the values ​​of a part of a photoplethysmographic signal corresponding to a systole phase and the coefficient a2 being determined by means of a second machine learning algorithm trained to estimate the value of the coefficient a2 from the values ​​of a part of a photoplethysmographic signal corresponding to a diastole phase.

[0044] The first algorithm may for example be trained using a data set comprising ultrasound images of patients' hearts and, for each image, a part of a photoplethysmographic signal corresponding to a systole phase of the heart represented by this image. The first algorithm may in particular be trained to determine the value of the coefficient al which minimizes a cost function depending on the volume of a cavity of the heart, in particular the left or right ventricle, estimated from the ultrasound image and the volume of this cavity, estimated using the equation of the biplane surface / length method modeling the volume of the heart during the systole phase and taking into account this coefficient al.This cost function could, for example, correspond to the ratio between the difference in the volume of the cavity estimated using the ultrasound image and the volume of the cavity estimated using the photoplethysmographic signal and the difference in the ejected volume estimated using the ultrasound image and the ejected volume estimated using the photoplethysmographic signal.

[0045] The second algorithm could for example be trained using a set of data comprising ultrasound images of patients' hearts and, for each image, a part of a photoplethysmographic signal corresponding to a diastole phase of the heart represented by this image. The second algorithm may in particular be trained to determine the value of the coefficient a2 which minimizes a cost function depending on the volume of a cavity of the heart, in particular the left or right ventricle, estimated from the ultrasound image and the volume of this cavity, estimated using the equation of the biplane surface / length method modeling the volume of the heart during the diastole phase and taking into account this coefficient a2.This cost function could, for example, correspond to the ratio between the difference in the volume of the cavity estimated using the ultrasound image and the volume of the cavity estimated using the photoplethysmographic signal and the difference in the filled volume estimated using the ultrasound image and the filled volume estimated using the photoplethysmographic signal.

[0046] In yet another embodiment of the invention, said heart volume change function may be estimated by one or more machine learning algorithms receiving said photoplethysmographic signal as input, said algorithm(s) being trained to determine a heart volume change function from a signal provided to it as input.

[0047] The step of determining the heart volume change function can thus be implemented by an artificial neural network, such as a dense type neural network comprising several successive layers of neurons and in which each of these layers is totally interconnected with the previous one and with the next one, or as a recurrent type neural network in which feedback loops are provided, or even a recurrent type with short and long term memory, also called LSTM (Long Short-Term Memory). The artificial neural network may for example have been trained from a training data set comprising both photoplethysmographic signals from patients and simultaneously acquired heart volume change functions from these patients.For each photoplethysmographic signal in the training set, the network predicted a heart volume change function, which was compared to the actual function associated with that signal to determine an error. This error was back-propagated to correct the network parameters to minimize this error, such as the synaptic weights and biases of each neuron. Iterating these predictions and back-propagations across the entire training dataset thus allows the neural network to be trained to reliably estimate a heart volume change function from a photoplethysmographic signal provided as input.

[0048] In a preferred embodiment, the step of determining the function of change in volume of the heart comprises a sub-step of segmenting the photoplethysmographic signal into sub-signals each corresponding to a cardiac cycle, that is to say a diastole phase and a systole phase, a sub-step of determining an average signal of all the segmented sub-signals, and a step of estimating the function of change in volume of the heart by means of a recurrent type neural network with short and long term memory, receiving as input said average signal and a volume of the heart at the end of the diastole phase.

[0049] For example, said network may have an encoder-decoder architecture and include an encoding stage comprising two layers of 256 LSTM type neurons and receiving as input the average signal and the volume, a decoding stage comprising two layers of 256 LSTM type neurons and receiving as input the output of the encoding stage, and a dense stage comprising a layer of 128 neurons fully connected to the output of the decoding stage.

[0050] Where appropriate, the method comprises a prior step of training the neural network using a training data set consisting of photoplethysmographic signals from 30 patients acquired during several cardiac cycles and measurements of the evolution of the volume of the patients' hearts during these cardiac cycles, obtained by imaging, the neural network thus being trained to determine a function of change in volume of the heart during a cardiac cycle.

[0051] The invention also relates to a computer program comprising a program code which is designed to implement the method according to the invention.

[0052] The invention also relates to a data medium on which the computer program according to the invention is recorded.

[0053] The invention also relates to a computer system for generating a spatiotemporal model of a part of a patient's heart, the computer system comprising a computing unit capable of receiving a plurality of ultrasound images of a patient's heart and a photoplethysmographic signal of said patient, the computing unit being arranged to implement the method according to the invention.

[0054] The present invention is now described using examples which are purely illustrative and in no way limitative of the scope of the invention, and from the appended drawings, drawings in which the various figures represent:

[0055] [Fig. 1] represents, schematically and partially, a method for generating a spatio-temporal model of a part of a patient's heart according to an embodiment of the invention;

[0056] [Fig.2] represents, schematically and partially, a sequence of echo images graphs of a patient's heart received during a step of the process of [Fig.l];

[0057] [Fig.3] represents, schematically and partially, the generation of a model predetermined initial of a core, implemented during a step of the process of [Fig.l]

[0058] [Fig.4] represents, schematically and partially, the transformation of the model of [Fig.3] for generating a spatial model of a portion of a heart corresponding to the plurality of ultrasound images of [Fig.2], implemented during a step of the method of [Fig.l];

[0059] [Fig.5] represents, schematically and partially, the generation of a motion map of a spatial model of a heart, implemented during a step of the method of [Fig.l]; and

[0060] [Fig.6] represents, schematically and partially, the determination of a volume change function of a heart from a photoplethysmographic signal, implemented during a step of the method of [Fig.l].

[0061] In the following description, elements that are identical, by structure or by function, appearing in different figures retain, unless otherwise specified, the same references.

[0062] [Fig.l] shows a method for generating a spatio-temporal model of a part of a patient's heart according to an embodiment of the invention. The method is implemented by a computer system (not shown).

[0063] In a first step E1 of the method, a plurality of EC ultrasound images of all or part of the patient's heart are received by the computer system.

[0064] These EC ultrasound images may have been acquired beforehand by a cardiologist's echocardiography device or by a portable ultrasound probe connected to a patient's own smartphone.

[0065] [Fig.2] shows a sequence of three-dimensional, 3D, ECi to ECN images of a left ventricle of the patient's heart, acquired during a cardiac cycle of the patient, from the start of a diastole phase to the end of a systole phase, forming said plurality of ultrasound images. It is conceivable that the EC images are two-dimensional images and / or that the acquisition period is shorter or longer than a cardiac cycle and / or that the EC images are disordered without departing from the scope of the present invention. It is conceivable that the EC images represent another part of the patient's heart, such as a right ventricle or a left or right atrium, or even the entirety of the patient's heart rather than just a part.

[0066] In a second step E2 of the method, a spatial model of the left ventricle of the patient's heart 3DLVM is generated from the plurality of received ultrasound images EC.

[0067] In the example described, the 3DLVM spatial model can be generated again with each new acquisition of EC ultrasound images; by the cardiologist or by the patient.

[0068] In order to generate the 3DLVM spatial model, a predetermined initial model SSM of a left ventricle of a heart was generated in a prior step E0.

[0069] [Fig.3] shows an example of the implementation of step E0 making it possible to generate an initial SSM model of the statistical shape model type.

[0070] This step EO comprises a sub-step E01 of receiving a plurality of spatial models LVTj, forming a training set. In the example described, each of the LVT training models was generated beforehand from ultrasound images of a left ventricle of the heart of one of the patients.

[0071] In a sub-step E02, each of the training models LVTj is recalibrated in a reference frame Xj aligned with the same common reference frame XA, namely the reference frame of an atlas A of a predetermined core. Each model LVTj can for example be transformed by one or more affine or linear transformations TA, of the translation, rotation, enlargement, reduction, symmetry type, to transform it into a model LVTixA oriented according to a reference frame Xj whose axes are aligned with those of the common reference frame XA.

[0072] The atlas A comprises a plurality of anatomical LM markers, for example defining the position of given anatomical structures, such as the aortas and / or the heart valves, geometric markers, for example defining borders of the ventricle, and reference markers, for example defining junction points of vessels.

[0073] In a sub-step E03, each of the LVTjxA training models is recalibrated in the common reference frame XA, by one or more elastic transformations TE of said spatial model LVTjxA, to obtain an LVTiA model oriented according to the common reference frame XA.

[0074] In a sub-step E04, each of the LM markers of the atlas A is positioned on each of the LVTiA models.

[0075] In a sub-step E05, each of the models LVTjA and provided with the LM references is brought back into the previous reference frame Xj whose axes are aligned with those of the common reference frame XA, by inverse elastic transformations Te1.

[0076] In a sub-step E06, an average model p is generated from all the spatial models LVTjxA recalibrated in their reference frames Xj and provided with the LM markers, as obtained at the end of step E05. The coordinates of the voxels of these different recalibrated spatial models LVTjA are for example averaged to obtain a set of voxels together defining said average model p.

[0077] Simultaneously, in a sub-step E07, a principal component analysis, or PCA, is carried out on the coordinates of the LM reference frames positioned on these recalibrated spatial models LVTjA to generate a statistical representation of the distribution of the coordinates of these LM reference frames, for example in the form of a matrix of covariance P of said coordinates.

[0078] The initial SSM model can then be obtained, in a sub-step E08, by a linear combination of the average model p and said statistical representation P, so that this initial SSM model has a variable form, defined by the statistical representation P, centered around the average model p. It will thus be possible, by varying the linear coefficients applied to the average model p and to said statistical representation P, to generate initial models similar to the training models.

[0079] In the case where the training models are complete heart models, it may be possible to provide a subsequent step of segmenting an initial model of a heart to obtain an initial SSM model of a left ventricle of the heart.

[0080] The method which has just been described may be replaced or supplemented by other methods of generating statistical shape models, linear or non-linear.

[0081] [Fig.4] shows an example of an embodiment of step E2 making it possible to generate, from the initial SSM model and the ECi ultrasound images, a 3DLVM spatial model of the patient's left ventricle.

[0082] As shown in [Fig.4], step E2 of the method according to the invention comprises, for each image of the sequence of ultrasound images ECi, a sub-step E21 of determining a transformation T;, or a combination of transformations T;, linear or elastic, which, when applied to the initial model SSM determined in step E0, minimizes an error function Err; between the 3DLVM model; resulting from the transformation T; applied to the initial model SSM, and this ultrasound image EQ.

[0083] In the example described, the error function Err; used is the distance between the contour of a section S_3DLVM; of the transformed initial model 3DLVM;, and the contour of the left ventricle in a section S_EC; of the ultrasound image EC;, the sections being considered according to the same plane, for example a transverse plane.

[0084] Said transformation T; may for example be determined by optimizing a vector of coefficients which, when applied to said statistical representation P of the initial model SSM in a linear combination of the average model p and this statistical representation P, minimize said error function Err;.

[0085] At the end of sub-step E21, the transformation T; which has been identified is applied to the initial SSM model to obtain a 3DLVM spatial model; corresponding to each EC image; and therefore to the shape of the patient's left ventricle at a given time.

[0086] In a sub-step E22, the set of 3DLVM spatial models is averaged to obtain a 3DLVM model globally corresponding to the set of ECi ultrasound images and thus forming the spatial model of the patient's left ventricle.

[0087] In the case where the ECi ultrasound images represent the entire heart of the patient, and where the initial model SSM is a complete model of a heart, step E2 may include a subsequent sub-step of segmentation of the 3DLVM spatial model resulting from sub-step E22, to generate a spatial model of the patient's left ventricle.

[0088] In a step E3, a MW motion map of the left ventricle is generated from the 3DLVM spatial model and the EC ultrasound images.

[0089] [Fig.5] shows an example of the implementation of step E3 making it possible to generate, from the 3DLVM model and the EC ultrasound images, a WM motion map of the patient's left ventricle.

[0090] Step E3 comprises a sub-step E31 of splitting the 3DLVM spatial model into a plurality of surface segments T_3DLVMk. In the example described, each surface segment T_3DLVMk is defined by a triangular mesh of the 3DLVM spatial model. Other types of polygonal mesh can be designed without departing from the scope of the present invention.

[0091] In a sub-step E32, for each surface segment T_3DLVMk, a value of a function Sk(t) representative of a geometric change of said segment T_3DLVMk at a given instant t of the complete cardiac cycle is determined from the echographic image EC; corresponding to this instant t.

[0092] In the example of [Fig.5], said value of the representative function Sk(t) corresponds to the ratio between the surface Ak(t) of the segment T_3DLVMk at the instant t of the cycle and the surface Ak(0) of the segment T_3DLVMk at the end of the diastole phase, or at the beginning of the systole phase.

[0093] Said value Sk(t) thus forms an element of the WM motion map associated with said surface segment T_3DLVMk of the 3DLVM spatial model. It indicates the capacity of the area of ​​the left ventricle corresponding to this surface segment T_3DLVMk to expand or contract during a change in the volume of the heart.

[0094] In a step E4, a photoplethysmographic signal from said PPG patient is received. This PPG signal, representing variations in blood volume in a blood vessel, may have been acquired in real time by a photoplethysmographic device worn by the patient. Step E4 may be implemented simultaneously with steps E1 to E3 or sequentially, following these steps E1 to E3. It may be envisaged that this step E4, and the steps described subsequently which result from it, are repeated more frequently than steps E1 to E3, so that the spatio-temporal model is generated with each new acquisition of a PPG signal.

[0095] In a step E5, an LVVC function, modeling the change or evolution of the volume of the left ventricle during the patient's cardiac cycle, is determined from the PPG signal. As shown in [Fig.6], said LVVC function is defined as a function of the relative value of the PPG signal.

[0096] Alternatively, in this step E5, it will be possible to determine from the photople- signal PPG thysmographic coefficients al and a2 of the equations of the biplane surface / length method modeling the volume of the left ventricle during the diastole and systole phases. The coefficient al could for example be determined by means of a first machine learning algorithm trained to estimate the value of the coefficient al from the values ​​of a part of a photoplethysmographic signal corresponding to a systole phase and the coefficient a2 could be determined by means of a second machine learning algorithm trained to estimate the value of the coefficient a2 from the values ​​of a part of a photoplethysmographic signal corresponding to a diastole phase.

[0097] The LVVC heart volume change function can thus be determined from the coefficients al and a2 thus estimated and the equations of the biplane surface / length method.

[0098] In an embodiment not shown, the heart volume change function LVVC may be determined by means of an LSTM type neural network receiving as input a signal representing the average of the PPG signal over all the cardiac cycles during which the PPG signal was acquired, as well as a value indicating the volume of the left ventricle at the end of the diastole phase, for example estimated from the 3DLVM spatial model or from an image of the heart acquired at the end of this phase.

[0099] In a step E6, a 4DLVM spatio-temporal model of the left ventricle is generated from the 3DLVM spatial model of the left ventricle, the LVVC function and the WM motion map.

[0100] In the example described, the set of the 3DLVM spatial model of the left ventricle, the LVVC function and the WM motion map makes it possible to generate an instance of a spatio-temporal model of the left ventricle at any time during the cardiac cycle. This set thus forms a 4DLVM spatio-temporal model of the left ventricle. Each new acquisition of a PPG signal thus leads to a new generation of the corresponding 4DLVM spatio-temporal model, in real time or within a short time, to the patient's left ventricle, thus allowing monitoring of the patient's cardiac mechanics.

[0101] In the example described, an instance 4DLVM(t) of a spatial-temporal model of the left ventricle at an instant t of the cardiac cycle may be determined, in a step E61, by means of the following equation:

[0102] [Math.l] 4DLVM(t) = 3DLVM(1+ , where 4DLVM(t) is the instance of the 4DLVM spatio-temporal model at time t, 3DLVM is the spatial model of the left ventricle, K a weighting coefficient calculated from the LVVC function, WM the left ventricular motion map and has a motion map weighting coefficient.

[0103] In particular, it will be possible to provide for fixing the value of the coefficient a to a value between 0 and 1.

[0104] The value of the weighting coefficient K may in particular be defined from the ratio between the volume of the 3DLVM spatial model and a volume estimated from said LVVC volume change function at said given instant.

[0105] In the example described, the coefficient K is calculated recursively, by correcting it using the ratio between the volume of the instant 4DLVM(t) determined during the previous iteration and the volume estimated from said volume change function LVVC at said given instant. Step E61 is thus repeated until the error between the volume estimated from the LVVC function and the volume of the instance the volume of the instant 4DLVM(t) is less than a predetermined threshold value.

[0106] Other methods for determining a 4DLVM(t) instance of a spatial-temporal model of the left ventricle at a time t of the cardiac cycle may be used from the 3DLVM spatial model and the LVVC function, without departing from the scope of the present invention.

[0107] The foregoing description clearly explains how the invention makes it possible to achieve the objectives it has set itself, namely, to be able to generate a spatio-temporal model of a part of a patient's heart, in an inexpensive, reliable manner that can be implemented easily, quickly or in real time, and regardless of the patient concerned. These objectives are achieved by combining on the one hand a spatial model of this part of the heart, generated by ultrasound images and on the other hand a volume change function determined from a photoplethysmo-graphic signal.

[0108] In any event, the invention cannot be limited to the embodiments specifically described in this document, and extends in particular to all equivalent means and to any technically effective combination of these means.

Claims

1.

2.

3. Claims A method for generating a spatio-temporal model (4DLVM) of a portion of a patient's heart, comprising the following steps: a. (El) Receiving a plurality of ultrasound images (EC;) of a patient's heart; b. (E2) Generation of a spatial model (3DLVM) of at least a given portion of the heart from the plurality of received ultrasound images; c. (E4) Receiving a photoplethysmographic (PPG) signal from said patient; d. (E5) Determination of a heart volume change function (LVVC), during all or part of a cardiac cycle of the patient's heart, from the photoplethysmographic signal; e. (E6) Generating a spatio-temporal model (4DLVM) of said part of the heart using the spatial model (3DLVM) of said part of the heart and said heart volume change function (LVVC). Method according to the preceding claim, wherein the step of generating said spatial model of said part of the heart comprises a sub-step of transforming a predetermined initial model (SSM) of a heart to generate a spatial model of said part of the heart corresponding to the plurality of received ultrasound images. Method according to the preceding claim, comprising a prior step (EO) of generating a predetermined initial model (SSM) of a heart, said step comprising the following sub-steps: a. (E01) Receiving a plurality of spatial models (LVTj) of the heart of a plurality of patients; b. (E02) Alignment of each of the spatial models in a reference frame (Xj) aligned with a common reference frame (XA); c. (E06) Determination of an average model (p) from the set of recalibrated spatial models (LVTixA), the predetermined initial model (SSM) being generated from said average model.

4. Method according to the preceding claim, in which the common reference frame (XA) comprises a plurality of predetermined reference points (LM), the prior step (EO) of generating the predetermined initial model (SSM) comprising the following sub-steps: a. (E04) Positioning of each of the predetermined reference points (LM) in each recalibrated spatial model (LVTiA); b. (E07) Statistical analysis of the coordinates of the reference points (LM) of the set of recalibrated spatial models (LVTjA) to generate a statistical representation (P) of the distribution of the coordinates of these reference points among the recalibrated models, the predetermined initial model (SSM) being generated from said average model (p) and said statistical representation (P).

5. Method according to one of claims 2 to 4, in which the step of generating (E2) said spatial model (3DLVM) of said part of the heart comprises, for each image of the plurality of received ultrasound images (ECi), a sub-step (E21) of transforming the predetermined initial model (SSM) of a heart to generate a spatial model (3DLVM;) of said part of the heart corresponding to said image, and a sub-step of averaging (E22) the spatial models (3DLVM;) to generate the spatial model (3DLVM) of said part of the heart.

6. Method according to one of the preceding claims, the method comprising a step (E3) of generating, from the spatial model (3DLVM) of said part of the heart, a motion map (WM) of said spatial model, each element (Sk(t)) of the map being associated with a zone (T_3DLVMk) of the spatial model and indicating the capacity of said zone to expand or contract during a change in volume of the heart; said spatio-temporal model of said part of the heart (4DLVM) being generated by means of the spatial model (3DLVM) of said part of the heart, said heart volume change function (LVVC) and the motion map (WM) of said spatial model.

7. Method according to the preceding claim, characterized in that said plurality of ultrasound images (EC;) comprises at least one sequence of ultrasound images acquired during a complete cardiac cycle of the patient's heart, and in that the step (E3) of generating the motion map (WM) of said spatial model comprises the following sub-steps: a. (E31) Splitting the spatial model (3DLVM) of said part of the heart into a plurality of surface segments (T_3DLVMk); b. (E32) For each surface segment, determining, from said sequence of images, at least one value and / or function (Sk(t)) representative of a geometric change of said segment during the complete cardiac cycle; said value and / or function forming an element of the motion map associated with said surface segment of the spatial model.

8. Method according to one of claims 6 or 7, in which the spatio-temporal model (4DLVM) of said part of the heart comprises a morphological function making it possible to generate an instance (4DLVM(t)) of a spatial model of said part of the heart at a given instant (t), said instance being determined by an affine transformation of the spatial model of said part of the heart by a weighting element (K) calculated from a value of said volume change function (LVVC) of the heart at said given instant and the motion map (WM) of said spatial model.

9. Method according to the preceding claim, characterized in that the generation of said instance (4DLVM(t)) is implemented recursively until the error between the volume of said part of the heart determined from said value of said heart volume change function (LVVC) and the volume of the instance is less than a predetermined threshold value.

10. Method according to one of the preceding claims, characterized in that at least a part of the heart volume change function (LVVC) is identical to the relative value of a part of the photoplethysmographic signal (PPG).

11. A computer program comprising program code which is designed to implement the method according to one of the preceding claims.

12. Data carrier on which the computer program according to claim 11 is recorded.

13. A computer system for generating a spatiotemporal model of a portion of a patient's heart, the computer system comprising a computing unit capable of receiving a plurality of ultrasound images (ECi) of a patient's heart and a photoplethysmo-graphic signal (PPG) of said patient, the computing unit being arranged to implement the method according to any one of claims 1 to 10.

Images