Method for generating a spatio-temporal model of a part of a patient's heart

Abstract

The invention relates to a method for generating a spatio-temporal model (4DLVM) of a part of a patient's heart, comprising the following steps: (E1) Receiving a plurality of ultrasound images (EG) of a patient's heart; (E2) Generating a spatial model (3DLVM) of at least a given part of the heart from the plurality of ultrasound images received; Receiving a photoplethysmographic (PPG) signal from said patient; Determining a function of change of volume of the heart (LVVC), during all or part of a cardiac cycle of the patient's heart, from the photoplethysmographic signal; Generating a spatio-temporal model (3DLVM) of said part of the heart by means of the spatial model of said part of the heart and said function of change of volume of the heart.

Description

Description Title of the invention: Method for generating a spatio-temporal model of a part of a patient's heart

[0001] The invention relates to the field of cardiology and medical imaging. More specifically, the invention relates to a method for generating a spatio-temporal model of a part of a patient's heart.

[0002] In the context of preparing a medical intervention targeting a patient's heart, it is common to perform a simulation of the intervention beforehand, particularly in order to reduce the risk of the intervention. This type of simulation requires modeling the patient's heart, both spatially and temporally, in order to obtain a model of the heart representative of the evolution of the heart's shape during the patient's cardiac cycle, particularly during the systole and diastole phases.

[0003] Conventionally, a spatio-temporal model of the patient's heart, or part of the patient's heart, also called a 4D model, is generated from 3D images of the heart obtained by a medical imaging method such as angiography, fluoroscopy or magnetic resonance imaging.

[0004] These 3D images are composed of stacks of 2D images of the heart, each representing a section or slice of the heart through a given plane. Each element of the 3D image thus represents a unit of volume of the patient's heart. When a sequence of 3D images is acquired, for example during a period corresponding to at least one cardiac cycle of the patient, we then have both spatial and temporal information about the shape of the heart, so that it is possible to derive a 4D model.

[0005] However, this solution has several drawbacks.

[0006] On the one hand, medical imaging equipment such as angiography, fluoroscopy or magnetic resonance imaging is particularly expensive and complex to handle. In addition, this equipment is incompatible with the presence of certain materials in the patient's body, particularly metals, or with certain pathologies likely to prevent the acquisition of 3D images. Finally, these medical imaging modalities can be harmful or contraindicated for the patient, insofar as they require the emission of X-rays and / or the injection of a contrast agent. These disadvantages constitute obstacles to the systematic implementation of a simulation of a cardiac intervention based on a 4D model, regardless of the patient.

[0007] On the other hand, each 3D image in a 3D image sequence is likely to form a 3D model of the heart, corresponding to the shape of the heart at a given moment in the cardiac cycle. The different 3D models thus have different volumes, or even different orientations. It is therefore necessary to carry out complex geometric or morphological transformations on each of these 3D models and / or to align them in a common reference frame in order to generate a 4D model.

[0008] Finally, given the drawbacks mentioned above, this solution does not allow to obtain a 4D model of the heart in real time, or at least in the context of monitoring cardiac mechanical activity. Indeed, this type of monitoring requires the acquisition of cardiac data over long continuous periods, which is incompatible with acquisition modalities such as angiography, fluoroscopy or magnetic resonance imaging.

[0009] There is therefore a need for a method for generating a spatio-temporal model of part of a patient's heart, which is inexpensive, reliable and which can be implemented easily, quickly or in real time, and regardless of the patient concerned.

[0010] The present invention is placed in this context and aims to meet this need.

[0011] For these purposes, the invention relates to a method for generating a spatio-temporal model of a portion of a patient's heart, comprising the following steps: a. Receiving a plurality of ultrasound images of a patient's heart; b. Generating a spatial model of at least one given portion of the heart from the plurality of received ultrasound images; c. Receiving a photoplethysmographic signal from said patient; d. Determining a heart volume change function, during all or part of a cardiac cycle of the patient's heart, from the photoplethysmographic signal; e. Generating a spatio-temporal model of said portion of the heart using the spatial model of said portion of the heart and said heart volume change function.

[0012] The invention thus proposes, on the one hand, to spatially model all or part of the patient's heart using images acquired by an echocardiography device. This medical imaging modality uses ultrasound waves to obtain 2D or 3D images of the patient's heart. It thus has various advantages, such as being risk-free for the patient's health, maintaining acceptable imaging quality and finally being able to be implemented using inexpensive and easy-to-use equipment. It is notably possible to connect an acquisition device to a portable device, such as a portable ultrasound scanner, or even a smartphone, so that the patient can perform their own echocardiography. It is thus possible to generate a spatial, or 3D, model of the heart or part of the heart, such as a right or left ventricle, a right or left atrium, a heart valve or a blood vessel, such as an aorta.

[0013] The invention also proposes to model a function of change in volume of the patient's heart from a photoplethysmographic signal. Photoplethysmography consists of measuring, by means of an optical sensor or a photodetector, the variations in the light emitted by a light source towards a blood vessel and absorbed or reflected by this blood vessel. A photoplethysmographic signal resulting from this measurement thus represents the evolution or variations in the blood volume in a blood vessel. It has thus been demonstrated that this photoplethysmographic signal can be correlated with the variations in volume of the heart during the cardiac cycle and in particular during the phases of systole and diastole. It is therefore possible to reconstruct, from this signal, a volume change function of the patient's heart. This modality requires only an optical sensor and a light source, it can be easily integrated into an acquisition device that can be worn by the patient, so that the volume change function can be generated in real time and in a simple, fast and inexpensive way.

[0014] According to the invention, it is therefore possible to combine the spatial information resulting from the 3D model of the heart, obtained through the ultrasound images, with the temporal information resulting from the volume change function of the patient's heart, obtained through the photoplethysmographic signal, to generate a spatio-temporal, or 4D, model of the patient's heart. Given the advantages mentioned above, this 4D spatio-temporal model can be generated in real time or in a short time, to meet a need for monitoring the patient's cardiac mechanical activity, and regardless of the type of patient. In particular, it will be possible to design the spatial model to be generated beforehand, and the spatio-temporal model to be regenerated in real time or with each new acquisition of a photoplethysmographic signal.The spatial model can thus be generated once by a prior ultrasound, or according to an acquisition frequency significantly lower than that of the photoplethysmographic signal.

[0015] The method according to the invention is advantageously implemented by a computer system. In the context of the present invention, a "computer system" may refer to, but is not limited to, a desktop computer, a laptop computer, a digital tablet, a smartphone, a test equipment, a network device, a controller, a digital signal processor, a computing engine in an appliance, a consumer electronics device, a portable computing device and / or another electronic device, and / or any combination thereof suitable for implementing the method according to the invention. Furthermore, the steps of the method according to the invention may all be performed on a single device. Alternatively, the steps of the method according to the invention may be performed on multiple devices, connected by wires or by a wireless connection.

[0016] In the context of the present invention, the term "ultrasound images of a patient's heart" or "echocardiogram" means a two-dimensional, 2D, or three-dimensional, 3D, image of all or part of the patient's heart, acquired by means of a probe comprising an ultrasound wave transmitter and one or more sensors, in particular arranged in a linear or matrix manner. These may be B-type or Doppler-type ultrasound images. The probe may in particular be connected, wired or wirelessly, to a portable unit, arranged to convert the ultrasound signals acquired by the sensors of the probe into ultrasound images.

[0017] In the context of the present invention, the term "photoplethysmographic signal", or PPG signal, means an alternating signal representing variations in blood volume in a blood vessel, obtained by means of a photoplethysmographic device, comprising a light source, visible or infrared, and an optical sensor or a photodetector. This may include an AC component of a signal acquired by a photoplethysmographic device. This device may be integrated into equipment intended to be worn by the patient, such as a smartphone, smartwatch, bracelet, ring, headset, or earpiece. The photoplethysmographic signal may also be or include a remote photoplethysmographic (rPPG) signal derived from a video of the patient. For example, changes in blood volume may be detected in this video by analyzing changes in skin color on the patient's face or other parts of the head or body, captured by a smartphone video camera or webcam. These rPPG signals can be obtained without direct contact with the patient's skin, thus providing a non-invasive and remote method for acquiring information on blood flow and heart rate.

[0018] In the context of the present invention, the term "heart volume change function" means a mathematical function representing the change in the volume of all or part of the heart, during all or part of a cardiac cycle of the patient's heart, in particular during the systole and diastole phases. It may in particular be composed of several sub-functions, in particular a first sub-function representing the ejection fraction of the left or right ventricle during a systole phase and a second sub-function representing the filling fraction of the left ventricle during a diastole phase.

[0019] If desired, the steps of receiving the photoplethysmographic signal and determining the heart volume change function may be implemented by a computer module, and in particular a software application, embedded in the electronic equipment intended to be worn by the patient and incorporating the photoplethysmographic device.

[0020] In the context of the present invention, the term "spatio-temporal model of a part of the heart" means a model representing a part of the heart during a given period of time, for example during a cardiac cycle. It may be a sequence of spatial models representing said part of the heart at different times of the cardiac cycle or a spatial model of which each element is associated with a displacement function so that it is possible to generate a spatial model representing said part of the heart at any time of the cardiac cycle. The spatial model may comprise a structured set of volume elements, or voxels, or an unstructured point cloud, and may represent an internal wall and / or an external wall and / or an approximation of the shape of said part of the heart.Where appropriate, anatomical regions of said part of the heart may be identified in the spatio-temporal model, for example by labeling the points or voxels defining these regions or by positioning markers associated with these regions on the spatio-temporal model.

[0021] In one embodiment of the invention, the step of generating said spatial model of said part of the heart comprises a sub-step of transforming a predetermined initial model of a heart to generate a spatial model of said part of the heart corresponding to the plurality of ultrasound images received. This type of modeling, also called model-based segmentation, has the advantage of being robust and automatic.

[0022] Advantageously, said generation step may comprise a sub-step of determining a transformation which, when applied to the predetermined initial model, minimizes an error function between the transformed initial model and at least one of the received ultrasound images. Said transformation may comprise one or more affine transformations, one or more elastic transformations, or a combination of affine and / or elastic transformations.

[0023] This sub-step may be based on 2D or 3D ultrasound images. For example, said sub-step of determining a transformation may be a step of determining a transformation which, when applied to the predetermined initial model, minimizes the distance between the contour of a section of the transformed initial model, by a plane corresponding to one of the received ultrasound images, and the contour of the heart in said received ultrasound image. It may be envisaged as a variant that the transformation is determined by correspondence between the transformed initial model and a 3D ultrasound image.Where appropriate, said transformation may be determined by means of one or more machine learning algorithms, trained to determine from an ultrasound image, a transformation which, when applied to the predetermined initial model, minimizes an error function between the transformed initial model and this received ultrasound image.

[0024] If desired, said step of generating said spatial model of said portion of the heart may comprise a sub-step of transforming a predetermined initial model of a heart to generate a spatial model of the patient's heart, corresponding to the plurality of received ultrasound images and a sub-step of segmenting said spatial model to generate said spatial model of said portion of the heart. For example, said spatial model of the heart may comprise a plurality of regions of interest each labeled with an identifier of an anatomical region of the heart, said segmentation sub-step comprising selecting one or more regions of interest corresponding to said portion of the heart.

[0025] Advantageously, the method according to the invention comprises a prior step of generating a predetermined initial model of a heart, said step comprising the following sub-steps: a. Receiving a plurality of spatial models of the heart of a plurality of patients; b. Registrar of each of the spatial models in a reference frame aligned with a common reference frame; c. Determining an average model from all of the reregistered spatial models, the predetermined initial model being generated from said average model.

[0026] This type of initial model is also called a statistical shape model or SSM (from the English “Statistical Shape Model”) and allows for a segmentation of a new model in a robust and reliable way.

[0027] According to an exemplary embodiment of the invention, each of the models of said plurality of spatial models is a training model generated beforehand from ultrasound images of a heart of one of the patients. Each training model is then recalibrated, by an affine transformation, to a reference frame aligned with a common reference frame, for example the reference frame of an atlas of a heart. Each spatial model is thus moved to a coordinate system whose axes are aligned with those of the common reference frame. The coordinates of the elements of the different recalibrated spatial models are then averaged to obtain a set of average elements together defining said average model.

[0028] It will be advantageous to segment the average model into regions of interest based on landmarks and / or regions of interest in the heart atlas corresponding to given anatomical areas of the heart, such as the ventricles, atria, aortas and / or heart valves.

[0029] Advantageously, the common reference system comprises a plurality of predetermined reference points, the prior step of generating the predetermined initial model comprising the following sub-steps: a. Positioning of each of the predetermined reference points in each recalibrated spatial model; b. Statistical analysis of the coordinates of the reference points of all the recalibrated spatial models to generate a statistical representation of the distribution of the coordinates of these reference points among the recalibrated models, the predetermined initial model being generated from said average model and said statistical representation.

[0030] Depending on these characteristics, the predetermined initial model can present a variable shape, defined by the statistical representation, centered around the average model.

[0031] According to an exemplary embodiment of the invention, the common reference system is that of an atlas of a heart comprising a plurality of anatomical landmarks, for example defining the position of given anatomical structures, and / or geometric landmarks, for example defining borders of the heart, and / or reference landmarks, for example defining vessel junction points.Where appropriate, for each of the recalibrated spatial models, the sub-step of positioning each of the predetermined reference points comprises the recalibration of said recalibrated spatial model in the common reference frame of the atlas, for example by means of an elastic transformation of said recalibrated spatial model, the positioning of each of said atlas reference points on the recalibrated spatial model in the common reference frame of the atlas, then the recalibration of said recalibrated spatial model from the common reference frame into its initial reference frame aligned with the common reference frame, for example by means of an inverse elastic transformation.

[0032] For example, the statistical analysis of the coordinates of the reference points of the set of recalibrated spatial models could include a principal component analysis making it possible to obtain, for example, a covariance matrix of said coordinates. Alternatively or cumulatively, the statistical analysis of the coordinates of the reference points of the set of recalibrated spatial models may include the implementation of other data analysis methods, linear or non-linear.

[0033] Preferably, the predetermined initial model can be obtained by a linear combination of the average model and said statistical representation.

[0034] Advantageously, the step of generating said spatial model of said part of the heart comprises, for each image of the plurality of received ultrasound images, a sub-step of transforming the predetermined initial model of a heart to generate a spatial model of said part of the heart corresponding to said image, and a sub-step of averaging the spatial models to generate the spatial model of said part of the heart. In this example, each of the ultrasound images makes it possible to obtain, from the same initial model, a spatial model corresponding to this image and therefore to the shape of the patient's heart at a given time. All of the spatial models are then averaged to obtain a model corresponding overall to all of the ultrasound images.

[0035] In one embodiment of the invention, the method comprises a step of generating, from the spatial model of said part of the heart, a motion map of said spatial model, each element of the map being associated with a zone of the spatial model and indicating the capacity of said zone to expand or contract during a change in volume of the heart. Where appropriate, said spatio-temporal model of said part of the heart is generated by means of the spatial model of said part of the heart, said volume change function of the heart and the motion map of said spatial model. In this embodiment, it is assumed that each region of the heart, or of a wall of the heart, moves, during the cardiac cycle, differently from other regions, in particular if it is a healthy region or a region likely to be affected by heart disease or diabetes.According to these characteristics, it is possible to generate, from the spatial model and / or the ultrasound images, a motion map, indicating, for each point or voxel or for groups of points or voxels of the spatial model, the capacity of a region of the heart or of a wall of the heart, at a position corresponding to said point or voxel or said group of points or voxels, to move. The combination of the motion map and the volume change function thus makes it possible to model, with great precision, the shape of the patient's heart at a given moment of his cardiac cycle.

[0036] Advantageously, said plurality of ultrasound images comprises at least one sequence of ultrasound images acquired during a complete cardiac cycle of the patient's heart. Where appropriate, the step of generating the motion map of said spatial model comprises the following substeps: a. Splitting the spatial model of said part of the heart into a plurality of surface segments; b. For each surface segment, determining, from said sequence of images, at least one value and / or function representative of a geometric change of said segment during the complete cardiac cycle; said value and / or function forming an element of the motion map associated with said surface segment of the spatial model.

[0037] For example, said function representing a geometric change of a segment can be determined from the evolution of the ratio between the surface area of ​​the segment at an instant of the cycle and the surface area of ​​the segment at a predetermined instant of the cycle, such as at the end of the diastole phase.

[0038] In particular, it will be possible to consider estimating the cardiac cycle duration using the patient's photoplethysmographic signal.

[0039] In one embodiment of the invention, the spatio-temporal model of said part of the heart comprises a morphological function making it possible to generate an instance of a spatial model of said part of the heart at a given instant, said instance being determined by an affine transformation of the spatial model of said part of the heart by a weighting element calculated from a value of said volume change function of the heart at said given instant and the motion map of said spatial model.

[0040] For example, said weighting element may be determined from the ratio between the volume of the spatial model of said part of the heart and a volume estimated from said volume change function at said given time.

[0041] Advantageously, the generation of said instance may be implemented recursively until the error between the volume of said part of the heart determined from said value of said heart volume change function and the volume of the instance is less than a predetermined threshold value.

[0042] In one embodiment of the invention, at least a portion of the heart volume change function is identical to the relative value of a portion of the photoplethysmographic signal.

[0043] In another embodiment, parameters and / or coefficients of an equation for determining the volume of all or part of the heart, and in particular of a cavity of the heart, are determined from the photoplethysmographic signal, said heart volume change function being determined from said parameters and / or coefficients. For example, said parameters and / or coefficients may be estimated by one or more machine learning algorithms, trained to estimate values ​​of said parameters and / or coefficients from data from a photoplethysmographic signal.

[0044] According to an exemplary embodiment of the invention, the method comprises a step of estimating the coefficients a1 and a2 of the equation of the biplane surface / length method, from the values ​​of the photoplethysmographic signal, the coefficient a1 being determined by means of a first machine learning algorithm trained to estimate the value of the coefficient a1 from the values ​​of a part of a photoplethysmographic signal corresponding to a systole phase and the coefficient a2 being determined by means of a second machine learning algorithm trained to estimate the value of the coefficient a2 from the values ​​of a part of a photoplethysmographic signal corresponding to a diastole phase.

[0045] The first algorithm may, for example, be trained using a data set comprising ultrasound images of patients' hearts and, for each image, a part of a photoplethysmographic signal corresponding to a systole phase of the heart represented by this image. The first algorithm may in particular be trained to determine the value of the coefficient al which minimizes a cost function depending on the volume of a cavity of the heart, in particular the left or right ventricle, estimated from the ultrasound image and the volume of this cavity, estimated using the equation of the biplane surface / length method modeling the volume of the heart during the systole phase and taking into account this coefficient al.This cost function could, for example, correspond to the ratio between the difference in the volume of the cavity estimated using the ultrasound image and the volume of the cavity estimated using the photoplethysmographic signal and the difference in the ejected volume estimated using the ultrasound image and the ejected volume estimated using the photoplethysmographic signal.

[0046] The second algorithm could for example be trained using a data set comprising ultrasound images of patients' hearts and, for each image, a part of a photoplethysmographic signal corresponding to a diastole phase of the heart represented by this image. The second algorithm could in particular be trained to determine the value of the coefficient a2 which minimizes a cost function depending on the volume of a cavity of the heart, in particular the left or right ventricle, estimated from the ultrasound image and the volume of this cavity, estimated using the equation of the biplane surface / length method modeling the volume of the heart during the diastole phase and taking into account this coefficient a2.This cost function could, for example, correspond to the ratio between the difference in the volume of the cavity estimated using the ultrasound image and the volume of the cavity estimated using the photoplethysmographic signal and the difference in the filled volume estimated using the ultrasound image and the filled volume estimated using the photoplethysmographic signal.

[0047] In yet another embodiment of the invention, said heart volume change function may be estimated by one or more machine learning algorithms receiving said photoplethysmographic signal as input, said algorithm(s) being trained to determine a heart volume change function from a signal provided to it as input.

[0048] The step of determining the heart volume change function can thus be implemented by an artificial neural network, such as a dense type neural network comprising several successive layers of neurons and in which each of these layers is totally interconnected with the previous and the next, or as a recurrent type neural network in which feedback loops are provided, or even a recurrent type with short and long term memory, also called LSTM (Long Short-Term Memory). The artificial neural network may for example have been trained from a training data set comprising both signals photoplethysmographic signals of patients and the volume change functions of the heart of these patients acquired simultaneously. For each photoplethysmographic signal of the training set, the network thus predicted a volume change function of the heart, which was compared to the actual function associated with this signal in order to determine an error. This error was back-propagated in order to correct the parameters of the network to minimize this error, such as in particular the synaptic weights and the biases of each neuron. The iteration of these predictions and back-propagations on the entire training data set thus makes it possible to train the neural network to reliably estimate a volume change function of the heart from a photoplethysmographic signal provided to it as input.

[0049] In a preferred embodiment, the step of determining the heart volume change function comprises a sub-step of segmenting the photoplethysmographic signal into sub-signals each corresponding to a cardiac cycle, i.e. a diastole phase and a systole phase, a sub-step of determining an average signal of all the segmented sub-signals, and a step of estimating the heart volume change function by means of a recurrent type neural network with short and long term memory, receiving as input said average signal and a volume of the heart at the end of the diastole phase.

[0050] For example, said network may have an encoder-decoder architecture and include an encoding stage comprising two layers of 256 LSTM-type neurons and receiving as input the average signal and the volume, a decoding stage comprising two layers of 256 LSTM-type neurons and receiving as input the output of the encoding stage, and a dense stage comprising a layer of 128 neurons fully connected to the output of the decoding stage.

[0051] Where appropriate, the method comprises a preliminary step of training the neural network using a training data set consisting of photoplethysmographic signals from 30 patients acquired during several cardiac cycles and measurements of the evolution of the volume of the patients' hearts during these cardiac cycles, obtained by imaging, the neural network thus being trained to determine a function of change in volume of the heart during a cardiac cycle.

[0052] In one embodiment of the invention, the method comprises a step of calculating, from the spatio-temporal model of said part of the heart, at least one cardiovascular physiological parameter of the patient.

[0053] In the case where said part of the heart is a left ventricle, said cardiovascular physiological parameter could for example be a telediastolic volume, or VTD, or EDV (End Diastole Volume), corresponding to the volume of said ventricle at the end of the diastole phase, when the ventricle relaxes and therefore presents its maximum volume. Said telediastolic volume could thus be determined as the maximum volume of said spatio-temporal model during a cardiac cycle.

[0054] Alternatively or cumulatively, said cardiovascular physiological parameter could, for example, be a telesystolic volume, or VTS, or ESV (End Systole Volume), corresponding to the volume of said ventricle at the end of the systole phase, when the ventricle contracts and therefore presents its minimum volume. Said telesystolic volume could thus be determined as the minimum volume of said spatio-temporal model during a cardiac cycle.

[0055] Alternatively or cumulatively, said cardiovascular physiological parameter could, for example, be a systolic ejection volume, or VES, or SV (Stroke Volume), corresponding to the volume of blood ejected by the ventricle at each cycle. Said systolic ejection volume could thus be determined as the difference between the end-diastolic volume and the end-systolic volume.

[0056] Alternatively or cumulatively, said cardiovascular physiological parameter could for example be an ejection fraction, or FE, or EF (Ejection Fraction), indicating the ratio between the volume of blood ejected by the ventricle at each cycle and the maximum volume of blood that the ventricle can contain. Said ejection fraction could thus be determined as the ratio between the systolic ejection volume and the end-diastolic volume, expressed as a percentage.

[0057] Alternatively or cumulatively, said cardiovascular physiological parameter could, for example, be an average flow rate of blood ejected by the ventricle. This flow rate could be determined from the stroke volume or the volume variation of the spatio-temporal model during the cardiac cycle, and from the duration of the cardiac cycle or the heart rate.

[0058] It will be possible to consider estimating other cardiovascular physiological parameters of the patient from the spatio-temporal model without departing from the scope of the present invention.

[0059] The invention also relates to a computer program comprising a program code which is designed to implement the method according to the invention.

[0060] The invention also relates to a data medium on which the computer program according to the invention is recorded.

[0061] The invention also relates to a computer system for generating a spatiotemporal model of a part of a patient's heart, the computer system comprising a computing unit capable of receiving a plurality of ultrasound images of a patient's heart and a photoplethysmographic signal from said patient, the computing unit being arranged to implement the method according to the invention.

[0062] The invention also relates to a method for determining a function of the volume of a patient's heart, the method being implemented by a computer module and comprising the following steps: a. Receiving a photoplethysmographic signal from said patient; b. Determining a function of change in volume of the heart, during all or part of a cardiac cycle of the patient's heart, from the photoplethysmographic signal.

[0063] The present invention is now described using examples which are purely illustrative and in no way limitative of the scope of the invention, and from the appended drawings, drawings in which the various figures represent:

[0064] [Fig. 1] represents, schematically and partially, a method for generating a spatio-temporal model of a part of a patient's heart according to an embodiment of the invention;

[0065] [Fig. 2] represents, schematically and partially, a sequence of ultrasound images of a patient's heart received during a step of the method of [Fig. 1];

[0066] [Fig. 3] represents, schematically and partially, the generation of a predetermined initial model of a heart, implemented during a step of the method of [Fig. 1];

[0067] [Fig. 4] represents, schematically and partially, the transformation of the model of [Fig. 3] to generate a spatial model of a part of a heart corresponding to the plurality of ultrasound images of [Fig. 2], implemented during a step of the method of [Fig. 1];

[0068] [Fig. 5] represents, schematically and partially, the generation of a motion map of a spatial model of a heart, implemented during a step of the method of [Fig. 1]; and

[0069] [Fig. 6] represents, schematically and partially, the determination of a volume change function of a heart from a photoplethysmographic signal, implemented during a step of the method of [Fig. 1],

[0070] In the following description, elements which are identical, by structure or by function, appearing in different figures retain, unless otherwise specified, the same references.

[0071] [Fig. 1] shows a method for generating a spatio-temporal model of a part of a patient's heart according to one embodiment of the invention. The method is implemented by a computer system (not shown).

[0072] In a first step E1 of the method, a plurality of EC ultrasound images of all or part of the patient's heart are received by the computer system.

[0073] These EC ultrasound images may have been previously acquired by a cardiologist's echocardiography device or by a portable ultrasound probe connected to the patient's own smartphone.

[0074] [Fig. 2] shows a sequence of three-dimensional, 3D, ECi to ECN images of a left ventricle of the patient's heart, acquired during a cardiac cycle of the patient, from the beginning of a diastole phase to the end of a systole phase, forming said plurality of ultrasound images. It is conceivable that the EC images are two-dimensional images and / or that the acquisition period is shorter or longer than a cardiac cycle and / or that the EC images are disordered without departing from the scope of the present invention. It is conceivable that the EC images represent another part of the patient's heart, such as a right ventricle or a left or right atrium, or even the entire patient's heart rather than just a part.

[0075] In a second step E2 of the method, a spatial model of the left ventricle of the patient's heart 3DLVM is generated from the plurality of received ultrasound images EC.

[0076] In the example described, the 3DLVM spatial model can be generated again with each new acquisition of EC ultrasound images, by the cardiologist or by the patient.

[0077] In order to generate the 3DLVM spatial model, a predetermined initial model SSM of a left ventricle of a heart was generated in a prior step E0.

[0078] [Fig. 3] shows an example of the implementation of step E0 allowing the generation of an initial SSM model of the statistical shape model type.

[0079] This step E0 comprises a sub-step E01 of receiving a plurality of spatial models LVTj, forming a training set. In the example described, each of the LVT training models was generated beforehand from ultrasound images of a left ventricle of the heart of one of the patients.

[0080] In a sub-step E02, each of the training models LVTj is recalibrated in a reference frame Xj aligned with the same common reference frame XA, namely the reference frame of an atlas A of a predetermined core. Each model LVTj can for example be transformed by one or more affine or linear transformations TA, of the translation, rotation, enlargement, reduction, symmetry type, to transform it into a model LVTIXA oriented according to a reference frame Xj whose axes are aligned with those of the common reference frame XA.

[0081] Atlas A comprises a plurality of anatomical LM landmarks, for example defining the position of given anatomical structures, such as aortas and / or heart valves, geometric landmarks, for example defining ventricular borders, and reference landmarks, for example defining vessel junction points.

[0082] In a sub-step E03, each of the LVTJXA training models is recalibrated in the common reference frame XA, by one or more elastic transformations TE of said spatial model LVTjXA, to obtain an LV A model oriented according to the common reference frame XA.

[0083] In a sub-step E04, each of the LM reference points of atlas A is positioned on each of the LVTIA models.

[0084] In a sub-step E05, each of the LVTJA models and provided with the LM references is brought back into the previous reference frame Xj whose axes are aligned with those of the common reference frame XA, by inverse elastic transformations TE -1 .

[0085] In a sub-step E06, an average model p is generated from all the spatial models LVTjXA recalibrated in their reference frames Xj and provided with the LM references, as obtained at the end of step E05. The coordinates of the voxels of these different recalibrated spatial models LVTjA are for example averaged to obtain a set of voxels together defining said average model p.

[0086] Simultaneously, in a sub-step E07, a principal component analysis, or PCA, is carried out on the coordinates of the LM reference frames positioned on these recalibrated spatial models LVTJA to generate a statistical representation of the distribution of the coordinates of these LM reference frames, for example in the form of a covariance matrix P of said contact details.

[0087] The initial SSM model can then be obtained, in a sub-step E08, by a linear combination of the average model p and said statistical representation P, so that this initial SSM model has a variable form, defined by the statistical representation P, centered around the average model p. It will thus be possible, by varying the linear coefficients applied to the average model p and to said statistical representation P, to generate initial models similar to the training models.

[0088] In the case where the training models are full heart models, we may possibly plan a subsequent step of segmentation of an initial model of a heart to obtain an initial SSM model of a left ventricle of the heart.

[0089] The method just described can be replaced or supplemented by other methods of generating statistical shape models, linear or non-linear.

[0090] [Fig. 4] shows an example of the implementation of step E2 allowing the generation, from the initial SSM model and the EC ultrasound images, of a 3DLVM spatial model of the patient's left ventricle.

[0091] As shown in [Fig. 4], step E2 of the method according to the invention comprises, for each image of the sequence of ultrasound images EC,, a sub-step E21 of determining a transformation T,, or a combination of transformations T,, linear or elastic, which, when applied to the initial model SSM determined in step E0, minimizes an error function Ern between the model 3DLVM] resulting from the transformation T, applied to the initial model SSM, and this ultrasound image EC,.

[0092] In the example described, the error function Ern used is the distance between the contour of a section S_3DLVMj of the initial transformed model 3DLVM], and the contour of the left ventricle in a section S_ECj of the EC ultrasound image, the sections being considered along the same plane, for example a transverse plane.

[0093] Said transformation T, may for example be determined by optimizing a vector of coefficients which, when applied to said statistical representation P of the initial model SSM in a linear combination of the average model p and this statistical representation P, minimize said error function Ern.

[0094] At the end of sub-step E21, the transformation T, which has been identified, is applied to the initial SSM model to obtain a spatial model 3DLVM] corresponding to each EC image, and therefore to the shape of the patient's left ventricle at a given time.

[0095] In a sub-step E22, the set of 3DLVM spatial models is averaged to obtain a 3DLVM model globally corresponding to the set of EC ultrasound images, and thus forming the spatial model of the patient's left ventricle.

[0096] In the case where the EC ultrasound images represent the entire patient's heart, and where the initial SSM model is a complete model of a heart, step E2 may include a subsequent sub-step of segmentation of the 3DLVM spatial model resulting from sub-step E22, to generate a spatial model of the patient's left ventricle.

[0097] In a step E3, a MW motion map of the left ventricle is generated from the 3DLVM spatial model and EC ultrasound images.

[0098] [Fig. 5] shows an example of the implementation of step E3 allowing the generation, from the 3DLVM model and the EC ultrasound images, of a WM motion map of the patient's left ventricle.

[0099] Step E3 includes a sub-step E31 of splitting the 3DLVM spatial model into a plurality of surface segments T_3DLVMk. In the example described, each surface segment T_3DLVMk is defined by a triangular mesh of the 3DLVM spatial model. Other types of polygonal mesh can be designed without departing from the scope of the present invention.

[0100] In a sub-step E32, for each surface segment T_3DLVMk, a value of a function Sk(t) representative of a geometric change of said segment T_3DLVMk at a given instant t of the complete cardiac cycle is determined from the ultrasound image ECi corresponding to this instant t.

[0101] In the example of [Fig. 5], said value of the representative function Sk(t) corresponds to the ratio between the surface Ak(t) of the segment T_3DLVMk at time t of the cycle and the surface Ak(0) of the segment T_3DLVMk at the end of the diastole phase, or at the beginning of the systole phase.

[0102] Said value Sk(t) thus forms an element of the WM motion map associated with said surface segment T_3DLVMk of the 3DLVM spatial model. It indicates the capacity of the area of ​​the left ventricle corresponding to this surface segment T_3DLVMk to expand or contract during a change in the volume of the heart.

[0103] In a step E4, a photoplethysmographic signal from said PPG patient is received. This PPG signal, representing variations in blood volume in a blood vessel, may have been acquired in real time by a photoplethysmographic device worn by the patient. Step E4 may be implemented simultaneously with steps E1 to E3 or sequentially, following these steps E1 to E3. It may be envisaged that this step E4, and the steps described subsequently which result from it, are repeated more frequently than steps E1 to E3, so that the spatio-temporal model is generated with each new acquisition of a PPG signal.

[0104] In a step E5, an LVVC function, modeling the change or evolution of the volume of the left ventricle during the patient's cardiac cycle, is determined from the PPG signal. As shown in [Fig. 6], said LVVC function is defined as a function of the relative value of the PPG signal.

[0105] Alternatively, in this step E5, the coefficients al and a2 of the equations of the biplane surface / length method modeling the volume of the left ventricle during the diastole and systole phases may be determined from the PPG photoplethysmographic signal. The coefficient al may, for example, be determined by means of a first machine learning algorithm trained to estimate the value of the coefficient al from the values ​​of a part of a photoplethysmographic signal corresponding to a systole phase and the coefficient a2 can be determined by means of a second machine learning algorithm trained to estimate the value of the coefficient a2 from the values ​​of a part of a photoplethysmographic signal corresponding to a diastole phase.

[0106] The LVVC heart volume change function can thus be determined from the coefficients al and a2 thus estimated and the equations of the biplane surface / length method.

[0107] In an embodiment not shown, the heart volume change function LVVC may be determined by means of an LSTM type neural network receiving as input a signal representing the average of the PPG signal over all the cardiac cycles during which the PPG signal was acquired, as well as a value indicating the volume of the left ventricle at the end of the diastole phase, for example estimated from the 3DLVM spatial model or from an image of the heart acquired at the end of this phase.

[0108] In a step E6, a 4DLVM spatio-temporal model of the left ventricle is generated from the 3DLVM spatial model of the left ventricle, the LVVC function and the WM motion map.

[0109] In the example described, the set of the 3DLVM spatial model of the left ventricle, the LVVC function and the WM motion map makes it possible to generate an instance of a spatio-temporal model of the left ventricle at any time of the cardiac cycle. This set thus forms a 4DLVM spatio-temporal model of the left ventricle. Each new acquisition of a PPG signal thus leads to a new generation of the 4DLVM spatio-temporal model corresponding, in real time or in a short time, to the patient's left ventricle, thus allowing monitoring of the patient's cardiac mechanics.

[0110] In the example described, an instance 4DLVM(t) of a spatial-temporal model of the left ventricle at a time t of the cardiac cycle can be determined, in a step E61, by means of the following equation: [YES] [Math. 1

[0112] 4DLVM) is the instance of the spa model tio-temporal 4DLVM at a time t, 3DLVM is the spatial model of the left ventricle, K a weighting coefficient calculated from the LVVC function, WM the motion map of the left ventricle and ci a weighting coefficient of the motion map.

[0113] In particular, it will be possible to plan to set the value of the coefficient a to a value between 0 and 1.

[0114] The value of the weighting coefficient K may in particular be defined from the ratio between the volume of the 3DLVM spatial model and a volume estimated from said LVVC volume change function at said given time.

[0115] In the example described, the coefficient K is calculated recursively, correcting it using the ratio between the volume of the instant 4DLVM(t) determined during the previous iteration and the volume estimated from said volume change function LVVC at said given instant. Step E61 is thus repeated until the error between the estimated volume from the LVVC function and the volume of the instance the volume of the instant 4DLVM(t) is less than a predetermined threshold value.

[0116] Other methods for determining a 4DLVM(t) instance of a spatial-temporal model of the left ventricle at a time t of the cardiac cycle may be used from the 3DLVM spatial model and the LVVC function, without departing from the scope of the present invention.

[0117] At the end of step E6, the method may include a calculation step, from the 4DLVM spatio-temporal model of the left ventricle, of one or more cardiovascular physiological parameters of the patient, such as the end-diastolic and end-systolic volumes, the systolic ejection volume and the ejection fraction.

[0118] The foregoing description clearly explains how the invention achieves its objectives, namely, to be able to generate a spatio-temporal model of a part of a patient's heart, in a low-cost, reliable manner that can be implemented easily, quickly or in real time, and regardless of the patient concerned. These objectives are achieved by combining, on the one hand, a spatial model of this part of the heart, generated by ultrasound images, and, on the other hand, a volume change function determined from a photoplethysmographic signal.

[0119] In any event, the invention cannot be limited to the embodiments specifically described in this document, and extends in particular to all equivalent means and to any technically effective combination of these means.

Claims

Claims

1. A method for generating a spatio-temporal model (4DLVM) of a portion of a patient's heart, comprising the following steps: a. (E1) Receiving a plurality of ultrasound images (EG) of a patient's heart; b. (E2) Generating a spatial model (3DLVM) of at least a given portion of the heart from the plurality of received ultrasound images; c. (E4) Receiving a photoplethysmographic signal (PPG) of said patient; d. (E5) Determining a heart volume change function (LVVC), during all or part of a cardiac cycle of the patient's heart, from the photoplethysmographic signal; e. (E6) Generating a spatio-temporal model (4DLVM) of said portion of the heart using the spatial model (3DLVM) of said portion of the heart and said heart volume change function (LVVC).

2. Method according to the preceding claim, wherein the step of generating said spatial model of said part of the heart comprises a sub-step of transforming a predetermined initial model (SSM) of a heart to generate a spatial model of said part of the heart corresponding to the plurality of received ultrasound images.

3. Method according to the preceding claim, comprising a prior step (EO) of generating a predetermined initial model (SSM) of a heart, said step comprising the following sub-steps: a. (E01) Receiving a plurality of spatial models (LVTj) of the heart of a plurality of patients; b. (E02) Registrating each of the spatial models in a reference frame (Xj) aligned with a common reference frame (XA); c. (E06) Determining an average model (p) from the set of recalibrated spatial models (IVTIX), the predetermined initial model (SSM) being generated from said average model.

4. Method according to the preceding claim, in which the common reference frame (X) comprises a plurality of predetermined reference points (LM), the prior step (EO) of generating the predetermined initial model (SSM) comprising the following sub-steps: a. (E04) Positioning of each of the predetermined reference points (LM) in each recalibrated spatial model (LVTiA); b. (E07) Statistical analysis of the coordinates of the reference points (LM) of the set of recalibrated spatial models (LVTJA) to generate a statistical representation (P) of the distribution of the coordinates of these reference points among the recalibrated models, the predetermined initial model (SSM) being generated from said average model (p) and said statistical representation (P).

5. Method according to one of claims 2 to 4, in which the step of generating (E2) said spatial model (3DLVM) of said part of the heart comprises, for each image of the plurality of received ultrasound images (EG), a sub-step (E21) of transforming the predetermined initial model (SSM) of a heart to generate a spatial model (3DLVMi) of said part of the heart corresponding to said image, and a sub-step of averaging (E22) the spatial models (3DLVMi) to generate the spatial model (3DLVM) of said part of the heart.

6. Method according to one of the preceding claims, the method comprising a step (E3) of generating, from the spatial model (3DLVM) of said part of the heart, a movement map (WM) of said spatial model, each element (S k(t)) of the map being associated with an area (T_3DLVMk) of the spatial model and indicating the capacity of said area to expand or contract during a change in volume of the heart; said spatio-temporal model of said part of the heart (4DLVM) being generated by means of the spatial model (3DLVM) of said part of the heart, said heart volume change function (LVVC) and the motion map (WM) of said spatial model.

7. Method according to the preceding claim, characterized in that said plurality of ultrasound images (EG) comprises at least one sequence of ultrasound images acquired during a complete cardiac cycle of the patient's heart, and in that the step (E3) of generating the motion map (WM) of said spatial model comprises the following substeps: a. (E31) Splitting the spatial model (3DLVM) of said part of the heart into a plurality of surface segments (T_3DLVMk); b. (E32) For each surface segment, determining, from said sequence of images, at least one value and / or function (Sk(t)) representative of a geometric change of said segment during the complete cardiac cycle; said value and / or function forming an element of the motion map associated with said surface segment of the spatial model.

8. Method according to one of claims 6 or 7, in which the spatio-temporal model (4DLVM) of said part of the heart comprises a morphological function making it possible to generate an instance (4DLVM(t)) of a spatial model of said part of the heart at a given time (t), said instance being determined by an affine transformation of the spatial model of said part of the heart by a weighting element (K) calculated from a value of said volume change function (LVVC) of the heart at said given instant and of the motion map (WM) of said spatial model.

9. Method according to the preceding claim, characterized in that the generation of said instance (4DLVM(t)) is implemented recursively until the error between the volume of said part of the heart determined from said value of said heart volume change function (LVVC) and the volume of the instance is less than a predetermined threshold value.

10. Method according to one of the preceding claims, characterized in that at least a part of the heart volume change function (LVVC) is identical to the relative value of a part of the photoplethysmographic signal (PPG).

11. Method according to one of the preceding claims, characterized in that it comprises a step of calculating, from the spatio-temporal model (4DLVM) of said part of the heart, at least one cardiovascular physiological parameter of the patient.

12. A computer program comprising program code which is designed to implement the method according to one of the preceding claims.

13. Data carrier on which the computer program according to claim 12 is recorded.

14. Computer system for generating a spatiotemporal model of a part of a patient's heart, the computer system comprising a computing unit capable of receiving a plurality of ultrasound images (EG) of a patient's heart and a photoplethysmographic signal (PPG) of said patient, the computing unit being arranged to implement the method according to any one of claims 1 to 11.

Images