Method for estimating body surface area of object and medical imaging equipment
The three-dimensional sample data of the object is obtained through sensor devices and combined with statistical shape models and neural network technology, the problem of insufficient reliability and accuracy of weight or BMI methods in the prior art is solved, and the rapid and reliable body surface area estimation is achieved. In terms of individual patient characteristics, the accuracy of drug scaling and diagnosis quality are improved.
Patent Information
- Application Number
- CN202410498151.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2023-05-30
- Filing Date
- 2024-04-24
- Publication Date
- 2024-12-03
AI Technical Summary
The prior art drug scaling methods based on body weight or BMI are insufficient in reliability and accuracy, and cannot effectively consider individual body size and shape differences, especially for obese patients, which may lead to overdose or poor diagnosis quality.
The three-dimensional sample data of the object is obtained by using sensor devices, and the body surface area is determined through algorithms. Combined with statistical shape models and neural network technology, the body surface area is calculated automatically or semi-automatically, simplifying the process and improving accuracy.
It achieves rapid and reliable estimation of body surface area, reduces the workload of medical staff, is suitable for individual patient characteristics, and improves the accuracy of drug scaling and diagnostic quality.
Smart Images

Figure CN119067912A_ABST
Abstract
Description
Technical Field
[0001] The invention relates to a method for estimating the body surface area of an object, such as a human being or an animal, a corresponding computer program, a body surface estimation system and a corresponding medical imaging device.
[0002] Independent of the use of the grammatical term, individuals with male, female, or other gender identities are also included in the term. Background Art
[0003] Determining the appropriate drug dose for treatment and / or examination for a patient (e.g., for applying a (chemo)therapeutic agent or contrast agent that can be used for medical imaging) is an important factor in medical workflow and patient outcomes. A relatively simple and therefore convenient method is drug scaling based on patient weight. This method is particularly convenient because only the patient's weight is required to determine drug scaling. To this end, weight can be obtained from a hospital system (e.g., a hospital information system (HIS) or a radiology information system (RIS)) or by using a personal scale. Therefore, drug scaling based on patient weight is a common practice in many clinical workflows. However, the disadvantage of this method is that reliability and accuracy are limited. For example, especially when based on information obtained from a hospital system, weight information may be outdated, and the patient's weight may have changed, for example due to disease. In addition, the patient's weight is not directly proportional to the patient's blood volume, which is a decisive factor in drug scaling. That is, this drug scaling based on patient weight does not take into account that the corresponding drug passes through the vascular system, internal organs and muscles to a greater extent than through body fat. Therefore, the use of this weighing-based method may not take into account the differences in body size and shape of different patients. Particularly for obese patients, this reduced contribution to drug dispersion and dilution may result in drug overdose, potentially leading to impaired therapeutic efficacy or increased risk of poor image-based diagnostic quality (e.g., due to artifacts from excessive contrast). An alternative to using patient weight as the basis for drug scaling is to use the patient's Body Mass Index (BMI). However, similar effects may occur when determining drug scaling by using the patient's BMI, as BMI may also be significantly affected by the amount of body fat and, therefore, is generally more of a measure of body fat than a relevant measure of a patient's blood volume.
[0004] It has been found that linear drug scaling based on a patient's body surface area (BSA) may be more accurate and reliable than drug scaling based on a patient's weight or BMI, because BSA tends to have an approximately linear relationship with blood volume. Furthermore, even outside of drug scaling, BSA may be an important variable for diagnosis and treatment planning in medical settings, such as in the case of skin burns.
[0005] However, manual calculation of BSA can be very time-consuming and hinder a smooth workflow. In addition, items that obstruct the patient's view, such as clothing, respirators, or tubes, can make manual determination of BSA more difficult. For example, systematic overestimation is a consequence. This is particularly important in emergency situations, such as those involving trauma. BSA can be estimated using one of a number of different formulas proposed in early and recent literature. Most of these formulas rely on patient variables such as height, weight, and sometimes even sex and age. However, these formulas often lack additional variables that indicate changes in physical or other characteristics, especially regarding individual characteristics of the patient, such as dwarfism, gigantism, anorexia, amputation. In addition, collecting the necessary variables for these formulas can still be a relatively cumbersome task. Summary of the invention
[0006] It is therefore an object of the present invention to provide a means for determining the body surface area in a more automatic and preferably more reliable manner, which means is preferably simultaneously adapted to the characteristics of the individual patient.
[0007] This object is achieved or exceeded by a method according to claim 1 , a computer program according to claim 13 , a body surface estimation system according to claim 14 and a medical imaging device according to claim 15 .
[0008] According to a first aspect of the present invention, a method for estimating the body surface area of an object, particularly a patient, is provided. For example, the object may be a human or an animal. The method comprises the following steps:
[0009] (a) sampling an object using at least one sensor device to obtain sample data of the object, in particular three-dimensional sample data;
[0010] (b) transmitting the sample data to a processing device;
[0011] (c) At the processing device, applying at least one algorithm to the sample data to determine a body surface area of the subject based on the sample data.
[0012] Therefore, the concept of the present invention is to use the sample data of the object acquired by the sensor device to determine the body surface area (BSA) of the object. The sensor device can advantageously be a sensor device that already exists on the clinical side, because the sensor device is usually included in the medical examination room. The sensor can be a three-dimensional (3D) sensor or a two-dimensional (2D) sensor. The 3D sensor is, for example, a 3D camera, while the 2D sensor can be a normal (2D) optical camera. On the other hand, it may be relatively easy and cheap to add such a sensor device to a clinical location. Advantageously, the sampling of the object by the sensor device can be performed automatically or semi-automatically, for example, by manually activating the sensor device, and then automatically collecting and transmitting the sample data. Therefore, the workload of medical staff may be kept at a low level. In addition, in the conventional clinical workflow, BSA can be obtained quickly and reliably. In addition to the steps of the conventional clinical workflow, the object may not even need to perform any additional actions, such as stepping on a scale, taking a specific posture or taking off clothes. Optionally, based on the determined body surface area, additional steps can be added to the method for planning treatment and / or determining drug scaling (e.g., contrast agent scaling). Optionally, the determined body surface area can be transmitted to another device, such as an imaging device, in particular a medical imaging device, a contrast agent injector, a hospital information system and / or a radiology information system. The imaging device can be, for example, a magnetic resonance tomography system, a computer tomography system and / or an ultrasound system.
[0013] For example, the sample data may be transmitted to the processing device via a data line or a wireless connection. The processing device may be or may include, for example, a computer, such as a personal computer, a cloud computer, a server. In addition, and / or alternatively, the processing device may be a processing unit of the imaging system. Advantageously, it has been found that the sample data may be sufficient information for the applied algorithm to determine a reliable and accurate estimate of the BSA. There are different applicable algorithm choices. Corresponding embodiments are given below. Optionally, more than one algorithm may be applied. At least two algorithms may be applied consecutively or simultaneously. For example, it is conceivable that two algorithms are applied simultaneously, and another algorithm is applied afterwards or before. At least two different algorithms may determine the BSA substantially independently of each other. Another algorithm may compare the results of the two different algorithms, or determine the BSA based on the results of the two different algorithms. Advantageously, even higher reliability may be achieved by such a combination of independent algorithms.
[0014] The sample data may include, in particular, optical, acoustic, electromagnetic, topographic and / or tomographic data of the object. Therefore, the sensor device may be, for example, an optical sensor device, an acoustic sensor device, an electromagnetic sensor device or a topographic sensor device. The sensor device is particularly configured to sample the surface of the object (usually a portion of the surface of the object, in particular one side of the object, i.e., the object facing the sensor device). Typically, the object will lie on a patient table, and the sensor device is configured to acquire sample data from above, capturing the upper side or a portion of the upper side of the object. The sensor device may automatically acquire sample data of the object. For example, the sensor device may be configured to automatically acquire sample data at a defined step of a clinical workflow. The defined steps may, for example, include a patient lying on a patient table, such as a patient table of a medical imaging system, or a patient entering an examination room, for example, through a door. Optionally, a plurality of sensor devices may be used to independently acquire multiple sets of three-dimensional sample data. Advantageously, redundant information may be used to minimize the overall error caused by the individual accuracy of the sensor device. In particular, complementary sample data may be fused together. For example, complementary sample data may be sample data measured from different angles of the object.
[0015] According to one embodiment, at least one sensor device comprises an optical sensor, in particular a 3D camera, and / or a radar sensor and / or an acoustic sensor. For example, the optical sensor may be an optical camera, in particular an optical 3D camera, or a laser sampling device based on a laser radar (LiDAR) (light detection and ranging). The acoustic sensor may be based on ultrasound, for example. The 3D camera may advantageously allow direct acquisition of three-dimensional sample data, in particular using a fixed sensor device. Sampling based on LiDAR may be performed by sending light pulses and detecting signals reflected from an object. The running time of the light may be used to calculate the distance to each part of the object from which the light is reflected. The light pulse may be, for example, a laser pulse or an infrared light pulse. Preferably, based on the calculated distance, the shape of the object is determined. The optical sensor may comprise a structured light scanner and may be adapted to project the structured light onto the object. The structured light may in particular be a structured pattern, such as a grid, and the sensor device may be adapted to project the structured pattern onto the object. The structured light scanner may be configured to measure the distortion of the structured pattern projected on the object. The positions of different surface points of the object may be calculated from the structured pattern measured on the object, in particular in order to determine the shape of the object.
[0016] According to one embodiment, three-dimensional sample data is acquired by moving the sensor device or the object, by triangulation or by applying a time-of-flight measurement of the object by the sensor device. In particular, the sensor device may comprise at least one two-dimensional sensor. The at least one two-dimensional sensor may be adapted to acquire three-dimensional sample data by moving to different positions relative to the object and to acquire position-dependent sample data of the object. The position-dependent sample data may be used to determine the three-dimensional sample data. In order to perform the triangulation, at least two sensor devices, in particular two-dimensional sensor devices, may be provided. The time-of-flight measurement may be based, for example, on a radar (radio detection and ranging) or a lidar. The time-of-flight measurement may be based on sending radiation, in particular infrared light, and calculating a phase difference of the reflected radiation.
[0017] According to one embodiment, a medical imaging device, in particular an X-ray-based medical imaging device, is used as a sensor device that provides medical image data as sample data or a part of sample data, in particular so that a pre-scan performed by the medical imaging device is used as medical image data. The pre-scan can be, for example, a topogram acquired before the main examination scan. The pre-scan can be a preliminary non-contrast agent scan. The X-ray-based imaging device can be, in particular, a computed tomography scanner. The sample data can be obtained, for example, from a topographic X-ray scan or a planar X-ray scan (for example, in the lateral and / or anterior-posterior directions). Advantageously, data that is to be measured in any case in the medical workflow can be used for the sample data. The acquisition of sample data can be integrated into the medical workflow in this way in a basically seamless manner. Optionally, sample data from the medical imaging device can be used to support sample data acquired with another sensor device. Therefore, sample data from the medical imaging device and another sensor device can be used as redundant information to minimize the overall error of the data. Sample data from the medical imaging device can be used to adjust sample data acquired with another sensor device.
[0018] According to one embodiment, determining the body surface area of the subject comprises:
[0019] - creating a virtual avatar model of the object surface based on the sample data and the statistical shape model, such that the shape and pose of the avatar model is a fit to the sample data, wherein the fit complies with boundary conditions defined by the statistical shape model,
[0020] wherein the statistical shape model comprises a database of a series of standard object shapes and poses, the series of standard object shapes and poses defining a series of shapes and poses that the avatar model must conform to;
[0021] - Calculating the surface area of the avatar model and using the calculated surface area of the avatar model as an estimate of the body surface area.
[0022] The avatar model is in particular a model that describes the surface of the object. The algorithm applied can be configured in particular to determine the avatar model so that the avatar model presents both the posture and body proportions of the observed object. In particular, the avatar model is determined so that it complies with the statistical shape model. The statistical shape model can preferably be based on a database of body variations of real objects (in particular real people or animals). For example, the avatar model can be determined as described in Singh, Vivek & Ma, Kai & Tamersoy, Birgi & Chang, Yao-Jen & Wimmer, Andreas & O'Donnell, Thomas and Chen, Terrence in "DARWIN: Deformable Patient Avatar Representation With Deep Image Network", 497-504, 10.1007 / 978-3-319-66185-8_56; 2017. The statistical shape model can preferably include shapes and positions containing multiple characteristic object variations. The characteristic object variations may include individual attributes of different objects, such as the size of the object. Individual attributes may preferably include extreme attributes, for example including being particularly small or large, being particularly thin or overweight and / or missing limbs. It has been found that applying the avatar model as an intermediate step to calculate the surface area can lead to accurate and reliable results. In addition, the applied algorithm may require only a relatively small amount of training data to achieve useful results for surface area determination, compared to a trained algorithm trained to determine the surface area directly. In addition, it has been found that this allows for rapid determination of the surface area, in particular allowing it to be integrated into a clinical workflow, without generally causing any detrimental delays in the workflow.
[0023] According to one embodiment, creating a virtual avatar model comprises assigning labels to at least some characteristic parts of an object in the sample data, in particular by applying a first trained neural network, wherein the avatar model is created by mapping the avatar model to the labels, in particular by applying a second trained neural network. The characteristic parts may include defined parts of the head (e.g. the top and / or bottom of the head), shoulders, elbows, wrists, torso (e.g. the center of the torso), groin, knees and ankles. For example, 10 to 20 labels may be used. The first and / or second trained neural network may preferably be a convolutional neural network. The first / second neural network can be configured and trained in particular as described in Singh, Vivek & Ma, Kai & Tamersoy, Birgi & Chang, Yao-Jen & Wimmer, Andreas & O'Donnell, Thomas and Chen, Terrence, "DARWIN: Deformable Patient Avatar Representation With Deep Image Network", 497-504, 10.1007 / 978-3-319-66185-8_56; 2017.
[0024] According to one embodiment, a first trained neural network is trained to assign a label despite the object being at least partially covered by a covering element, in particular by clothing and / or other objects; wherein the first trained neural network is particularly trained such that the applied training data comprises at least some input sample data in which the object is at least partially covered by a covering element. The covering element may in particular comprise clothing and / or a blanket. The covering element may also comprise objects such as an oxygen tank or a component of a medical imaging device. Thus, advantageously, this embodiment may allow a realistic estimation of the surface area of the object without the need for the object to undress. Existing medical workflows may advantageously not need to be adapted, for example by removing a blanket or undressing. This may save time in the workflow. Furthermore, not needing to undress may be particularly advantageous for people or patients who are ill and / or weak and may therefore have difficulty undressing. The first trained neural network may be trained with a data set containing different objects with different coverings. The training data may comprise sample data of objects in different positions. The training data may preferably comprise sample data with and without a covering, in particular such that the training data comprises sample data in which the same object in the same pose comprises with and without a covering.
[0025] According to one embodiment, after mapping the avatar model to the markers and before calculating the surface area, a model fitting algorithm is applied so that the avatar model is adapted to better fit the sample data within the boundaries defined by the statistical shape model, wherein the statistical shape model particularly includes degrees of freedom that allow adjustment of the joints of the object and the current orientation of the joints. Therefore, preferably, three algorithms or an algorithm with three parts can be applied. The first algorithm or the first part of the algorithm can define the markers, the second algorithm or the second part of the algorithm can perform the mapping of the avatar model to the markers, and the third algorithm or the third part of the algorithm (i.e., the model fitting algorithm) can adjust the avatar model. Optionally, the model fitting algorithm can be configured to comply with the degrees of freedom in which the joints of the avatar model can be adjusted. The joints of the avatar model can, for example, include one, more than one or all of the shoulders, elbows, wrists, knees and ankles of the object.
[0026] According to one embodiment, the avatar model is a three-dimensional geometric mesh of a body surface or is represented by a three-dimensional geometric mesh of a body surface, in particular a triangular mesh. The mesh is very suitable for calculating the surface. For example, the coordinates of individual elements of the mesh (in particular the corner points) can be used to calculate the area of the individual elements. The areas of the individual elements can be added up in order to calculate the entire surface area of the object. The elements of the mesh can in particular be cells of the mesh. It has been found that triangular meshes can be particularly advantageous for reliably and accurately calculating surface areas. For example, the mesh can include N number of triangles Ti, each triangle being defined by three corner points vi,1,vi,2,vi,3. The corner points can be vectors defining the positions of the corner points. The area of a mesh element can be calculated by applying the vector product of the coordinates defining the element. BSA can be calculated based on the area of the element, in particular by adding the areas of the elements. For example, the individual area of triangle Ti can be calculated by the following formula
[0027] A i =0,5(v i,2 –v i,1 )×(v i,3 –v i,1 ),
[0028] × is the vector product of the vectors. Therefore, the surface area BSA of the avatar model can be determined by the following calculation.
[0029]
[0030] According to one embodiment, applying at least one algorithm includes applying a convolutional neural network (CNN) that is trained to derive an estimate of body surface area based on sample data. This embodiment is particularly useful when a large amount of training data is available. For example, when a training database with an amount of data from objects in the order of hundreds of thousands is available, this embodiment can provide particularly reliable and robust results. Therefore, for this embodiment, it is not necessary to determine an avatar model, but the surface area can be determined directly from the sample data. The CNN can be a residual convolutional neural network. For example, the CNN can include a series of residual blocks, each of which consists of a series of convolution blocks. Each convolution block can consist of the following: a series of convolution operators, followed by an activation function, followed by a normalization function. Normalization layers can ensure efficient training. Residual connections between convolution blocks can ensure sufficient gradient flow in the network. Each residual block can be followed by an average pooling operator. The CNN can preferably include a fully connected layer that performs body surface regression and produces a single output in the form of body surface area. The output surface area can be a z-normalized BSA estimate.
[0031] CNN can be trained through supervised training. CNN can be trained by using paired data tuples, such as<x,y> . The data tuples may each include an input image (e.g., x) and a target variable (e.g., y). The input image may be obtained from a sensor device (such as an optical or acoustic sensor). The target variable is, in particular, a surface area (BSA). Therefore, the target variable may be the actual ground truth of the training process. The BSA used for training may be measured or estimated by a given formula, for example, based on the patient's weight and height. Alternatively, the BSA of the target variable may be estimated based on the object morphology. The object morphology may provide an accurate measurement of the patient's surface. Alternatively, the target variable may be derived based on any embodiment using an avatar model. Therefore, embodiments of avatar models that require less training data may be used to establish a training database for this embodiment. Some or all of the alternatives described herein for obtaining the target variable may be combined, for example, in order to obtain a larger amount of training data in different ways. Advantageously, it has been found that during training, CNN may automatically derive relevant criteria, such as those defined by the statistical shape model of other embodiments. Given a sufficiently large amount of training data, CNN may be able to outperform the quality of the training data, particularly by generalizing BSA factors other than patient weight and height, such as less common patient geometry or limb loss.
[0032] According to one embodiment, before applying the convolutional neural network, an orthographic projection of the sample data is generated, and the convolutional neural network is applied to the orthographic projection. It has been found that the orthographic projection can lead to more reliable results, because the input to the CNN can therefore be independent of the specific positioning of the sensor device, in particular with respect to the distance between the sensor device and the object. Therefore, by applying the orthographic projection, the input data to the CNN can be standardized, so that the CNN may have to process a smaller range of different data. Preferably, the orthographic projection is always performed from substantially the same viewing angle as the object.
[0033] According to one embodiment, the body surface area is calculated based on an estimate by a convolutional neural network and an estimate derived by an avatar model, in particular by determining an average or weighted average of the two estimates. Preferably, the two estimation methods can each be independently applied to the same sample data, and the results can then be combined. This combination may lead to more accurate and reliable results. The weighted average can be particularly applied when it is found that one of the two estimates tends to be more accurate. In this case, the weights may be such that a greater weight is placed on the more accurate alternative. The weighting may be determined by other factors. For example, the weighting may depend on the object attributes. This may be advantageous, for example, when one of the independent estimates is more reliable for some specific attributes but less reliable for other specific attributes.
[0034] According to one embodiment, an estimate of the body surface area derived from the convolutional neural network is compared with an estimate of the body surface area derived by the avatar model, wherein, in case the difference in the estimated body surface areas is found to be greater than a predefined threshold, a warning is sent to the user. Advantageously, two independently derived BSAs can be used for a plausibility check. Thus, when the difference between the two estimated BSAs is greater than a threshold, it can be assumed that at least one of the estimates is wrong. Therefore, it may be advantageous to repeat the entire method, in particular using new sample data or to discard one of the estimates.
[0035] According to another aspect of the invention, a computer program comprising instructions, when the program is executed on a system comprising a processing device and at least one sensor, the instructions cause the system to perform the steps of the method described herein. All features and advantages of the method are applicable to the computer program and vice versa.
[0036] According to another aspect of the present invention, a body surface estimation system is provided. The body surface estimation system comprises at least one sensor device, in particular a sensor device as described herein. The body surface estimation system further comprises a processing device configured to receive sensor signals from the at least one sensor device. Optionally, the body surface estimation system may comprise an output device configured to output derived information from the processing device. The processing device is configured to apply at least one algorithm to determine the body surface area of an object based on sample data. In particular, the body surface estimation system may be configured to perform the method steps of the method for estimating the body surface area of an object as described herein. All features and advantages of the method and the body surface estimation system may be applicable to a computer program, and vice versa.
[0037] According to another aspect of the invention, there is provided a medical imaging device comprising a body surface estimation system as described herein.All features and advantages of the method, body surface estimation system and computer program are applicable to the medical imaging device and vice versa.
[0038] Unless otherwise stated, the embodiments described herein can be combined with each other. BRIEF DESCRIPTION OF THE DRAWINGS
[0039] The drawings illustrate various exemplary embodiments and methods of various aspects of the invention.
[0040] Figure 1 A flow chart showing a method for estimating the body surface area of an object according to an embodiment of the present invention is shown;
[0041] Figure 2 A flow chart of a method for estimating the body surface area of an object according to another embodiment of the present invention is shown;
[0042] Figure 3 A flow chart of a method for estimating the body surface area of an object according to another embodiment of the present invention is shown;
[0043] Figure 4 A flow chart of a method for estimating the body surface area of an object according to another embodiment of the present invention is shown;
[0044] Figure 5 shows a surface image reconstructed from three-dimensional sample data sampled with a 3D camera;
[0045] Figure 6 An avatar model fitted to sample data according to a method of an embodiment of the present invention is shown;
[0046] Figure 7 yes Figure 6 Magnified view of the area marked by a dashed box;
[0047] Figure 8 shows an avatar model and a partial reconstruction of sample data according to one embodiment of the present invention;
[0048] Fig. 9 The network architecture of a residual convolutional neural network according to an embodiment of the present invention is shown; and
[0049] Fig.10 A medical imaging apparatus according to an embodiment of the present invention is shown, comprising a body surface estimation system for estimating a body surface area of a subject. DETAILED DESCRIPTION
[0050] Figure 1 A flow chart of a method for estimating a body surface area of an object 5 according to an embodiment of the present invention is shown. In a first step 101, the object 5 is sampled using at least one sensor device 2 to obtain three-dimensional sample data 20 of the object 5. In this embodiment and any other embodiment, the sensor device 2 may include an optical sensor, such as a 3D camera and / or an acoustic sensor. Optionally, the sample data 20 may be obtained by triangulation, by moving the sensor device 2 or by applying a time-of-flight measurement of the object 5 by the sensor device 2. Additionally or alternatively, the sample data 20 or some of the sample data 20 may be provided by a medical imaging device 1. Thus, the medical imaging device 1 may be used as the sensor device 2. For example, a pre-scan performed by the medical imaging device 1 may be used as the sample data 20. In a further step 102, the sample data 20 is transmitted to a processing device 3. The processing device 3 may, for example, be part of a control unit of the medical imaging device 1. In a further step 103, at least one algorithm is applied on the processing device 3 to determine the body surface area of the object 5 based on the sample data 20.
[0051] Figure 2A flow chart of a method for estimating the body surface area of an object 5 according to another embodiment of the present invention is shown. In a first step 201, the object 5 is sampled using at least one sensor device 2 to obtain three-dimensional sample data 20 of the object 5. In a further step 202, the sample data 20 is transmitted to a processing device 3. A further step 203 of applying at least one algorithm to determine the body surface area of the object 5 based on the sample data 20 includes two sub-steps 204, 205. That is, the first sub-step 204 includes creating a virtual avatar model of the object surface based on the sample data 20 and a statistical shape model. The statistical shape model includes a database of shapes and postures of a series of standard objects 5 that define a series of shapes and postures that the avatar model must comply with. Accordingly, the avatar model is created so that the shape and posture of the avatar model are a fit to the sample data 20 that complies with the boundary conditions defined by the statistical shape model. Preferably, the avatar model can be a three-dimensional geometric mesh 11 of the body surface. Optionally, the first sub-step 204 can include consecutive steps 2041 to 2043. The first of the consecutive steps 2041 to 2043 comprises assigning labels to at least some characteristic parts of the object 5 in the sample data 20. A further consecutive step 2042 comprises mapping the avatar model to the labels so as to create the avatar model. Preferably, both steps 2041 and 2042 can be performed by applying a neural network, in particular a convolutional neural network. Preferably, the trained neural network for the step 2041 of assigning labels is trained to assign labels even though the object 5 is at least partially covered by covering elements, such as clothes, blankets and / or other objects. In another consecutive step 2043, performed after step 2042, a model fitting algorithm is applied so that the avatar model is adapted to better fit the sample data 20 within the boundaries defined by the statistical shape model. To this end, the statistical shape model may include degrees of freedom that allow the joints of the object 5 and the current orientation of the joints to be adjusted. In a further step 205, the surface area of the avatar model, in particular the adapted avatar model, is calculated, and the calculated surface area of the avatar model is used as an estimate of the body surface area.
[0052] Figure 3A flow chart of a method for estimating a body surface area of an object 5 according to an embodiment of the invention is shown. In a first step 301, the object 5 is sampled with at least one sensor device 2 to obtain three-dimensional sample data 20 of the object 5. In a further step 302, the sample data 20 are transmitted to a processing device 3. On the processing device 3, at least one algorithm is applied 303 in order to determine the body surface area of the object 5 based on the sample data 20. The step of applying the algorithm comprises or consists of a step 306 of applying a convolutional neural network, which is trained to derive an estimate of the body surface area based on the sample data 20. In an optional step 3021, before the step 306 of applying the convolutional neural network, a positive projection of the sample data 20 may be generated, and during the step 306, the convolutional neural network may be applied to the positive projection.
[0053] Figure 4 A flow chart of a method for estimating the body surface area of an object 5 according to an embodiment of the present invention is shown. In a first step 401, the object 5 is sampled using at least one sensor device 2 to obtain three-dimensional sample data 20 of the object 5. In a further step 402, the sample data 20 is transmitted to a processing device 3. Optionally, in this step 402, the sample data 20 may be transmitted to two different processing devices 3, in particular when the following steps 404 and 406 are performed on different processing devices 3. On the processing device 3 or two different processing devices 3, in a further step 403, at least two algorithms are applied to independently determine the body surface area of the object 5 based on the sample data 20. In a step 406, a convolutional neural network is applied, which is trained to derive an estimate of the body surface area based on the sample data 20. This step may in particular be performed as follows: Figure 3 The execution, for example, optionally includes a positive projection of the sample data 20. In a further step 404, an avatar model of the object surface based on the sample data 20 and the statistical shape model is created. The statistical shape model includes a database of a series of standard object shapes and postures, which define a series of shapes and postures that the avatar model must comply with. Accordingly, the avatar model is created so that the shape and posture of the avatar model are a fit to the sample data 20 complying with the boundary conditions defined by the statistical shape model. In a further step 405, the surface area of the avatar model, in particular the adapted avatar model, is calculated, and the calculated surface area of the avatar model is used as an estimate of the body surface area. These steps 404 and 405 can in particular be as described with respect to Figure 2 In a further step 407, the body surface area is calculated based on the estimates of the routes of steps 404 and 405 and the estimate of step 406. In particular, this step 407 may comprise determining an average or a weighted average of the two estimates.
[0054] Figure 5 1 shows a surface image reconstructed from three-dimensional sample data 20 sampled with a 3D camera. In this image, it can be seen that the subject 5 is lying on the patient table 13. Figure 6 In the embodiment, an avatar model is fitted to sample data 20 using a method according to an embodiment of the present invention. The avatar model includes a three-dimensional geometric mesh 11 of a body surface. Figure 7 Shown in Figure 6 1 is an enlarged view of the area marked by the dashed box 12 in FIG. As can be seen, the geometric mesh 11 is a triangular mesh. The area of an individual triangle can be calculated by the vector product of the triangle angle coordinates. The entire body surface area can then be calculated by adding the calculated areas of the individual triangles. Figure 8 An avatar model is shown together with a partial reconstruction of the sample data 20. It can be seen that the avatar model is fitted to the subject's body so that the subject's 5 clothing is actually placed on top of the avatar model. This shows that the avatar model determined by an embodiment of the present invention is not a fit to the outer clothing covering the subject 5, but a fit to the surface of the actual subject 5. Therefore, it can be seen that the method according to the present invention can accurately calculate the surface area of the subject 5 even if the subject 5 is partially covered by clothing. In addition, the avatar model follows the surface of the subject 5 above the patient table 13, so the patient table 13 can be subtracted from the determination of the body surface area.
[0055] Fig. 9 The network architecture of a residual convolutional neural network according to one embodiment of the present invention is shown. The convolutional neural network is configured to determine the body surface area of an object 5 without calculating a virtual avatar model based on three-dimensional sample data 20 of the object 5. The network architecture consists of a series of N residual blocks 21 with K sequential convolution blocks 22, and the K sequential convolution blocks 22 have residual connections 23. The residual connections 23 between the convolution blocks 22 ensure that there is sufficient gradient flow in the network. Each convolution block can be composed as follows: a series of convolution operators, followed by an activation function, followed by a normalization function. After each residual block, an average pooling operator 23 is applied (shown here only after the first residual block 23). The average pooling operator 23 reduces the image by half in each dimension (represented by the smaller residual block 21 after the average pooling operator 23). Finally, a fully connected layer 25 is attached to produce a BSA estimate 26 of the sample data 20 of the object 5.
[0056] Fig.10A medical imaging device 1 according to an embodiment of the invention is shown, comprising a body surface estimation system for estimating a body surface area of a subject 5. The system comprises a sensor device 2, such as a 3D camera. The sensor device 2 is in particular configured to detect sample data 20 from the subject 5 lying on a patient table 13. The system further comprises a processing device 3, which is configured to receive sensor signals from the sensor device 2, in particular sensor signals transmitting the sample data 20. The processing device 3 is configured to apply at least one algorithm in order to determine / estimate the body surface area of the subject 5 based on the sample data 20. The estimation of the body surface area may be performed, for example, according to Figures 1 to 4 The system may further include an output device 4 configured to output information derived by the processing device 3.
Claims
1. A method for estimating the body surface area of an object (5), in particular a patient, comprising the following steps: (a) sampling the object (5) using at least one sensor device (2) to obtain sample data (20) of the object (5), in particular three-dimensional sample data (20); (b) transmitting the sample data (20) to a processing device (3); (c) applying, on the processing device (3), at least one algorithm to the sample data in order to determine the body surface area of the object (5) based on the sample data (20).
2. The method according to claim 1, wherein determining the body surface area of the object (5) comprises: - creating a virtual avatar model of the surface of the object based on the sample data (20) and the statistical shape model, such that the shape and pose of the avatar model are a fit to the sample data (20), wherein the fit complies with boundary conditions defined by the statistical shape model, wherein the statistical shape model comprises a database of a series of standard object shapes and poses, wherein the series of standard object shapes and poses define a series of shapes and poses to which the avatar model must comply; - calculating a surface area of the avatar model and using the calculated surface area of the avatar model as an estimate of the body surface area.
3. The method according to claim 2, wherein creating the virtual avatar model comprises assigning labels to at least some characteristic parts of the object (5) in the sample data (20), in particular by applying a first trained neural network, The avatar model is created by mapping the avatar model to the markers, in particular by applying a second trained neural network.
4. The method according to claim 3, wherein the first trained neural network is trained to: assign a label despite the fact that the object (5) is at least partially covered by covering elements, in particular clothing and / or other objects; The first trained neural network is specifically trained such that the applied training data comprises at least some input sample data (20) in which the object (5) is at least partially covered by the covering element.
5. A method according to claim 3 or 4, wherein after mapping the avatar model to the markers and before calculating the surface area, a model fitting algorithm is applied so that the avatar model is adapted to better fit the sample data (20) within the boundaries defined by the statistical shape model, The statistical shape model comprises in particular degrees of freedom allowing adjustment of the joints of the object (5) and the current orientation of the joints.
6. Method according to any one of claims 2 to 5, wherein the avatar model is a three-dimensional geometric mesh (11), in particular a triangular mesh, of a body surface.
7. A method according to any of the preceding claims, wherein applying at least one algorithm comprises applying a convolutional neural network trained to derive an estimate of the body surface area based on the sample data (20).
8. The method according to claim 7, wherein before applying the convolutional neural network, a positive projection of the sample data (20) is generated and the convolutional neural network is applied to the positive projection.
9. The method according to claim 7 or 8 and any one of claims 2 to 6, Therein the body surface area is calculated based on both an estimate of the convolutional neural network and an estimate derived via the avatar model, in particular by determining an average or a weighted average of these two estimates.
10. The method according to any of the preceding claims, wherein the at least one sensor device (2) comprises an optical sensor, in particular a 3D camera, and / or an acoustic sensor.
11. A method according to any of the preceding claims, wherein the three-dimensional sample data (20) are acquired by triangulation, by moving the sensor device (2) or the object (5), or by the sensor device (2) applying a time-of-flight measurement of the object (5).
12. A method according to any of the preceding claims, wherein a medical imaging device (1), in particular an X-ray based medical imaging device (1), is used as a sensor device (2), the sensor device (2) providing medical image data as sample data (20) or part of the sample data (20), in particular so that a pre-scan performed by the medical imaging device (1) is used as medical image data.
13. A computer program comprising instructions which, when said program is executed on a system comprising a processing device (3) and at least one sensor, cause said system to perform the steps of the method according to any one of claims 1 to 12.
14. A body surface estimation system, comprising: at least one sensor device (2), in particular a sensor device (2) according to any one of claims 10 to 12, a processing device (3) configured to receive a sensor signal from the at least one sensor device (2), and Optionally, an output device, configured to output the derived information from the processing device (3), wherein the processing device (3) is configured to apply at least one algorithm in order to determine the body surface area of the object (5) based on the sample data (20), The system is particularly configured to perform the steps of the method according to any one of claims 1 to 12.
15. A medical imaging device (1) comprising a body surface estimation system according to claim 14.
