Patient weight estimation using patient model from surface data

Abstract

For patient weight estimation in a medical imaging system, a patient model, such as a grid, is fitted to a depth image. One or more feature values are extracted from the fitted patient model, thereby reducing noise and clutter in the values. The weight estimation is regressed based on the extracted features.

Description

Background Art

[0001] This embodiment relates to patient weight estimation for dosing patients. Weight is important information when adjusting drug or radiation doses in a clinical setting. For example, without weight-based adjustments, patients are exposed to 17-43% higher radiation doses from chest computed tomography (CT) scans.

[0002] Accurate weight information is not always available. In a study for emergency stroke treatment, 14.6% of patients were weighed, with the weight of the remainder of the patients guessed visually by the physician or caregiver. Health care workers have only moderate accuracy in estimating patient weight. The lack of accurate weight information is due to several complex reasons. For example, it is impossible to use the common standing weighing method for patients with serious conditions (such as trauma or stroke). In addition, the time requirements for emergency treatment may not allow for weighing.

[0003] Anthropometric measurements can be used to estimate human weight. In one approach, anthropometric features are manually measured from images to learn their correlation with weight through regression. In a more automated approach, features are extracted directly from depth images. Direct feature extraction from depth images can be problematic when the scene is cluttered and noisy. Summary of the Invention

[0004] Systems, methods, and instructions on a computer-readable medium are provided for patient weight estimation in a medical imaging system. A patient model (such as a mesh) is fitted to a depth image. One or more feature values ​​are extracted from the fitted patient model to reduce noise and clutter in the values. A weight estimate is regressed based on the extracted features.

[0005] In a first aspect, a method for estimating patient weight based on surface data in a medical imaging system is provided. The surface data is data representing the surface or exterior of a patient, such as by capturing the patient's exterior surface using a sensor. A patient model is fitted to the surface data. The patient model is a mesh, a statistical shape model, or other general parameterization of the exterior surface. The fitting deforms the patient model to fit the surface data of a specific patient. Values ​​for each of one or more features are extracted from the fitted patient model. Features are shape characteristics or other characterizations of the patient model. The patient's weight is estimated by inputting one or more values ​​for the one or more features into a machine-learned regressor. The patient is dosed based on the weight.

[0006] The sensor can be a depth sensor or a camera, where the surface data is based on optical measurements. The patient model can be fitted to the surface data as a statistical shape model. Dosing can apply radiation to the patient, where the amount (e.g., frequency, amplitude level, number of applications, or other dosing levels) is based on body weight.

[0007] In one embodiment, the shape feature is the body volume indicated by the fitted patient model. In other embodiments, one or more features are used for each of the different body parts. Weights are estimated for the different body parts, such as estimating partial weights for the different body parts and summing the partial weights. In another embodiment, the values ​​extracted from the fitted patient model are shape manifold coefficients. The coefficients are parameterizations of the shape, such as by a mathematical manifold based on the fitted mesh.

[0008] The estimation uses any learned regressor, such as a machine-learned regressor of a fully convolutional network. Machine-learned or other linear regressors can be used.

[0009] In a second aspect, a medical imaging system is provided for patient weight estimation. A depth sensor is configured to measure the depth of the patient. An image processor is configured to form a grid for the patient using the depth and regress the patient's weight based on the grid formed for the patient. The weight can be regressed using a machine-learned regressor. A memory is configured to store the regressed weight, such as for use in determining or controlling dose.

[0010] In one embodiment, the depth sensor is a depth camera. After fitting the grid, the image processor is configured to perform regression in response to input to the machine-learned regressor. The therapeutic radiation scanner is configured to administer a radiation dose to the patient, wherein the amount of the radiation dose is based on body weight.

[0011] The image processor can be configured to extract values ​​for shape features from a mesh such as that formed for the patient and regress weight based on the values. The shape features can be volumes for the entire mesh or for a portion of the mesh. The image processor can be configured to extract values ​​for shape manifolds from a mesh such as that formed for the patient and regress weight based on the values.

[0012] In a third aspect, a method is provided for estimating patient weight in a medical imaging system. A sensor captures an outer surface of a patient. Shape feature values ​​are extracted from a patient model fitted to the outer surface captured by the sensor. The patient's weight is estimated by regression based on the shape feature values.

[0013] In one embodiment, the estimation is performed via a machine-learned regressor. A dosage is assigned to the patient, wherein the amount of the dosage is based on the estimated body weight.

[0014] Any one or more of the above aspects may be used alone or in combination. These and other aspects, features and advantages will become apparent from the following detailed description of the preferred embodiments, which will be read in conjunction with the accompanying drawings. The present invention is defined by the following claims, and nothing in this section should be construed as limiting those claims. Further aspects and advantages of the present invention are discussed below in conjunction with the preferred embodiments and may be claimed later, independently or in combination. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] The components and the figures are not necessarily to scale, emphasis instead being placed upon illustrating the principles of the embodiments. Moreover, in the figures, like reference numerals designate corresponding parts throughout the different views.

[0016] Figure 1 is a flowchart illustration of one embodiment of a method for patient weight estimation from surface data in a medical imaging system;

[0017] Figure 2 illustrates an exemplary workflow for weight estimation using a fitted patient model;

[0018] Figure 3 is an exemplary depth image;

[0019] Figure 4 An exemplary fitted patient model is shown;

[0020] Figure 5 is a graph showing the correlation between weight and volume;

[0021] Figure 6 is a graph showing predicted weight versus actual weight based on volume;

[0022] Figure 7 is a block diagram of one embodiment of a system for patient weight estimation. DETAILED DESCRIPTION

[0023] Patient weight is estimated from a depth image using a three-dimensional (3D) model of the patient. To overcome the challenges caused by noise or clutter, regression-based weight estimation from the patient's depth image uses a 3D surface model of the patient. Instead of capturing shape information directly from the cluttered depth image of the scene, using a fitted patient model as an intermediate layer to represent the patient's body provides a more robust weight estimation. Weight estimation regression is based on an input depth image and a fitted patient model. Various reliable human shape features (such as shape volume, shape manifold, etc.) can be extracted from the fitted model. The extracted features can then be used to estimate the patient's weight through a regression method. The correlation of the patient's weight with the captured image is precisely formulated as the correlation between the patient's weight and the shape features extracted from the fitted patient model.

[0024] In one embodiment, a depth image is input to a weight estimation framework. A patient model, represented by a human surface mesh, is then extracted from the input. The human surface mesh is obtained by fitting a template human surface mesh to the current patient input. The fitted patient model captures an accurate estimate of the patient's shape, from which various relevant body measurements can be extracted to estimate human weight.

[0025] Because depth images captured from real-world environments are often noisy, directly extracting patient body features from depth images is error-prone. Instead, a fitted patient model captures the patient's shape information for weight estimation without being affected by noise from surrounding objects. The correlation between the extracted shape features and the patient's weight is learned using a regression method to robustly estimate the patient's weight.

[0026] Robust weight estimation can be used in many clinical applications. For example, when a patient is on a medical scanner (e.g., a diagnostic imager and / or therapeutic scanner), accurate weight can be estimated based on depth sensor images. Weight can be estimated regardless of the severity of the patient's condition. The estimated weight can then help in the decision-making process for dose control, which is an important factor for both scan quality and patient health.

[0027] In one embodiment, the shape features are coefficients of a shape manifold. More complex features can be used, such as features for body part information, from which different body part shape information is used to more robustly estimate weight. The framework can also be extended for other applications, such as estimating patient height or other characteristics.

[0028] Figure 1The present invention is a flowchart illustrating an embodiment of a method for estimating the weight of a patient in a medical imaging system. The weight estimation is based on imaging the external surface of the patient. The medical imaging system uses an imaging sensor to obtain surface data (e.g., the external surface) of the patient.

[0029] Figure 2 Shown Figure 1 FIG2 is an exemplary representation of a method for measuring body weight. The method takes a depth image 20 as input and fits a (personalized) patient surface model 22 based on the depth image. The personalized patient surface model 22 is then used to extract various features 24, which are used to estimate 16 body weight 26 through regression. Based on fitting the surface model 22 to data from the sensor, the body weight is estimated based on the characteristics of the fitted surface model 22.

[0030] Figure 1 The method is performed by a medical imaging system. A sensor (such as a depth camera) captures the patient's surface. An image processor fits a model, extracts shape features, and estimates the patient's weight. The image processor, a physician, and / or a scanner (e.g., an x-ray machine) sets a dose based on the weight. Other devices may be used to perform any of the steps.

[0031] The method is performed in the order shown (e.g., top to bottom or numerical), but other orders may be used. Additional, different, or fewer actions may be provided. For example, action 18 is not provided.

[0032] In act 10, a sensor captures the exterior surface of the patient. The sensor is a depth sensor, such as a 2.5D or RGBD sensor (e.g., Microsoft Kinect 2 or ASUS Xtion Pro). The depth sensor can measure depth directly, such as by using time of flight, interferometry, or a coded aperture. The depth sensor can be one or more cameras that capture a grid projected onto the patient. The sensor can be multiple cameras that capture 2D images from different directions, allowing the exterior surface to be reconstructed from multiple images without transmitting structured light. Other optical or non-ionizing sensors can be used.

[0033] The sensor is directed toward the patient. The sensor captures the patient's exterior surface from one or more viewing angles. Any portion of the exterior surface can be captured, such as the entire patient viewed from one side, head to toe and hand to hand, or just the torso. The sensor captures the exterior surface while the patient is in a specific position, such as capturing the front-facing surface while the patient is lying in a bed or on a table for treatment or imaging.

[0034] The outer surface is the patient's skin. In other embodiments, the outer surface comprises clothing. The sensor can use frequencies that pass through clothing and detect the skin surface. Alternatively, the outer surface is clothing, and the patient model is fitted taking clothing into account.

[0035] The exterior surface is captured as: depth from the sensor to different locations on the patient, an image or photograph of the patient's exterior, or both. The sensor outputs the sensed image and / or depth. The measurements of the exterior surface from the sensor are surface data for the patient. Figure 3 An exemplary image 20 from surface data is shown where intensity in grayscale is mapped to sensed depth. Alternatively, sensor measurements are processed to determine exterior surface information, such as using image processing to stereoscopically determine the exterior surface from camera images from different angles.

[0036] In one embodiment, the surface data can include different representations of the patient. Two or more channels are created. For example, two images have pixel intensities modulated by the amplitude of the information for each channel (e.g., one by depth, and the other by color). In one embodiment, given a 3D surface of the patient's body (the skin surface), 2D projections of this data—a skin surface image (e.g., the height of the surface from the scanner table at each location in the image) and a depth image (e.g., a measurement of the thickness of the person at each location in the image)—are formed through image processing based on the sensor output. Each channel provides different information. One channel provides the distance or height of the front surface position to the bed or table on which the patient lies, to the sensor, and / or relative to another location. The sensed outer surface and the known position of the sensor to the bed are used to determine the distance. The other channel is the patient's thickness. The thickness can be the difference between a given depth and the maximum and minimum depths. Other thicknesses can also be used. The first channel stores the depth of the body surface as viewed from the front or looking at the patient resting on the patient bed, and the second channel stores the thickness calculated by measuring the distance between the nearest and farthest points as viewed from the front. Other channels may be used, such as one channel for depth from a sensor, and another channel for an optical image of the patient.Other surface data may be used.

[0037] The surface data is used at the resolution of the sensor. For example, the surface data is in 256×256 pixels. Other sizes can be used, including rectangular fields of view. The surface data can be filtered and / or processed. For example, the surface data is changed to a given resolution. As another example, the surface data is downsampled, such as from 256×256 to 64×64 pixels. Each pixel can represent any area, such as: each pixel downsampled to 64×64 represents 1 cm 2or larger. Alternatively, the sensor captures at this lower resolution. The surface data can be cropped, such as to limit the field of view. Cropping and downsampling can both be used together, such as to create 64×64 channel data from 256×312 or other input channel data. Larger or lower resolution can help with regression.

[0038] In another approach, the surface data is normalized before input. The surface data is rescaled, resized, distorted, or shifted (eg, interpolated). The surface data may be filtered, such as low-pass filtered.

[0039] In act 12, the image processor fits a patient model to the surface data. A patient model is a general representation of the surface of a person or part of a person. Different models can be used for different body types, such as male or female models. The patient model is not patient-specific. For example, the patient model is a statistical shape model. The patient model is not specific to any other patient, or specific to the patient that meets the specifications.

[0040] Any representation can be used for the model. In one embodiment, the model is formed by a mesh (such as a mesh of triangles). Other meshes can be used. Other representations of 3D surfaces can be used.

[0041] The image processor fits a patient model to the surface data for the patient to be treated or imaged. The fitting transforms or deforms the patient model based on the sensed external surface of the patient. A generic patient model is personalized to the patient's external surface by fitting to the surface data. The fitting can be an iterative optimization, such as testing different variations of the model, where the variations are controlled based on the closeness of the fit or the difference between the model and the surface data.

[0042] Any now known or later developed fit of the body surface model to the captured surface data for the patient can be used. For example, a SCAPE model is fitted to the surface data based on minimization of differences. In one embodiment, a depth camera image 20 of a subject is converted into a 3D point cloud. A plurality of anatomical landmarks are detected in the 3D point cloud. A 3D avatar mesh is initialized by aligning a template mesh to the 3D point cloud based on the detected anatomical landmarks. A personalized 3D avatar mesh of the subject is generated by optimizing the 3D avatar mesh using a trained parametric deformable model (PDM). The optimization is subject to constraints that take into account the clothing worn by the subject and the presence of a table on which the subject lies.

[0043] In another embodiment, a statistical shape model is fitted to the depth as surface data. The statistical shape model is a grid or other representation of an average or other statistical representation of the exterior of a person or part of a person. The statistical shape model includes probabilities or other constraints on variations so that the fit maintains the statistically based shape.

[0044] In yet another embodiment, a personalized 3D mesh of a person is generated by fitting a human skeleton model to the person's depth image data via a model-based approach. The estimated pose skeleton is then used to initialize a detailed parameterized deformable mesh (PDM) that is learned in an offline training phase. The PDM is then optimized to fit the input depth data by perturbing the body pose and shape. A sampling-based optimization procedure fits the PDM to the depth data. Unlike the Shape Restoration and Character Animation (SCAPE) model, which is only applicable to data with skin-covered subjects, the sampling-based approach copes with changes in the subject's clothing. In addition, the sampling-based approach also enables embodiments to cope with bias introduced due to sensor noise.

[0045] Figure 4 An exemplary fitted patient model 22 is shown. The body surface mesh indicates the body shape. Based on this fitted body surface mesh, value(s) of feature(s) used to estimate weight can be extracted.

[0046] In act 14, the image processor extracts values ​​for each of the one or more features from the patient model as fitted to the surface data for the patient.Shape feature values ​​are extracted from the patient model fitted to the exterior surface captured with the sensor.

[0047] The extraction is by calculation, measurement and / or image processing.The image processor uses the model as fitted to extract values ​​that are less susceptible to noise and noise than if extracted from depth or other sensor measurements.

[0048] Any one or more features may be used. Features are shape features, such as parameters that characterize the shape of the patient's outer surface as represented by the fitted patient model. Example shape features may include volume, area, diameter, circumference, length, width, curvature, other anthropometric measurements, and / or other geometric measurements.

[0049] In one embodiment, a value for body volume is extracted. Body volume is one of the indicators of a person's weight. Figure 5 A plot of volume information and corresponding weight from 196 human samples from the public dataset CAESER is shown. There is a linear correlation between volume features and weight.

[0050] The volume of the fitted body mesh is calculated and then used to estimate weight information through learned regression. The volume enclosed by the fitted patient model 22 is represented by the fitted mesh. The volume here is the volume of the fitted mesh, which is different from the patient's body volume. If the patient model 22 does not include one or more parts of the patient (e.g., feet, hands, arms, head, and / or legs), the body volume can be that of the parts included in the patient model 22. The fitted model can accurately represent the patient's volume.

[0051] In another embodiment, a value is extracted for each of a plurality of different body parts. Values ​​for the same or different features or feature sets are extracted for different body parts. In humans, different body parts may have different densities. For example, the legs and arms are typically denser than the upper body (torso), which includes various organs. Having different volumes in different body parts may have an impact on body weight. Based on this observation, values ​​are extracted separately from the fitted body mesh for each individual body part. For example, the volumes of the legs, arms, torso, and head are extracted.

[0052] exist Figure 4 In the fitted body mesh of , different parts of the mesh are labeled for different parts. Figure 4 Different gray levels in ℓ show different parts. The patient model 22 as fitted may include additional, different or fewer body parts. Values ​​of features such as volume and / or individual shape coefficients are extracted for each of the parts.

[0053] In yet another embodiment, the image processor extracts values ​​for shape manifold coefficients as features. Instead of simple, straightforward shape features such as volume, shape manifold information is used. For example, by using SCAPE or other fitted patient model 22, fitted shape coefficient information for the fitted mesh is extracted. Any parameterization or mathematical representation of the shape may be used. For example, assuming V = {v1, v2, v3, ..., v n}, where V is a shape manifold vector with n dimensions. The extracted coefficient vector has n dimensions. i , i∈[1,n] denotes the i-th element of the coefficient. Any shape manifold can be used.

[0054] exist Figure 1In act 16, the image processor estimates the patient's weight by regression based on one or more shape feature values. In one embodiment, a lookup table is used that incorporates the correlation between one or more shape features and weight. In another embodiment, a function such as a fitted line is used to determine weight based on the input values ​​of the features. For example, the volume of the fitted body mesh is used to estimate weight through learned regression. Any regression based on various past examples or samples can be used, such as linear regression, least squares, another parametric regression, and / or non-parametric regression.

[0055] In one embodiment, the image processor estimates weight by inputting one or more values ​​for the one or more features into a machine-learned regressor. A machine-learned regressor is any machine-learned classifier that outputs a continuous variable, such as a machine-learned linear regression. For example, a neural network is used to regress the relationship between the input features and the output weight. A fully connected neural network or another neural network can be used. A support vector machine, cluster-based machine learning, Bayesian, or other machine-learned regressor can be used.

[0056] In order to train the machine-learned network, the machine-learned network arrangement is defined. The definition is by programming or configuration of learning. The number of layers or units, the type of learning, and other characteristics of the network are controlled by the programmer or user. In other embodiments, one or more aspects (such as the number of nodes, the number of layers or units, or the type of learning) are defined and selected by the machine during learning. Training data comprising many samples of one or more values ​​for one or more features and corresponding ground truth (i.e., weight) is used for training. Machine learning is performed on the relationship between input and output. Once trained, the machine-learned model (machine-learned regressor) can be applied to estimate weight based on one or more values ​​for one or more features of the patient.

[0057] In one embodiment, the machine-learned regression is a trained fully convolutional network. A mapping function W = f(V, θ) is learned to estimate weight, denoted by W. The mapping function is modeled as a fully connected neural network with parameters θ. Through training, the parameters are learned, such as the parameter θ used to estimate weight W based on the input volume as a shape feature.

[0058] In another embodiment, different regressors are learned for different body parts. Partial weights are estimated for different body parts and summed to obtain the patient's weight. For example, separate shape coefficient values ​​or volumes are extracted for different body parts. The extracted value or values ​​for each body part are then used as input to each separate regression function (e.g., modeled as a different fully connected neural network). Denote the set of mapping functions as F = {f1, f2, f3, ..., f m}, where f j ,j∈[1,m] denotes the mapping function of the jth body part, and m denotes the total number of body parts. The mapping function is modeled as having parameters θ j The fully connected neural network, that is, w j =f j (x j ,θ j ), where x j Then, use W = ∑w j In an alternative embodiment, the values ​​of the features from the different parts are input to a machine-learned regressor that outputs the total weight without direct regression or output of the individual part weights.

[0059] Figure 6 Comparison of weight estimates from a machine-learned regressor with actual weight. A machine-learned regressor in the form of a fully connected neural network was trained to estimate weight based on the volume of the fitted meshes. Volume information was extracted from 1766 fitted body meshes and used to learn a regression based on this feature to estimate weight. From this trained regression on an additional 196 test samples, it was found that Figure 6 The line y = x is a linear regression where the prediction exactly matches the ground truth. This yields a mean average error of 1.79 kg. This error is better than the weight estimates of healthcare workers.

[0060] In act 18, the patient is dosed based on weight. The physician determines the amount of the medication dose. The patient ingests the medication, or the medication is infused. The amount of medication ingested or infused is based at least in part on the estimated weight.

[0061] In another embodiment, a patient will receive radiation during a treatment or diagnostic scan. The amount of radiation is based on body weight. This amount can be the intensity, duration, frequency, and / or other settings of the x-ray machine that control how much radiation is applied to the patient and / or a given area of ​​the patient. Patients with less weight receive less radiation. The radiation dose is based on body weight.

[0062] In one example, an injured patient is being treated in a hospital. Due to the injury, the patient cannot stand on a scale. Instead, the patient is placed on a bed (such as a table of a therapeutic x-ray system). A depth camera in the room, such as one mounted to the therapeutic x-ray system, images the patient. Based on fitting a patient model to the surface data from the depth camera, the patient's weight is estimated. The weight is used to control the dose delivered by the therapeutic x-ray system. The weight is used automatically or by manual input to configure the x-ray system to apply a weight-corrected dose specific to the patient.

[0063] In another example, an image processor configures a diagnostic medical scanner (e.g., computed tomography, fluoroscopy, or x-ray) based on weight. The medical scanner can configure itself. The image processor can provide information to a controller of the medical scanner for configuration. The image processor can configure the medical scanner by directly controlling the medical scanner. Alternatively, a user manually configures the medical scanner based on weight entered using one or more controls.

[0064] One or more settings for imaging or scanning a patient can be adjusted or set, at least in part, using body weight. Body weight can help plan medical scans, such as limiting the field of view and / or x-ray intensity to control the dose from the radiation. For CT scans, body weight can be used to determine the scan range. This can reduce the amount of ionizing radiation applied to the patient. Body weight can help with the scan sequence, coil placement, and / or scan position for magnetic resonance (MR) scans. For ultrasound scans, body weight can help with imaging frequency, with lower frequencies used for heavier patients. For fluoroscopy using dyna-CT (dynamic-CT) scanning, body weight can be useful for positioning the patient and / or scanner and controlling the x-ray source. Any setting or parameter of a medical scanner can be determined or configured based on body weight.

[0065] The configured medical scanner scans the patient. The patient is imaged. The imaging is performed based on the configuration of the medical scanner. For treatment, the treatment system applies radiation based on the configuration.

[0066] Figure 7 An embodiment of a medical imaging system for patient weight estimation is shown. The medical imaging system includes a display 70, a memory 74, and an image processor 72. The display 70, image processor 72, and memory 74 may be part of a medical treatment system 76, a computer, a server, a workstation, or other system for image processing medical images scanned from a patient. A workstation or computer without the medical treatment system 76 may be used as the medical imaging system. The medical imaging system also includes a sensor 77 for sensing (imaging) the patient's external surface.

[0067] Additional, different, or fewer components may be provided. For example, a computer network is included for remote image generation of locally captured surface data, or for local weight estimation based on remotely captured surface data. Machine-learned regressors or other regressions are applied as standalone applications on a workstation or local device, or as services deployed on a network (cloud) architecture. As another example, a user input device (e.g., a keyboard, buttons, sliders, dials, trackballs, mice, or other devices) is provided for user change or placement of one or more markers (e.g., landmarks). In yet another example, a medical treatment system 76 is not provided, such as in the case where medication is to be applied or in the case where a medical diagnostic imager is used instead.

[0068] Sensor 77 is a depth sensor or camera. LIDAR, 2.5D, RGBD, stereo optical sensors, or other depth sensors may be used. One sensor 77 is shown, but multiple sensors may be used. A light projector may be provided. Sensor 77 may measure the depth directly from sensor 77 to the patient. Sensor 77 may include a separate processor for determining the depth measurement from the image, or image processor 72 may determine the depth measurement from the image captured by sensor 77. The depth may be relative to sensor 77 and / or bed or table 79.

[0069] The sensor 77 is directed toward the patient 78. The sensor 77 may be part of or connected to the medical treatment system 76, or separate from the medical treatment system 76.

[0070] Sensor 77 is configured to measure depth to or against a patient. Depth is the distance from sensor 77, table 79, or other location to the patient, at various locations on the patient. Any sample pattern on the patient can be used. Sensor 77 outputs a depth measurement and / or a surface image.

[0071] The image processor 72 is a control processor, a general-purpose processor, a digital signal processor, a three-dimensional data processor, a graphics processing unit, an application-specific integrated circuit, a field-programmable gate array, an artificial intelligence processor, a digital circuit, an analog circuit, a combination thereof, or other now known or later developed device for processing surface data. The image processor 72 is a single device, a plurality of devices, or a network. For more than one device, parallel or sequential partitioning of processing can be used. The different devices that make up the image processor 72 can perform different functions, such as fitting a mesh as a patient model by one device and estimating weight by another device. In one embodiment, the image processor 72 is a control processor or other processor of the medical treatment system 76. The image processor 72 operates in accordance with stored instructions, hardware, and / or firmware and is configured by the stored instructions, hardware, and / or firmware to perform the various actions described herein.

[0072] The image processor 72 is configured to train a machine learning architecture. Based on the network architecture and user-provided or other sources of training data, the image processor 72 learns to associate one or more input variables (e.g., volume and / or other shape features) with output variables (e.g., weight and / or height) to train a regressor. The result of the training is one or more machine-learned regressors for weight or other predictions.

[0073] Alternatively or additionally, the image processor 72 is configured to apply one or more machine-learned regressors. In response to input of one or more values ​​for one or more shape features of the patient, the machine-learned regressors output weight. In other embodiments, the image processor 72 regresses the output without the machine-learned regressors.

[0074] The image processor 72 is configured to form a mesh for the patient using the depth from the sensor 77. The mesh is fitted to the depth or other output of the sensor as a model of the patient. The mesh may be fitted by detecting the surface of the patient and / or by minimizing the difference between a predetermined mesh shape and a shape represented by the output of the sensor.

[0075] The image processor 72 is configured to extract values ​​for shape features from the mesh formed for the patient. Any shape features may be used, such as volumes for the mesh as a whole and / or for a portion of the mesh (e.g., arm, leg, head, and torso volumes extracted separately based on the boundaries defined by the mesh of the patient model). In one embodiment, the shape features are defined by a shape manifold, and thus the image processor 72 extracts values ​​for shape manifold coefficients from the fitted mesh.

[0076] The image processor 72 is configured to regress the weight based on the extracted one or more values. The weight of the patient is regressed based on the grid formed for the patient via the values ​​extracted from the fitted grid. Multiple regressors can be used, such as in the case of regressing and summing the weight of various parts of the patient.

[0077] Display 70 is a CRT, LCD, projector, plasma, printer, tablet computer, smart phone or other now known or later developed display device for displaying output (such as an image of or including weight). Display 70 displays a medical image of the patient and / or the patient's weight.

[0078] Sensor measurements, fitted shape models, surface data, network definitions, features, machine-learned regressors, extracted values, regressed weight, and / or other information are stored in a non-transitory computer-readable memory, such as memory 74. Memory 74 is an external storage device, RAM, ROM, database, and / or local memory (e.g., a solid-state drive or hard drive). The same or different non-transitory computer-readable media can be used for instructions and other data. Memory 74 can be implemented using a database management system (DBMS) and resident on a memory, such as a hard disk, RAM, or removable media. Alternatively, memory 74 is internal to processor 72 (e.g., a cache).

[0079] Instructions for implementing the training or application processes, methods, and / or techniques discussed herein are provided on a non-transitory computer-readable storage medium or memory, such as a cache, buffer, RAM, removable media, a hard drive, or other computer-readable storage medium (e.g., memory 74). Computer-readable storage media include various types of volatile and non-volatile storage media. The functions, actions, or tasks illustrated in the figures or described herein are performed in response to one or more instruction sets stored in or on a computer-readable storage medium. The functions, actions, or tasks are independent of the specific type of instruction set, storage medium, processor, or processing strategy, and can be performed by software, hardware, integrated circuits, firmware, microcode, etc., operating individually or in combination.

[0080] In one embodiment, the instructions are stored on a removable media device for reading by a local or remote system. In other embodiments, the instructions are stored in a remote location for transmission over a computer network. In still other embodiments, the instructions are stored within a given computer, CPU, GPU, or system. Because some of the system components and method steps of the components depicted in the accompanying drawings can be implemented in software, the actual connection between the system components (or process steps) may vary depending on the manner in which the present embodiment is programmed.

[0081] Medical therapy system 76 is a therapeutic radiation scanner, such as an x-ray or particle therapy system. Medical therapy system 76 operates according to one or more settings to apply radiation to a patient. These settings control the location and radiation dose. The intensity, frequency at a given location, duration of application to a location, and / or other settings are controlled, at least in part, based on body weight. Body weight is used to adjust or initially set the amount of radiation dose to be applied. Body weight can be used to constrain the dose. Weight-based configuration can be automatic or manual.

[0082] Once configured by the setup, the medical treatment system 76 applies radiation to the patient. The amount of radiation applied is based at least in part on body weight. Heavier patients may receive more radiation or less radiation than lighter patients.

[0083] In an alternative embodiment, the medical scanner is configured to scan an internal region of the patient and generate diagnostic information based on the scan.The medical scanner is a CT, MR, PET, SPECT, X-ray or ultrasound scanner.

[0084] The medical scanner is configured to generate diagnostic image information. The configuration uses settings for one or more parameters, such as X-ray source voltage, table position and / or range of motion, gantry position and / or range of motion, focus, field of view, scan density, detector threshold, transmission sequence, image processing settings, filter settings, or image generation settings. One or more settings of the medical scanner are automatically or manually set based on a weight generated based on shape features of a model fitted to the surface data. The medical scanner uses the settings to image a patient 78.

[0085] The various improvements described herein may be used together or separately. Although illustrative embodiments of the present invention have been described herein with reference to the accompanying drawings, it is to be understood that the invention is not limited to those precise embodiments and that various other changes and modifications may be effected therein by those skilled in the art without departing from the scope or spirit of the invention.

Claims

1. A method for estimating patient weight based on surface data in a medical imaging system, the method comprising: capturing an outer surface of the patient using a sensor, the surface data being derived from capturing the outer surface of the patient; fitting a patient mesh to the surface data; extracting one or more values ​​for each of a plurality of different body parts of the patient from the patient mesh as fitted to the surface data for the patient; estimating a weight for each of a plurality of different body parts of the patient by inputting one or more values ​​for the plurality of different body parts into a respective machine-learned regressor for each of the plurality of different body parts; as well as A medical scanner is configured based on the weight. The method of claim 1 , wherein capturing comprises capturing using a sensor that is a depth sensor. The method of claim 1 , wherein capturing comprises capturing with a sensor being a camera, wherein the surface data is based on optical measurements. The method of claim 1 , wherein extracting comprises extracting a value of a shape feature as one of the one or more features. The method of claim 4 , wherein extracting shape features comprises extracting a body volume of the fitted patient mesh. The method of claim 1 , wherein estimating comprises estimating partial weights for different body parts and summing the partial weights, the sum comprising the patient's weight. The method of claim 1 , wherein extracting comprises extracting values ​​for shape manifold coefficients as features.

8. The method of claim 1, wherein estimating comprises estimating using a machine-learned regression comprising a fully convolutional network.

9. The method of claim 1, wherein estimating comprises estimating using a machine-learned regression, wherein the machine-learned regression comprises a machine-learned linear regression.

10. A system for estimating a patient's weight, comprising: Medical imaging systems, including: a depth sensor configured to measure depth to the patient; an image processor configured to form a mesh for the patient by using the depth, extract one or more values ​​for each of a plurality of different body parts from the mesh as formed for the patient, and regress the patient's weight from the mesh as formed for the patient by inputting the one or more values ​​for the plurality of different body parts into a respective machine-learned regressor for each of the plurality of different body parts; and a memory configured to store the regressed weight; and A medical scanner is configured based on the weight.

Images