Analysis subject

Abstract

In one aspect, a system 10 for analyzing a subject 14 using a device 12 is disclosed. In one arrangement and embodiment, the device 12 includes: a controller 18; a storage device 20 storing electronic program instructions for controlling the controller 18; and an input component. In one form, the controller is operable under the control of the electronic program instructions to: receive input via the input component, the input including at least one representation of the subject 14; process the input to perform an analysis of the subject 14 and generate an output based on the analysis, the processing including using a database 40; and transmit the output via a display 22. In one embodiment, the output includes an estimate of a three-dimensional (3D) body shape and associated body measurements, composition, and health care risks of an individual from a representation including an image of a human.

Description

Technical Field

[0001] The present invention generally relates to analytical subjects.

[0002] The present invention will be described with particular reference to analyzing the human body to provide estimates of an individual's three-dimensional (3D) body shape and its associated body composition and health care risks. However, it will be appreciated that the present invention is not limited to this particular field of use, and that it may be used for a host of other things, and for performing analysis for additional and / or alternative purposes. Background Art

[0003] Any discussion of the background art throughout the specification should not be taken as an admission that the background art is prior art or that the background art is widely known or forms part of the common general knowledge in the field, either in Australia or worldwide.

[0004] All references cited in this specification, including any patents or patent applications, are incorporated herein by reference. No admission is made that any reference constitutes prior art. The discussion of the references states what the authors assert, and the applicant reserves the right to challenge the accuracy and relevance of the references. It is to be clearly understood that although a number of prior art publications are cited herein, such citation does not constitute an admission that any of these documents constitutes part of the common general knowledge in the art in Australia or any other country.

[0005] The management, treatment and monitoring of prediabetes, chronic health, premature mortality and musculoskeletal conditions are increasingly becoming global healthcare concerns, costing governments around the world trillions of dollars each year. According to a recent study published by the World Economic Forum (Kate Kelland. Chronic disease to cost $47 trillion by 2030: WEF. Published online, September 19, 2011), it is estimated that the management of chronic diseases alone will reach more than $46 trillion within 20 years.

[0006] It has been determined that non-communicable diseases (i.e., chronic diseases) such as cardiovascular disease and type 2 diabetes are becoming a growing concern for global healthcare (GBD 2016 Causes of Death Collaborators. Global, regional, and national age-sex specific mortality for 264 causes of death, 1980–2016: a systematic analysis for the Global Burden of Disease Study 2016. The Lancet. 390(10100): 1151–1210; and WHO. Global action plan for the prevention and control of noncommunicable diseases 2013–2020. Geneva, Switzerland: World Health Organization, 2013. Keys, A., Karvonen, N., Kimura, N., Taylor, HL). Fat is a characteristic of obesity that has been identified by the World Health Organization (WHO) as a major goal for reducing chronic diseases and related deaths worldwide. Obesity can be assessed directly or indirectly through measurements of an individual's body shape / composition, and as obesity is revealed to be linked to the development of chronic disease, it is important for researchers, insurers, and healthcare professionals alike to understand which measurements are reliable and relevant to estimates of chronic disease risk. This is important so that more reliable, sensitive, and specific risk estimates can be developed, which can translate into: 1) reductions in global healthcare costs through population surveillance and early targeted(er) interventions; and 2) reductions in claims paid by insurers and reinsurers as underwriting standards improve.

[0007] Since 1980, the incidence of cardiovascular disease and diabetes has increased worldwide (Anand, S. and Yusuf, S. (2011). Stemming the global tsunami of cardiovascular disease. The Lancet. 377 (9765): 529-532.; NCD Risk Factor Collaboration (2016). Worldwide trends in diabetes since 1980: a pooled analysis of 751 population-based studies with 4.4 million participants. The Lancet. 387 (10027): 1513–1530.). In a relatively short period of time in human history, the world's diabetes incidence has risen from 108 million to a staggering 422 million people. The number of people classified as prediabetes has also increased dramatically, and it is estimated that by 2030, more than 470 million people worldwide will be diagnosed with prediabetes (Tabák, AG, Herder, C., Rathmann, W Brunner, EJ., M. (2012). Prediabetes: A high-risk state for developing diabetes. 379(9833): 2279–2290.). Recent statistics also show that as the global population ages, knee and hip osteoarthritis are increasingly contributing to the world's healthcare costs (Allen, K. and Golightly, Y., 2015 Epidemiology of osteoarthritis: state of the evidence). With the increasing number of elderly people in the current world population, healthcare costs associated with fall risk and sarcopenia (i.e., excessive loss of lean muscle mass during human aging) have been growing rapidly. In the United States (US), the prevalence of sarcopenia in the elderly population in long-term care facilities is estimated to be between 14% and 33%. It was estimated that in 2000, the annual healthcare costs associated with sarcopenia in the United States were $42.1 billion (Cruz-Jentoft AJ, Landi F, Schneider SM, et al. Prevalence of and interventions for sarcopenia inageing adults: a systematic review. Report of the International Sarcopenia Initiative (EWGSOP and IWGS). Age Ageing. 2014; 43: 748-759).

[0008] A contributing but modifiable risk factor for prediabetes, chronic health, mortality (GBD, 2016), musculoskeletal disorders, sarcopenia, and fall risk is an individual’s body composition (i.e., lean muscle mass, total fat, and central fat).

[0009] In response to these worrisome world trends, healthcare systems and government agencies worldwide have and continue to invest significant time and money in monitoring the body composition of their populations, with a particular focus on their body composition (i.e., lean body mass, adiposity, and bone mineral density). It is also important for governments and allied health professionals to understand which measures are associated with predicting prediabetes, chronic disease, mortality, and musculoskeletal disorders. This is important so that reliable, sensitive, and specific relative and predictive health risk techniques and equations can be developed.

[0010] The main body composition measurements used to predict chronic disease and mortality are those around the central or abdominal fat (WHO; Pischon et al (2008). General and Abdominal Adiposity and Risk of Death in Europe. N Engl J Med 2008; 359: 2105-2120. DOI: 10.1056 / NEJMoa0801891; Jacobs et al (2010). Waist Circumference and All-Cause Mortality in a Large US Cohort. Arch Intern Med. 2010 Aug 9; 170(15): 1293-301.). Central fat measurements include androgenetic fat, gynecological fat, visceral fat, waist circumference, and waist-to-hip ratio.Among the many existing central fat measurements, visceral fat has become a key body composition variable used to predict an individual's risk of chronic health conditions such as heart disease and diabetes (Ding Y., Gu D., Zhang Y., Han W., Liu H., Qu Q. (2015). Significantly increased visceral adiposity index in prehypertension. PLoS One. 10; 10(4): e0123414; Hanley AJ, Wagenknecht LE, Norris JM, Bryer-Ash M, Chen YI, Anderson AM, Bergman R, Haffner SM. Insulin resistance, beta cell dysfunction and visceral adiposity as predictors of incident diabetes: the Insulin Resistance Atherosclerosis Study (IRAS) Family study. Diabetologia 52:2079–2086,2009; Ghroubi, S., Elleuch, H., Guermazi, M., Kaffel, N., Feki, H.; Abid, M., Baklouti, S.; Elleuch, MH (2007). "Abdominal obesity and knee ostheoarthritis". Annales de Réadaptation et de Médecine Physique.50(8):661–666.doi:10.1016 / j.annrmp.2007.03.005.).

[0011] There are many studies showing that individuals in the prediabetes and diabetes population have elevated total and central fat measurements compared to normal healthy populations. With the relationship between an individual's body composition and prediabetes and diabetes, there is an opportunity to develop relative risk or predictive equations to estimate an individual's risk of prediabetes and diabetes from their body composition and relevant participant characteristics, anthropometrics, and epidemiological (i.e., prediabetes and diabetes status) information. The current traditional method for screening or diagnosing prediabetes or diabetes is an invasive, painful, time-consuming, and costly blood test (i.e., fasting plasma glucose, HbA1c), so there is a need in the market for a low-cost, non-invasive, and accurate method.

[0012] Taking the effective monitoring and screening of sarcopenia as an example, lean body composition measurements will be of interest, and in particular lean muscle mass and limb lean muscle mass (Matthew J. Delmonico, PhD, MPH and Darren T. Beck, PhD. (2017). The Current Understanding of Sarcopenia: Emerging Tools and Interventional Possibilities. American Journal of Lifestyle Medicine.). Lean muscle mass, and especially the muscles of the lower extremities, is also important for screening and monitoring the risk of falls in the elderly. In people over 70 years old with hip or pelvic fractures caused by falls, the one-year mortality rate increases to 90%, so it is important that we have cost-effective and accurate methods to screen and monitor this population.

[0013] The reason body composition measures such as lean muscle mass and fat are of particular interest for the management of prediabetes, chronic health conditions, mortality, and musculoskeletal disorders is because they are modifiable factors. If individuals could easily and affordably measure relevant body composition variables, they could then focus on modifying these body composition variables to improve their health care risks. This could lead to reductions in prediabetes diagnoses, chronic disease, premature mortality, musculoskeletal disorders, and falls-related risks, improved health outcomes, and reduced health-related expenditures. However, doing this effectively requires body composition measurement tools that approach the theoretical ideal.

[0014] An ideal body composition measurement tool for the general population, in theory, must first be able to measure the right variables—meaning variables that are relevant to an individual’s general health and overall wellness.

[0015] Second, measurements of body composition variables must be affordable, accessible, and accurate.

[0016] In the attached figure Figure 1 A schematic diagram of an ideal body composition measurement model is depicted in This should be used in the context of the prior art to better understand the current state of the art in the current market space, and also to understand how embodiments of the present invention seek to bridge the current market gap.

[0017] Without valid, affordable, and easy-to-use tools to estimate body composition measurements, government agencies and health care providers have incomplete health information to effectively monitor and / or treat chronic health and musculoskeletal conditions, as well as individual risk of premature death, throughout their population’s lifespan. This severely impedes the ability of the health care system to prevent, manage, and treat chronic health and musculoskeletal conditions, such as heart disease, prediabetes, diabetes, osteoarthritis of the joints, and sarcopenia, as well as to predict premature death.

[0018] The established standard measures for reliable in vivo measurements of body composition variables, such as central and total body fat and lean muscle mass, are medical imaging techniques (Duren, D., Sherwood, R., Czerwinski, S. et al. (2008). Body Composition Methods: Comparisons and Interpretation. Journal of Diabetes Science and Technology 2(6): 1139-1146.). These measurement methods include, but are not limited to, positron emission tomography (PET), computed tomography (CT), dual energy absorptiometry (DXA), and magnetic resonance (MR). Since 1990, measurement of central fat in large populations has been limited primarily by costly and time-consuming CT (Fox CS, Massaro JM, Hoffmann U et al. Abdominal visceral and subcutaneous adipose tissue compartments: association with metabolic risk factors in the Framingham Heart Study. Circulation 116:39-48, 2007.; Liu J, Fox CS, Hickson DA, May WD, Hairston KG, Carr JJ, Taylor HA. Impact of abdominal visceral and subcutaneous adipose tissue on cardiometabolic risk factors: the Jackson Heart Study. J Clin Endocrinol Metab 95:5419-5426, 2010.) and, to a lesser extent, MR (Neeland IJ, Turer AT, Ayers CR, Powell-Wiley TM, Vega, GL Farzaneh-Far R, et al. Abdominal visceral and subcutaneous adipose tissue compartments: association with metabolic risk factors in the Framingham Heart Study. Circulation 116:39-48, 2007.); Liu J, Fox CS, Hickson DA, May WD, Hairston KG, Carr JJ, Taylor HA. Impact of abdominal visceral and subcutaneous adipose tissue on cardiometabolic risk factors: the Jackson Heart Study. J Clin Endocrinol Metab 95:5419-5426, 2010.). al, Dysfunctional Adiposity and the Risk of Prediabetes and Type 2diabetes in Obese Adults. JAMA. 2012; 308(11):1150–1159.).With recent technological advances, lower-cost DXA systems have been validated for measurement of systemic and central obesity (Kaul S, Rothney MP, Peters DM, et al, Dual-energy X-ray absorptiometry for quantification of visceral fat. Obesity 2012; Doi: 10.1038: 1-6.). Through recent advances, the cost and time associated with medical imaging have been mitigated, but not eliminated. Although medical imaging technology is reliable, it is expensive, requires medical referral to obtain, and may expose patients to harmful ionizing radiation. Reference ideal body composition measurement tools, such as. Figure 1 As shown, medical imaging technology is accurate (102), but not affordable (103) or accessible (101).

[0019] Indirect methods of body composition estimation, such as body mass index (BMI), waist circumference (WC), waist-to-hip ratio (WHR), and body impedance analysis (BIA), are cost-effective and accessible, but have high levels of measurement error. The reason is that all indirect methods make multiple assumptions related to body tissue density, distribution, and (multiple) water concentrations (Duren et al., 2008). These assumptions make these methods particularly erroneous in obese, elderly, and pathological populations, which are, ironically, the very population groups that most institutions are trying to identify and treat. It is this combination of factors that severely limits indirect body composition methods as effective measurement tools for the management of chronic health and musculoskeletal conditions. In the context of the ideal body composition measurement tool, indirect body composition estimation methods are affordable (103) and accessible (101), but imprecise (102).

[0020] Although it is well established that BMI is not an accurate representation of an individual's body composition and, in turn, is a relatively poor measure of an individual's chronic health (Bray et al (2008). Relation of central adiposity and body mass index to the development of diabetes in the Diabetes Prevention Program.) and mortality (Pischon et al, 2008) risk, BMI remains a commonly used measurement tool in population-based research and insurance agencies. It is also common to use self-reported height and weight measurements in BMI calculations, adding further measurement uncertainty to an already inaccurate measurement. When self-reported height and weight are used to calculate an individual's BMI, the probability of misclassification is 22.4% for men and 18.0% for women (Spencer et al (2001). Validity of self-reported height and weight in 4808 EPIC–Oxford participants. Public Health Nutrition: 5(4), 561–565). Targeting population researchers and insurers who still rely on BMI as a key measurement, a method to automatically and accurately estimate and validate an individual's self-reported height and weight from accurate, accessible and affordable measurement tools would add significant value to population-based research and the insurance sector.

[0021] Since 2000, researchers have attempted to bridge the gap between affordability, accessibility, and accuracy of measurements by combining measurement information derived from high-precision 3D surface scanners. The interesting aspect of this approach is that a 3D human body can be segmented into multiple regions (i.e., upper / mid / lower torso, arms, legs, etc.) and a range of linear measures, perimeters, surfaces, or volumes using software that is manually guided by the user. From this a wide variety of volumes, surface areas, perimeters, and linear measures and measurement ratios can be calculated.

[0022] Using linear regression methods, 3D surface scanner measurements and measurement ratios are combined with information on participant characteristics (height, weight, sex, age) to estimate body composition variables calculated from in vivo medical imaging techniques. There are many regression equations published in the literature using these techniques. Two of the most notable equations are those published by Lee et al. (Lee et al. (2014). Predictive Equations for Central Obesity via Anthropometrics, Stereovision Imaging and MRI in Adults.) and Ng et al. (Ng et al. (2016). Clinical anthropometrics and body composition from 3D whole-body surface scans.), who predicted total body fat (R) with high accuracy. 2 =0.95) and abdominal fat (R 2 =0.79-0.94). Visceral fat estimation is a key variable used to classify individual health risk, with a moderate level of accuracy (R 2 =0.72-0.75) (Lee et al., 2014; Ng et al., 2016).

[0023] As shown below, such regression using 3D surface scanners encounters five significant limitations. 1) Moderately expensive high-precision 3D body surface scanners are required to acquire an individual's 3D body model in a specialized testing environment. 2) These methods require manual processing of the scanned 3D data, limiting their application to general or untrained users. 3) In most cases, the application of the regression-based computational model is limited to the homogeneous test population for which it was developed; this is often due to low statistical power and model specification (i.e., overfitting / underfitting) (McLean, D., Redfern, D., and Pyper, K. (2014). A Survey of Regression Methods for Proxy Functions. Moody's Analytics Research. Consultancy report: B&H Research Insurance ERS.). 4) The segmentation of an individual's body volume is not guided by the individual's anatomy (i.e., joint centers and anatomical landmarks); instead, simple shapes and volumes are fitted or registered to the individual's 3D volume data. 5) These 3D volume and regression based methods still cannot predict key health risk variables from the predicted visceral fat with the accuracy level required for medical use, which is R 2>0.90. Taking these limitations into account, and referring to an ideal body composition measurement tool, 3D scanning methods with regression are accurate for limited population sizes (102), but not all body composition variables, moderately affordable (103), and moderately accessible (101). Most importantly, the best reported accuracies (e.g., for visceral fat) are those methods based on cross-validation type analyses or reported accuracies based on actual fitted or trained data, so the accuracy for any unseen data set will be lower than the reported accuracy. This is not intentional with current technology, but is due to the lack of larger data sizes that would ideally be required for accurate analysis and modeling of populations or portions of populations.

[0024] Patent-pending Farahbakhshian et al. (Naked Labs), publication number US2018 / 0137640A1, have attempted to address affordability (103) and accessibility (101) issues by designing moderately affordable 3D scanners, however, the accuracy of actual 3D scans can suffer when it comes to suturing of rotational, deforming, permeable shapes like the human body, and estimates of body composition are limited to precise global volumetric measurements combined with publicly available linear regression equations, e.g., Farahbakhshian et al. discuss using the Siri or Brozak equations developed over a century ago to estimate body composition. Furthermore, the calculated volumes have no physical meaning as they are not constrained by any anatomically or physically valid bone joints.

[0025] Developed by Select Research Limited and disclosed in patents US 8,374,671 B2 and EP 1 993443 B1, a laboratory-based system uses multiple pre-calibrated cameras to create a visual shell of a person. The shell is then manually cleaned and annotated by experts and then processed by a software program to separate it into different volume layers. Their method combines a linear regression method with 3D volume measurements and participant characteristic information to estimate the in vivo body composition of an individual and focuses on the proportion of the volume of each part of the body to estimate health risks. The method has been widely used and published by many scientists in the same industry, and the main contribution made by Select Research Limited in the above patents is a scanning system called a body volume index scanner. Like numerical integration errors, these body volumes or geometries are not accurate, cannot adapt to the subtle individual nuances of the human form, and have no physical meaning because they are not constrained by the bones or joints of any anatomically or physically valid subject.

[0026] The limitations of the use of linear regression and 3D scanner methods (including the 3D scanner method of Select Research Limited mentioned above) are twofold, namely: i) the requirement to use a 3D scanner makes these methods only moderately affordable and accessible; and ii) the use of linear regression based equations in combination with body volumes limits their accuracy when applied to heterogeneous populations. Furthermore, linear regression has limited degrees of freedom and cannot model the infinite shape and composition variations of human bodies across the globe. Furthermore, none of the existing regression based inventions for body composition have enhanced anatomical information, nor anatomically valid body segmentation or actual human shape variations including shape features in their techniques, all of which are contemplated in the embodiments of the invention disclosed herein.

[0027] Thus, given the limited accuracy and complex setup required to provide current technologies, individuals, insurers, government agencies, health care providers, and others remain without affordable, ideal body composition monitoring tools for effectively monitoring and / or treating chronic health, musculoskeletal disorders, and more positively impacting premature mortality.

[0028] The present invention was developed in this context. Summary of the invention

[0029] It is an object of the present invention to overcome or ameliorate at least one or more disadvantages of the prior art, to provide a useful alternative, or to offer consumers, government agencies, and healthcare providers an informed or commercial choice.

[0030] Other objects and advantages of the present invention will become apparent from the following description taken in conjunction with the accompanying drawings wherein, by way of illustration and example, preferred embodiments of the present invention are disclosed.

[0031] According to a first broad aspect of the present invention, there is provided a device for analyzing a subject, the device comprising:

[0032] Controller;

[0033] a memory device storing electronic program instructions for operating the controller; and

[0034] Input components;

[0035] The controller is operable and directed by the instructions of the electronic program to:

[0036] receiving input via an input component, the input comprising at least one representation of a subject;

[0037] processing the input to perform an analysis of the subject and generating an output based on the analysis, the processing including using a database; and

[0038] Transmit output.

[0039] The representation may be in the form of any type of numbers and / or text and / or data and / or images.

[0040] Using a database may include accessing and / or querying the database.

[0041] Optionally, the system includes a display for displaying a user interface, and the controller is operable under the control of the electronic program instructions to transmit the output by displaying the output via the display. In this manner, the visualization of the analysis can be depicted by at least one of text, image, grid, 3D, video, icon, virtual reality and graphics.

[0042] Optionally, the device is an implementation of the system.

[0043] Optionally, the input includes details of the subject. These details may include data and / or information of the subject, data and / or information associated with the subject, and / or data and / or information related to the subject. The data and / or information may be obtained by one or more of retrieving, receiving, extracting, and identifying data and / or information from one or more sources.

[0044] Optionally, the subject is a human body, or one or more parts thereof. In such a case, where the human body is an individual person, the output may include an estimate of the individual's 3D body shape and at least one of its associated anthropometry, body composition, and health-care risks. The output may include an estimate of the individual person's: shape, physical and ecological characteristics and / or at least one three-dimensional (3D) shape and its associated anthropometry, body composition, and health-care risks.

[0045] Optionally, the subject is the body of a living being, or one or more parts thereof.

[0046] Optionally, the subject is the body of a non-living object, or one or more parts thereof.

[0047] Optionally, the subject is a human body.

[0048] The input component may include at least one sensor, which may be part of a sensor system or a group of sensors.

[0049] Optionally, the representation includes a visual, non-visual and / or non-visible representation of the subject, which may include an image of the subject. In such an implementation, at least one sensor may include an imaging component that is operable to capture a visual, non-visual and / or non-visible representation of the subject. The imaging component may be a digital camera. The non-visual representation may refer to data that cannot be visualized, such as a matrix or vector of floating point numbers of the subject.

[0050] The individual sensors within a set of sensors may include: motion sensor; gyroscope sensor; gravity sensor, infrared sensor; depth sensor; three-dimensional imaging sensor; inertial sensor; microelectromechanical (MEMS) sensor; imaging component; acceleration sensor; orientation sensor; direction sensor; position sensor; and light source sensor.

[0051] Optionally, the representation includes one or more visual, non-visual, or invisible representations of the subject. In such an embodiment, the one or more sensors (if provided) may include one or more imaging components that are operable to capture one or more visual, non-visual, and / or non-visible representations of the subject. In addition, the one or more sensors may include an orientation sensor that is operable to provide orientation data for use during the capture of the one or more visual, non-visual, and / or invisible representations of the subject to facilitate alignment thereof with a plane for improved accuracy.

[0052] Optionally, subjects belong to categories and / or groups and / or classes.

[0053] In this case, the database may include details of a plurality of different subjects belonging to the same and / or similar categories and / or groups and / or classes as the subject being analyzed. These details may include data and / or information for each of the plurality of different subjects, data and / or information associated with each subject and / or data and / or information related to each subject.

[0054] Where the category and / or group and / or class includes humans, for each human participant among a plurality of human participants who contribute details of their respective bodies to the database, the data and / or information may include one or more of human medical imaging, body composition, anthropometry, participant characteristics, participant attributes, medical physiology and epidemiological information collected at the same time as and linked to a representation of each participant's respective body.

[0055] Where the category and / or group and / or class includes humans, the data and / or information comprises, for each human participant of a plurality of human participants contributing their respective body details to said database, one or more human videos, photographs, full and / or partial body shape or surface scans, medical imaging, body composition, anthropometry, participant characteristics, participant attributes, medical physiology and epidemiological information, which data and / or information have been collected at the time of collection and are linked and / or registered and / or aligned and / or matched to a unified representation of each participant's respective body.

[0056] Optionally, collecting the details and representation of each participant's respective body includes capturing the details and representation of the body of each participant contributing to the database according to quality assurance protocols and pre-processing procedures and steps.

[0057] Optionally, the collection of details of each participant and representation of the respective body is preferably done simultaneously and in parallel, but may be done at different times. Each participant may do this.

[0058] Optionally, additional data is collected at different time intervals to enable development of temporal modeling of participant (human)(s) shapes and data collected over time, and to derive statistically meaningful diagnostic trackers, as well as to enable physically meaningful risk analysis and / or trending.

[0059] The data and / or information may include sensory input or intrinsic and / or extrinsic data and / or information of the device.

[0060] Optionally, details of database contributors are captured according to quality assurance and pre-processing steps and protocols.

[0061] Optionally, the collection of details and representations of each participant's respective body comprises capturing details and representations of the body of each participant contributing to the database according to quality assurance and pre-processing steps.

[0062] The quality assurance and pre-processing steps may include techniques related to the capture phase of human images, which may be in the form of images, videos or video frames, 2.xD images including depth images and / or 3D body scans for the database.

[0063] The quality assurance and pre-processing steps may include techniques related to the capture phase of the anthropometric data of the database.

[0064] Quality assurance and pre-processing steps may include techniques related to the capture phase of dynamic data for the database.

[0065] Optionally, the processing includes actions performed by statistical analysis and / or at least one computer vision (CV), machine learning (ML) and / or artificial intelligence (AI) model.

[0066] The action may include at least one of the following:

[0067] Analyze the collected data and images using advanced multidimensional statistical techniques and / or machine learning techniques and / or artificial intelligence techniques, and exclude those data and images identified as outliers;

[0068] Classify, cluster, group data and images into one or more of groups, regions, types, and categories based on one or more labeling mechanisms for said data types and / or associated outputs linked to the data types;

[0069] Estimating features and / or body surface landmarks and / or anatomically relevant body surface landmarks and / or key points and / or joint centers and / or bone links from the body image;

[0070] estimating body part segmentation from images of the body constrained by anatomical features extracted from medical imaging and establishing 2D and / or 3D body parts and / or 2D and / or 3D regions of interest (ROIs); and

[0071] 2D and / or 3D image and shape features are extracted as well as identified image segmentation(s) and / or image clusters and body shapes.

[0072] The action may include at least one of the following:

[0073] Analyze the collected data and images using advanced multidimensional statistical techniques and / or machine learning techniques and / or artificial intelligence techniques, and exclude those data and images identified as outliers;

[0074] Classify, cluster data and images into one or more of groups, regions, types, characteristics and categories based on one or more labeling and / or annotation mechanisms of data types and / or associated outputs linked to data types;

[0075] Estimating salient features and / or body surface markers from any type of image of a body, the salient features and / or body surface markers including anatomically relevant body surface markers and / or key points and / or joint centers and / or bone links;

[0076] Identify and segment the body from background clutter in an image;

[0077] estimating body part segmentation based on an image of the body constrained by anatomical features extracted from medical imaging and establishing a region of interest (ROI); and

[0078] Image or overall shape features and identified image segmentation(s) and / or image clusters are extracted and their correspondence with the body data is estimated.

[0079] The processing may include processing techniques where a CV / ML / AI model estimates subject or user specific anatomically relevant body surface landmarks, bone links and / or joint centers from human imaging data; video, 2.xD or 3D images.

[0080] The processing may include processing techniques in which an ML / AI model estimates body part segmentation from human images including 3D images constrained by anatomical features extracted from medical imaging, and establishes subject or user specific 2D and / or 3D regions of interest.

[0081] The processing may include processing techniques where the ML / AI model extracts 2D and / or 3D image and shape features and labeled (multiple) image segmentations and / or image clusters and corresponding 2D and / or 3D body shapes and / or body segmentations.

[0082] The processing may include processing techniques where the ML / AI model links, correlates, and / or co-registers human imaging data, participant characteristics and attributes, and medical physiological data with in vivo measurements of individual body composition and physical anthropometric methods. Such processing may include using at least one algorithm trained to accurately estimate various body composition and anthropometric measurements from individual or user-specific human imaging data, participant characteristics and attributes, and medical physiological data.

[0083] The processing may include processing techniques where the ML / AI model links / learns the healthcare scores / activities with databases, 2D and / or 3D body images, shapes, body parts, body segmentations and / or medical data.

[0084] The process may include steps where the user's estimated body composition and anthropometry are displayed as output and / or saved as a file and / or transmitted via a secure web link.

[0085] The process may include steps where current and historical body composition and body measurements are retrieved from storage in the device and / or cloud and / or server and then tracked and displayed over time as output.

[0086] The process may include steps in which various health care indicators and risk factor comparisons are provided as output, which may include, but are not limited to, one or more of the following:

[0087] i) estimates of measurements such as total or fractionated body fat, lean mass, visceral fat, androgenetic fat, gynecological fat, bone mass;

[0088] ii) categories of measurements, such as categories for body fat, e.g., lean, normal, overweight, and obese; and for visceral fat, e.g., low risk, normal risk, increased risk, high risk;

[0089] iii) comparison with the standard population distribution in the database;

[0090] iv) Estimation based on trained ML / AI models and predictive risk methods or equations for specific users:

[0091] a. The user is at risk of prediabetes;

[0092] b. They increase the risk of chronic health conditions, musculoskeletal disorders, mortality and falls;

[0093] v) Methods for detecting sarcopenia, indicated by the user or triggered by an estimated low lean muscle mass to height ratio or other relevant anthropometric results.

[0094] The processing may include outputting and displaying methods for implementing the customized intervention. This may be useful in promoting positive health behavior changes, thereby improving health and wellness outcomes for individuals.

[0095] The optional display, user interface and input components may be integrated, for example, in a touch screen. Alternatively, they may be discrete.

[0096] In one embodiment, the input comprises a user instruction input by a user via the input component. The user instruction may comprise a command to perform an action, in which case the controller is operable under the control of the electronic program instructions to perform the action in accordance with the received user instruction.

[0097] In one embodiment, the electronic program instructions comprise software. The device may be a mobile communication device, in which case it may comprise a smartphone, a laptop / tablet / desktop computer, a camera, or a portable media device on which the software is installed. The software may be provided as a software application downloadable to the device, and / or run as a service on a server and / or cloud.

[0098] Preferably, the operations performed by the device occur automatically without the need for human intervention.

[0099] According to a second broad aspect of the present invention, there is provided a method for analyzing a subject, the method comprising:

[0100] storing electronic program instructions for controlling the controller; and

[0101] The controller is controlled by electronic program instructions to:

[0102] receiving input via an input component, the input comprising at least one representation of a subject;

[0103] Input is processed to perform analysis of the subject and output is generated based on the analysis, the processing including use of a database.

[0104] The representation includes a visual representation including an image of the subject and / or a non-visual representation, wherein the visual representation refers to data that cannot be visualized.

[0105] The processing includes actions performed by at least one computer vision (CV) method, machine learning (ML), and / or artificial intelligence (AI) model.

[0106] Optionally, the subject being analyzed belongs to a category and / or group and / or class, and the database includes details of multiple different subjects belonging to the same and / or similar category and / or group and / or class as the subject being analyzed, the details including data and / or information of each of the multiple different subjects, data and / or information associated with each of the multiple different subjects, and / or data and / or information related to each of the multiple different subjects.

[0107] Where the categories and / or groups and / or categories include humans, for each of a plurality of human participants who contribute details of their respective bodies to the database, the data and / or information includes one or more human videos, photographs, full and / or partial body shape or surface scans, medical imaging, body composition, anthropometry, participant characteristics, participant attributes, medical physiology, and epidemiological information, the data and / or information having been collected at the same time as the collection and linked, and / or registered, and / or aligned, and / or matched to a unified representation of each participant's respective body.

[0108] Optionally, the collection of details and representations of each participant's respective body preferably occurs simultaneously and in parallel, but may be completed at different times.

[0109] Optionally, additional data is collected at different time intervals to enable the development of temporal modeling of participant (human) (multiple) shapes and data collected over time, and to derive statistically meaningful diagnostic trackers, as well as to enable physically meaningful risk analysis and / or trends.

[0110] Optionally, the collection of details and representations of each participant's respective body includes capturing the details and representations of each participant's body contributing to the database according to quality assurance protocols and pre-processing procedures and steps.

[0111] The processing may include actions performed by statistical analysis and / or at least one computer vision (CV), machine learning (ML), and / or artificial intelligence (AI) model.

[0112] The action may include at least one of the following:

[0113] Analyze the collected data and images using advanced multidimensional statistical techniques and / or machine learning techniques and / or artificial intelligence techniques, and exclude those data and images identified as outliers;

[0114] Classify, cluster data and images into one or more of groups, regions, types, characteristics and categories based on one or more labeling and / or annotation mechanisms of data types and / or associated outputs linked to data types;

[0115] Estimating salient features and / or body surface markers from any type of image of the body, the salient features and / or body surface markers including anatomically relevant body surface markers and / or key points and / or joint centers and / or bone links;

[0116] Identify and segment the body from background clutter in an image;

[0117] estimating body part segmentation from images of the body constrained by anatomical features extracted from medical imaging and establishing regions of interest (ROIs); and

[0118] Image or overall shape features and identified image segmentation(s) and / or image clusters are extracted and their correspondence to body data is estimated.

[0119] The method may further include controlling the controller via the electronic program instructions to transmit the output by displaying the output via the display. In this manner, the visualization of the analysis may be depicted by at least one of text, image, grid, 3D, video, icon, virtual reality, and graphics.

[0120] The method may further comprise transmitting the output. The transmitting may comprise displaying the output via a display and / or as readable data on a file or printed.

[0121] Optionally, the subject is an individual human body and the output comprises an estimate of the individual human's 3D body shape and at least one of its associated anthropometry, body composition, and health care risks. The output may comprise an estimate of the individual human's: shape, physical and ecological properties and / or at least one three-dimensional (3D) shape and its associated anthropometry, body composition, and health care risks.

[0122] According to a third aspect of the present invention, there is provided a computer-readable storage medium having instructions stored thereon, which, when executed by a computing component, causes the computing component to execute the method according to the second aspect of the present invention as described above.

[0123] According to a fourth aspect of the present invention, there is provided a computing component programmed to execute the method according to the second aspect of the present invention as described above.

[0124] According to a fifth aspect of the present invention, there is provided a data signal comprising at least one instruction that can be received and interpreted by a computing system, wherein the instruction implements the method according to the second aspect of the present invention as described above.

[0125] According to a sixth aspect of the present invention, there is provided a system for analyzing a subject, the system comprising the apparatus according to the first aspect of the present invention as described above.

[0126] One embodiment provides a computer program product for performing the methods described herein.

[0127] One embodiment provides a non-transferable carrier medium for carrying computer executable code which, when executed on a processor, causes the processor to perform a method as described herein.

[0128] One embodiment provides a system configured to perform a method as described herein. BRIEF DESCRIPTION OF THE DRAWINGS

[0129] In order to more fully understand the invention and to enable it to be put into practice, although any other forms are possible within the scope of the invention, preferred embodiments of the invention will now be described by way of example only with reference to the accompanying drawings, in which:

[0130] Figure 1 Describes the ideal principal component measurement model, showing the connections required for an ideal principal component measurement instrument;

[0131] Figure 2 and Figure 3 A flow chart depicting user-performed actions of a first embodiment of a method of using a first embodiment of a system according to aspects of the present invention provides an overview of an apparatus for estimating body composition and health care risk from human images;

[0132] Figure 4 Describes the use Figure 2 and Figure 3 A general overview of the steps involved in data collection and processing of the BCT database 40 of the system;

[0133] Figure 5 Describes the Figure 2 and Figure 3 A more detailed description of the data collected by the BCT database of the system shown;

[0134] Figure 6a and Figure 6b Describes the Figure 2 and Figure 3 A general overview of systematic data, medical imaging, and human imaging quality control procedures;

[0135] Figure 7 Describes from Figure 2 and Figure 3 General overview of the anatomical landmarks in a medical image of the system process. The joint center position can be observed on the right. The region of interest can be observed on the left.

[0136] Figure 8 Depicted Figure 2 and Figure 3Training profile of the system’s ML / AI algorithms to estimate joint centers;

[0137] Fig. 9 Defined regions of interest and anatomical body surface landmarks are depicted;

[0138] Fig.10 Depicted Figure 2 and Figure 3 A systematic overview of predictive health risk equations for prediabetes, chronic health conditions, and mortality;

[0139] Fig.11 Depicted Figure 2 and Figure 3 Examples of calculations of the system's relative health risk calculator;

[0140] Fig.12 depicts a schematic diagram of an embodiment of an apparatus according to an aspect of the present invention;

[0141] Fig.13 Depicted Figure 2 and Figure 3 A simplified system diagram of the system;

[0142] Fig.14a , Fig.14b , Fig.14c and Fig.14d Depicts example images used to train a CV / ML / AI model that can be operated to Figure 2 and Figure 3 The system estimates object or user specific anatomical information and joints from human images;

[0143] Fig.15a and Fig.15b Describes how specific anatomically relevant body landmarks and joint centers are estimated based on specific frontal and side profile human images using CV / ML / AI models. Figure 2 and Figure 3 The system is trained with data in the BCT database 40;

[0144] Fig.16 Depicts 3D segmentation for body part segmentation without using anatomical landmarks and joint centers;

[0145] Fig.17 depicts 3D segmentation using anatomical landmarks and joint centers for body part segmentation; and

[0146] Fig.18 An example of how one type of 2D image features may be prepared for a specific human image is depicted.

[0147] definition

[0148] The following definitions are provided as general definitions and in no way should the scope of the present invention be limited to these terms, but are set forth for a better understanding of the following description.

[0149] Unless otherwise defined, all technical and scientific terms used in this article have the same meaning as those of ordinary skill in the art to which the present invention belongs. It will be further understood that the terms used in this article should be interpreted as having a meaning consistent with their meaning in the context of this specification and the associated technology, and unless clearly defined as such in this article, it will not be interpreted as an idealized or overly formal meaning. For purposes of the present invention, additional terms are defined below. In addition, all definitions defined and used in this article should be understood to control dictionary definitions, definitions in documents incorporated by reference and / or the ordinary meaning of the defined terms, unless there is doubt about the meaning of a particular term, in which case the ordinary dictionary definition and / or common usage of the term will be preferred.

[0150] For purposes of the present invention, the following terms are defined below.

[0151] As used herein, the articles "a" and "an" are used to refer to one or more (ie, to at least one) of the grammatical object of the article. For example, "an element" refers to one or more elements.

[0152] The term "approximately" as used herein refers to an amount that varies by up to 30%, preferably up to 20%, and more preferably up to 10% relative to a reference amount. The use of the word "approximately" to qualify a number is merely a clear indication that the number should not be interpreted as an exact value.

[0153] In this specification, unless the context requires otherwise, "comprise", "comprises" and "comprising" will be understood to imply the inclusion of a stated step or element or a group of steps or elements but not the exclusion of any other step or element or a group of steps or elements.

[0154] Any of the terms used in this document: "including" or "which includes" or "that includes" is also an open term, which also means including at least the elements / features following the term, but not excluding other elements / features. Therefore, "including" is a synonym and meaning of "comprising".

[0155] In the claims, as well as in the summary above and the description below, all transitional phrases, such as "comprises," "comprising," "carrying," "having," "containing," "involving," "holding," "consisting of," etc., should be understood as open-ended, i.e., meaning "including but not limited to." Only the transitional phrases "consisting of" and "consisting essentially of" alone are closed or semi-closed transitional phrases, respectively.

[0156] The term "real time", as in "displaying real-time data", means displaying the data without intentional delay, subject to system processing limitations and the time required to accurately measure the data.

[0157] The term "near real-time", as in "obtaining real-time or near real-time data", means obtaining data without intentional delay ("real time") or as close to real time as possible (i.e., with small but minimal delay, intentional or not, within the constraints and processing limitations of the systems acquiring, recording, or transmitting the data).

[0158] Although any methods and materials similar or equivalent to the methods and materials described herein can be used in the practice or testing of the present invention, preferred methods and materials are described. It is understood that the methods, devices and systems described herein can be implemented in various ways and for various purposes. The description herein is only illustrative.

[0159] As used herein, the term "exemplary" is used in the sense of providing an example, rather than being indicative of a quality. That is, an "exemplary embodiment" is an embodiment provided as an example, rather than necessarily of exemplary quality, such as serving as a model for what is desired or representing the best of its kind.

[0160] The various methods or processes outlined herein may be encoded as software that can be executed on one or more processors using any of a variety of operating systems or platforms. In addition, such software may be written using any of a number of suitable programming languages ​​and / or programming or scripting tools, and may also be compiled into executable machine language code or intermediate code that is executed on a framework or virtual machine.

[0161] In this regard, various inventive concepts may be embodied as a computer-readable storage medium (or multiple computer-readable storage media) encoded with one or more programs (e.g., computer memory, one or more floppy disks, compressed disks, optical disks, tapes, flash memory, circuit configurations in field programmable gate arrays or other semiconductor devices, or other non-transient media or tangible computer storage media), which, when executed on one or more computers or other processors, performs methods for implementing various embodiments of the present invention described above. One or more computer-readable media may be transportable so that the programs stored thereon can be loaded onto one or more different computers or other processors to implement various aspects of the present invention as described above.

[0162] Herein, the term "program" or "software" is used in a general sense to refer to any type of computer code or computer executable instruction set that can be deployed on a programmed computer or other processor to implement various aspects of the embodiments described above. In addition, it should be understood that, according to one aspect, one or more computer programs that perform the methods of the present invention when executed do not need to reside on a single computer or processor, but can be distributed in a modular manner between multiple different computers or processors to implement various aspects of the present invention.

[0163] Computer executable instructions can be in many forms, such as program modules, that are executed by one or more computers or other devices. Typically, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. Typically, in various embodiments, the functionality of program modules can be combined or distributed as needed.

[0164] Moreover, the data structure can be stored in a computer-readable medium in any suitable form. For simplicity of illustration, the data structure can be shown as having fields that are related by position in the data structure. Such a relationship can also be achieved by allocating storage for fields with positions that convey the relationship between the fields in the computer-readable medium. However, any suitable mechanism can be used to establish a relationship between the information in the fields of the data structure, including by using pointers, tags, or other mechanisms that establish a relationship between data elements.

[0165] Furthermore, various inventive concepts may be embodied as one or more methods, of which an example has been provided. The actions performed as part of a method may be ordered in any suitable manner. Thus, embodiments may be constructed in which actions are performed in an order different from that shown, which may include performing some actions simultaneously, even though shown as sequential actions in an illustrative embodiment.

[0166] The phrase "and / or" as used in the specification and claims should be understood to mean "either or both" of the elements so connected, i.e., elements that are present connected in some cases and separated in other cases. Multiple elements listed with "and / or" should be interpreted in the same manner, i.e., "one or more" of the elements so connected. In addition to the elements specifically identified by the "and / or" clause, other elements may optionally be present, whether or not they are related to the specifically identified elements. Thus, as a non-limiting example, in one embodiment, when used in conjunction with an open language such as "comprising", a reference to "A and / or B" may refer only to A (optionally including elements other than B); in another embodiment, to only B (optionally including elements other than A); in yet another embodiment, to both A and B (optionally including other elements); and so on.

[0167] As used in the specification and claims herein, it should be understood to have the same meaning as the "and / or" defined above. For example, when separating items in a list, "or" or "and / or" should be interpreted as including, that is, including at least one of multiple elements or lists, but also including more than one, and optionally including additional unlisted items. Only terms that clearly indicate the opposite, such as "only one" or "exactly one of them", or when used in the claims, "consisting of..." will refer to including exactly one element in a list of multiple elements or multiple elements. In general, the term "or" used in this article should only be interpreted as indicating an exclusive choice (i.e., "one of them", not "both"), such as "any one", "one of them", "only one" or "exactly one". "Mainly consisting of..." used in the claims should have the general meaning used in the field of patent law.

[0168] As used in the specification and claims herein, when referring to a list of one or more elements, the phrase "at least one" is understood to mean at least one element selected from any one or more elements in the list of elements, but does not necessarily include at least one of each element specifically listed in the list of elements, and does not exclude any combination of elements in the list of elements. This definition also allows that in addition to the elements specifically identified in the list of elements to which the phrase "at least one" refers, elements may optionally be present, whether or not they are related to the specifically identified elements. Thus, as a non-limiting example, in one embodiment, "at least one of A and B" (or equivalently, "at least one of A or B", or equivalently "at least one of A and / or B") may refer to at least one, optionally including more than one A, without B (and optionally including elements other than B); in another embodiment, at least one, optionally including more than one, without A (and optionally including elements other than A); in yet another embodiment, at least one, optionally including more than one A and at least one, optionally including more than one B (and optionally including other elements); and so on.

[0169] For the purposes of this specification, if method steps are described in a sequence, that sequence does not necessarily imply that the steps are performed in that chronological order, unless there is no other logical way to explain that sequence.

[0170] In addition, where features or aspects of the invention are described in terms of Markush groups, those skilled in the art will recognize that the invention is also thereby described in terms of any individual member or subgroup of members of the Markush group. DETAILED DESCRIPTION

[0171] In the following description it should be noted that the same or identical reference numerals in different embodiments represent the same or similar features.

[0172] exist Figure 2 and Figure 3 In the figure, it is shown that the apparatus 12 according to various aspects of the present invention analyzes the subject Figure 2 (System) Actions performed during use of the first embodiment of the system 10 shown.

[0173] In the depicted embodiment, the subject is a human 16 , who is a subject 14 and is a user of the system 10 .

[0174] The present invention will make particular reference to analyzing the human body to provide an estimate of an individual's three-dimensional (3D) body shape and its associated body composition and health care risks using, without limiting the generality, one or more of images and / or features and / or any type of data, representation or information that is capable of defining, explaining or describing one or more of ecologically and physically valid models or analogies of body shape and its molecular composition. Thus, embodiments of system 10 are used to estimate shape, body composition and health risks from human images.

[0175] That is, it should be understood that the present invention may be used with the bodies of other subjects, and for performing analyses for additional and / or alternative purposes or objectives.

[0176] It should be understood that the present invention is not limited to the subject being analyzed, or the purpose or manner in which it is analyzed, and in alternative embodiments, the present invention can be applied to analyze the body of additional and / or alternative objects for the additional and / or alternative purposes described. Depending on the implementation, the subject can be the body of a living organism, or one or more parts thereof, or the subject of a non-living organism, or one or more parts thereof. Embodiments of the present invention are particularly suitable for analyzing the subject of something where there is a difference between the subject of one and the subject of another, such as animals (including livestock) and humans (as in the described embodiments).

[0177] As will be described in further detail, an embodiment of the present invention includes a system 10 that implements a method for estimating an individual's three-dimensional (3D) body shape and its associated body composition and health care risks from human images that the inventor has developed. The method includes using various data capture devices to collect a unique and diverse global multidimensional database of details of the human body of a participant or contributor, including human images, medical imaging, medical physiology, participant characteristics and attributes, anthropometrics, and epidemiological data. Then, new and advanced statistical methods, computer vision (CV), machine learning (ML), and artificial intelligence (AI) methods use the database to understand human images, extract unique features and characteristics from them, and associate them with human body shape, body composition, medical physiology, and health care risks. In an embodiment, the entire process described above is facilitated to be performed offline under a strict quality assurance protocol to produce an independent system or application that can be used by the general public and scientists. The system 10 of an embodiment of the present invention provides an affordable and accessible method for estimating body composition and health care risks within the medical and health, fitness, insurance, and government sectors.

[0178] The advanced machine learning technology used by the embodiments of the present invention may include convolutional neural network (CNN) technology.

[0179] The protocols and processes developed by the inventors in this invention allow it to handle this complex type of human data types and structures in embodiments.

[0180] Conceptually, embodiments of the present invention can be seen to include six broad levels. As will be described in further detail, some levels are performed in an offline phase to facilitate actions including capturing, controlling, handling, processing, and linking all types of collected data required to produce or implement the present invention, and in embodiments, other levels are performed in an online phase to facilitate actions in which subsets of data, such as (multiple) human images, are captured, processed, and output or used to predict other subsets (e.g., output body composition).

[0181] With respect to the six levels, the first offline level involves collecting a large and diverse database of human physical details of participants or contributors around the world, including human medical imaging, body composition, anthropometrics, physical characteristics and attributes, and medical physiology and epidemiology data, while collecting human images using various types of data capture devices. An advantageous feature of an embodiment of the present invention is to provide a system 10 that is operable to collect, prepare, analyze, and process a large database of human visual images, medical imaging, body composition, anthropometrics, physical characteristics and attributes, medical physiology, and epidemiology data while simultaneously collecting human images.

[0182] The second offline level involves developing methodological procedures for the collection of data that provide a reliable and uniform database, and incorporating new and advanced image processing techniques and data quality control procedures to facilitate actions including evaluation, correction, analysis, preparation and linking of the collected multidimensional data.

[0183] The third offline level involves the development of computer vision (CV), machine learning (ML) and artificial intelligence (AI) methods and models that are operable to facilitate actions, including studying, analyzing and linking human images and related human features, participant characteristics and medical physiology, and linking them with body composition data, anatomical bone joints (from medical images), anatomically valid body part segmentations (from medical images) and 3D body shape (reconstructed from images or obtained via 3D shape scanners).

[0184] The actions may include at least one of the following:

[0185] Analyze the collected data and images using advanced multidimensional statistical techniques and / or machine learning techniques and / or artificial intelligence techniques, and exclude those data and images identified as outliers;

[0186] Classify, cluster, group data and images into one or more of groups, regions, types, and categories based on one or more labeling mechanisms for data types and / or associated outputs linked to data types;

[0187] Estimating features and / or body surface landmarks and / or anatomically relevant body surface landmarks and / or key points and / or joint centers and / or bone links from the body image;

[0188] Estimate body part segmentation from image features extracted from medical imaging and / or images of the body constrained by anatomical features, and create 2D and / or 3D body part and / or

[0189] or 2D and / or 3D regions of interest (ROI); and

[0190] 2D and / or 3D image and shape features are extracted as well as labeled image segmentation(s) and / or image clusters and body shapes.

[0191] The actions may include at least one of the following:

[0192] Analyze the collected data and images using advanced multidimensional statistical techniques and / or machine learning techniques and / or artificial intelligence techniques, and exclude those data and images identified as outliers;

[0193] Classify, cluster data and images into one or more of groups, regions, types, characteristics and categories based on one or more labeling and / or annotation mechanisms of said data types and / or associated outputs linked to said data types;

[0194] estimating salient features and / or body surface markers from any type of image of said body, said salient features and / or body surface markers including anatomically relevant body surface markers and / or key points and / or joint centers and / or bone links;

[0195] identifying and segmenting the body from background clutter in the image;

[0196] estimating body part segmentation from images of the body constrained by anatomical features extracted from medical imaging and establishing regions of interest (ROIs); and

[0197] Extract image or overall shape features and mark (multiple) image segmentation and / or

[0198] or image clusters, and estimate their correspondence with body data.

[0199] It should be understood that CV / ML / AI is not limited to the models of the embodiments described herein, and in alternative embodiments of the invention, additional and / or alternative models may be learned and outputs generated depending on the body or object to be analyzed. Constraints on the learning process may include additional and / or alternative training, testing and validation, more or less data, and (multiple) additional and / or alternative types of data compared to the described embodiments, as appropriate for implementation of the invention, depending on the body or object to be analyzed and the decisions to be made.

[0200] The fourth online level involves using the device 12 to capture inputs including human images and related human features, characteristics, and medical physiology. In addition to many control modules and applications integrated with novel CV, ML, and AI methods, the device 12 is then operable and used to facilitate actions including handling and processing the images to extract unique image features, body surface landmarks, key points (visible as landmarks, or unmarked, i.e., invisible), estimate human joints, bone links, body part segmentation, and reconstruct effective 3D body shape.

[0201] The fifth online level involves the use of additional CV, ML, and AI models and methods (driven by offline trained data and machine learning models) to facilitate actions that include processing human images and related human features, participant characteristics, medical physiology, and epidemiological data captured and processed online to estimate body composition, health care risks, and any other parameters in the offline collected datasets.

[0202] The sixth online level involves the use of processed human images and related anthropometric features, participant characteristics and medical physiology, three-dimensional (3D) body shape and estimated body composition data, and additional health and wellness data, all of which are analyzed, studied and modeled as outputs from the fifth level above to facilitate actions, including actions that allow individuals to classify, assess and monitor their health and wellness throughout their lifetime.

[0203] Although the present invention will be described with reference to estimating an individual's body composition, anthropometrics, and health care risks from human imaging data and other individual personal data for medical and healthcare, fitness, insurance, and government sectors; it should be understood that it can be used for other objects and for additional and / or alternative purposes. In addition, although embodiments of the present invention achieve the link between central (i.e., male, female, and visceral fat) and total body fat and lean muscle mass and an individual's chronic health disease risk, such as diabetes, cardiovascular disease, musculoskeletal diseases, such as frailty, knee and hip osteoarthritis, and prediabetes, premature death, and sarcopenia, it should be understood that a wider range of body composition, anthropometric variables, and medical physiology can be estimated and linked to other relevant conditions, disorders, or diseases for the medical, health, and medical research sectors, more broadly categorized as personal health care risks. These body composition, anthropometric variables, and medical physiology data include, but are not limited to, for example, organ tissue, muscle tissue, bone mineral density, blood pressure, heart rate, blood glucose, HbA1c, and other relevant blood / plasma information.

[0204] The device 12 is carried by a person who is a user 16 .

[0205] like Fig.12 As shown, the device 12 includes a plurality of components, subsystems and / or modules that are operably coupled via appropriate circuits and connections to enable the device 12 to perform the functions and operations described herein. The device 12 includes appropriate components necessary to receive, store and execute appropriate computer instructions, such as a method for analyzing a body according to an embodiment of the present invention.

[0206] In particular, if Fig.12 As shown, the device 12 includes a computing component, which in this embodiment includes a controller 18 and a storage device 20 for storing electronic program instructions for controlling or operating the controller 18, as well as information and / or data; a display 22 for displaying a user interface 24; and an input component 26; all of which are housed in a container or housing 28.

[0207] As will be described in further detail, the controller 18 is operable, under the control of electronic program instructions or otherwise directed, to: receive input via an input component, the input including at least one representation of the subject 14; process the input to analyze the subject and generate output based on the analysis, the processing including using a database; and transmit output by displaying the output via the display 22.

[0208] In particular, in this embodiment, the controller 18 is operable to control and run an application for estimating shape and health risk from images of a human being.

[0209] As will be described in further detail, the at least one representation may be in the form of any type of numbers and / or text and / or data and / or images.

[0210] Using a database may include accessing and / or querying the database.

[0211] In embodiments of the present invention, input may include data and / or information that may be obtained by capturing, retrieving, receiving, extracting, and identifying from one or more sources. One or more data sources may reside on storage device 20, and / or elsewhere remote from device 12. Data and / or information may include intrinsic and / or extrinsic data and / or information derived from sensory input or a device.

[0212] In an embodiment, the subject 14 belongs to a category and / or group and / or class. The database may include details of a plurality of different subjects belonging to the same and / or similar categories and / or groups and / or classes as the subject being analyzed. These details may include data and / or information for each of the plurality of different subjects, data and / or information associated with each subject and / or data and / or information related to each subject.

[0213] Where the categories and / or groups and / or classes include humans, as in an embodiment, for each human participant of a plurality of human participants who contribute details of their respective bodies to the database, the data and / or information may include one or more of human medical imaging, body composition, anthropometry, participant characteristics, participant attributes, medical physiology, and epidemiological information collected at the same time as the representation of each participant's respective body is collected and linked to.

[0214] Where the categories and / or groups and / or classes include humans, the data and / or information may include, for each human participant among a plurality of human participants who contribute their respective body details to the database, one or more human videos, photographs, full and / or partial body shape or surface scans, medical imaging, body composition, anthropometry, participant characteristics, participant attributes, medical physiology and epidemiological information that has been collected at the time of collection and linked, and / or registered, and / or aligned, and / or matched to a unified representation of each participant's respective body.

[0215] The controller 18 comprises processing means in the form of a processor.

[0216] The memory 20 includes a read only memory (ROM) and a random access memory (RAM).

[0217] Device 12 is capable of receiving instructions that may be held in ROM or RAM and that may be executed by a processor. The processor is operable to perform actions under the control of electronic program instructions, including processing / executing instructions and managing data and information flow through device 12, as will be described in further detail below.

[0218] In this embodiment, electronic program instructions for device 12 are provided via a single standalone software application (app) or module, which may be referred to as a subject analysis app and / or a software development kit (SDK) included or executed from within other applications and / or services running on a server and / or cloud. In the described embodiment, the app and / or SDK and / or service are provided under the trademark BCT TM Marketing is performed and may be downloaded from a website (or other suitable electronic device platform), or otherwise saved to or stored on the storage device 20 of the device 12, and / or executed via an application programming interface (API).

[0219] In a preferred embodiment of the present invention, device 12 is a mobile communication device and comprises a smartphone, such as a smartphone sold under the trademark Apple Inc. or by other vendors such as Nokia Corporation or Samsung Group. Device 10 may alternatively include other computing components, such as a personal, notebook or tablet computer, such as those sold under the trademarks of Apple Inc. or by other providers such as Hewlett-Packard Co. or Dell Inc. or IPOD Computers for sale, or other suitable equipment.

[0220] The device 12 also includes an operating system that is capable of issuing commands and is arranged to interact with the application to cause the device 12 to perform actions including corresponding steps, functions and / or procedures according to the embodiments of the present invention described herein. The operating system may be suitable for the device 12. For example, when the device 12 includes In the case of a smartphone, the operating system may be iOS.

[0221] like Fig.13 As shown, the device 12 is operable to communicate via one or more communication links 30, which may be variously connected to one or more remote devices 32, such as a server, a personal computer, a terminal, a wireless or handheld computing device, a fixed line communication device, or a mobile communication device, such as a mobile (cell) phone. At least one of the plurality of communication links 30 may be connected to an external computing network via a telecommunications network.

[0222] In the depicted embodiment, remote devices 32 include other devices 12 owned and / or operated by other persons, as well as computing systems 34 owned and operated by an administrator.

[0223] In the present embodiment, administrator computing system 34 is in the form of server 36. Server 36 may be used to execute applications and / or system services, such as systems and methods for analyzing subjects according to embodiments of the present invention.

[0224] In an embodiment, server 36 is implemented via cloud computing and maintained on a cloud-based platform with appropriate resources and infrastructure. In an alternative embodiment, it may be physically located at a centrally managed administrative center.

[0225] Similar to device 12, the infrastructure supporting server 36 includes appropriate components necessary to receive, store and execute appropriate electronic program instructions. These components include processing means in the form of a server processor, server storage including read-only memory (ROM) and random access memory (RAM), one or more server input / output devices such as disk drives, and an associated server user interface. Telecommunications device 32 (including device 12) is arranged to communicate with server 36 via one or more communication links 30.

[0226] The server 36 is capable of receiving instructions that may be retained in ROM, RAM or a disk drive and that may be executed by a server processor. The server processor is operable to perform actions under the control of the electronic program instructions, as will be described in further detail below, including processing / executing instructions and managing data and information flow through the computing system 34.

[0227] The server 36 includes a server operating system that is capable of issuing commands to access a plurality of databases or databanks residing on its storage devices. In this embodiment, two such databases or databanks are provided, including: one of the registered users (RUs) of the system 10, which may be referred to as the RU database 38; and one of the above databases, which may be referred to as the BCT database 40. The operating system is arranged to interact with the databases 38 and 40 and with one or more computer programs of a set / set of server software to cause the server 36 to perform actions including corresponding steps, functions and / or procedures according to the embodiments of the present invention described herein.

[0228] Computer programs for applications, server software suites, and other electronic instructions or programs for computing components of device 12 and server 36 may be written in any suitable language, as is known to those skilled in the art. The operation on the device 12 of the smart phone, the main analysis application can be written in Objective-C language. In an embodiment of the present invention, the electronic program instructions can be provided as (multiple) independent applications, as a group or multiple applications, via a network, or added as middleware, depending on the requirements of the implementation or embodiment.

[0229] In an alternative embodiment of the present invention, the software may include one or more modules and may be implemented in hardware. In this case, for example, the modules may be implemented using any one or combination of the following technologies, which are well known in the art: (multiple) discrete logic circuits with logic gates for implementing logic functions on data signals, application specific integrated circuits (ASICs) with appropriate combination of logic gates, (multiple) programmable gate arrays (PGAs), field programmable gate arrays (FPGAs), etc.

[0230] The corresponding computing component can be any suitable type of system, including: a programmable logic controller (PLC); a digital signal processor (DSP); a microcontroller; a personal, laptop or tablet computer, or a dedicated server or a networked server.

[0231] The corresponding processor may be any custom or commercially available processor, central processing unit (CPU), data signal processor (DSP) or auxiliary processor among a plurality of processors associated with the computing device. In embodiments of the present invention, the processing component may be, for example, a semiconductor-based microprocessor (in the form of a microchip) or a macroprocessor.

[0232] In an embodiment of the present invention, the corresponding storage device may include any one or a combination of volatile memory elements (e.g., random access memory (RAM) such as dynamic random access memory (DRAM), static random access memory (SRAM)) and non-volatile memory elements (e.g., read-only memory (ROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic tape, compact disk read-only memory (CD-ROM), etc.). The corresponding storage device may include electronic, magnetic, optical and / or other types of storage media. In addition, the various storage devices may have a distributed architecture in which various components are remote from each other but can be accessed by the processing unit. For example, the ROM may store various instructions, programs, software or applications to be executed by the processing unit to control the operation of the device 12, and the RAM may temporarily store variables or results of the operation.

[0233] The use and operation of computers using software applications are well known to those skilled in the art and need not be described in further detail herein except as relevant to the present invention.

[0234] In addition, any suitable communication protocol may be used to facilitate connection and communication between any subsystem or component of device 12, any subsystem or component of server 36, and device 12 and server 36, as well as other devices or systems, including wired and wireless, which is well known to those skilled in the art and need not be described in further detail herein except as relevant to the present invention.

[0235] When the words "store," "hold," and "save," or similar words, are used in the context of the present invention, they should be understood to include references to retaining or maintaining data or information permanently and / or temporarily in a storage component, device, or medium for later retrieval, as well as temporarily or transiently, such as as part of a processing operation being performed.

[0236] Furthermore, when the terms "system," "device," and "machine" are used in the context of the present invention, they should be understood to include references to any group of functionally related or interacting, interrelated, interdependent or associated components or elements, which may be proximate, separate, integrated or separate from each other.

[0237] Furthermore, in the embodiments of the present invention, the term "determining" is understood to include receiving or accessing relevant data or information.

[0238] In an embodiment of the present invention, the display 22 for displaying the user interface 24 and the user input component 26 are integrated into the touch screen 42. In alternative embodiments, these components may be provided as discrete elements or items.

[0239] Touch screen 42 is operable to sense or detect the presence and location of a touch within the display area of ​​device 12. The sensed "touch" of touch screen 42 is input to device 12 as a command or instruction and is transmitted to controller 18. It should be understood that user input component 26 is not limited to including a touch screen, and in alternative embodiments of the present invention, any suitable device, system or machine for receiving input, commands or instructions and providing controlled interaction may be used, including, for example, a keypad or keyboard, a pointing device or composite device, and systems including voice activation, voice and / or thought control, and / or holographic / projected imaging.

[0240] Input may also be received via at least one sensor or a group of sensors 44 of the device 12 as part of a sensor system. Individual sensors within a group of sensors 44 may be operable to monitor, sense and capture or otherwise collect or measure sensor data and / or information associated or related to one or more characteristics, properties and parameters of the device 12, the surrounding environment or components, systems or devices associated or coupled thereto. For example, the group of sensors 44 may be operable to sense and collect sensor data related to the state of the device 12 and / or the state of the environment around the device 12. In one embodiment, the state of the device 12 includes the position of the device 12. In one embodiment, the state of the device 12 further includes the speed and / or velocity of the device 12. A group of sensors 44 includes an inertial sensor system including an accelerometer and an orientation sensor, a direction sensor and a position sensor. Alternative embodiments of the present invention may include additional and / or alternative sensors including motion sensors, infrared sensors, depth sensors, three-dimensional imaging sensors, inertial sensors, light source sensors and micro-electromechanical (MEMS) sensors.

[0241] The acceleration sensor is operable to measure the acceleration of the device 12 and generate acceleration data. For example, the acceleration sensor can be an accelerometer. The orientation sensor is operable to measure the rate of change of orientation (i.e., angular velocity) of the device 12 and generate orientation data. For example, the orientation sensor can be a gyroscope. The direction sensor is operable to determine the direction relative to the earth's magnetic poles and generate orientation data. For example, the direction sensor can be an electronic compass. The position sensor is operable to determine the position of the device 12 and generate position data. For example, the position sensor can be a global positioning system (GPS). The use and operation of such sensors are well known to those skilled in the art and do not need to be described in further detail herein except as relevant to the present invention.

[0242] At least one representation (which may be a first representation) may include one or more visual representations of the main subject 14. In the described embodiment, the first representation includes a set of visual representations of the main subject 14. Thus, the set of sensors 44 includes an imaging component in the form of a digital camera, operable to capture images or images including visual representations. In this embodiment, the camera is integrated with the device 12. The imaging component may include any suitable system or device that facilitates the acquisition of still and / or moving images. For example, in the device 12 includes In the case of a smartphone, the imaging component may be an iSight TM Camera. The use and operation of a camera is well known to those skilled in the art and need not be described in further detail here except as relevant to the present invention.

[0243] In this manner, the device 12 is operable to capture image(s) / imagery of any type or any human-shaped representation of the subject 14 of the user 16 .

[0244] In embodiments of the invention, the at least one representation may comprise a non-visual and / or non-visible representation of the subject.

[0245] The controller 18 may be operable via execution of an application such as an analysis app (in Figure 2 201 ) to collect and process user input related to the user 16 and the analysis being performed (at step Figure 2 202 of step 204). In embodiments, user input includes demographic information and other participant characteristics and attributes, medical physiology, and activity data. It also includes, but is not limited to, details such as gender, age, race, fitness, medical history, and medical physiology (e.g., heart rate, blood pressure, hemoglobin, blood glucose, and activity data are collected through operable connection with a smartwatch or other device). Although embodiments of the present invention do not require all of these inputs, the additional information provided to the AI / ML process will improve the accuracy of the body composition and anthropometric estimates of embodiments. Furthermore, in embodiments, with more information, individuals will advantageously have an improved level of accuracy in estimating their relative health care risk as well as their predicted health care risk.

[0246] Device 12 includes operatively connected / coupled components that facilitate performance and operation as described above, including appropriate computer chips (integrated circuits), transceiver / receiver antennas, and software for the sensing technology used.

[0247] One or more sensors in a set of sensors 44 may be integrated with device 12, such as including Alternatively, the device 12 may be operatively coupled to one or more sensors of the set of sensors 44 described above.

[0248] A device database 46 or data bank also resides on the storage device 20 and is accessible under the control of the app by the controller 18. The controller 18 is arranged to interact appropriately with the device database 46 to cause the device 12 to perform actions including corresponding steps, functions and / or procedures according to the embodiments of the invention described herein.

[0249] Details stored or maintained remotely, for example, in one or more remote database modules residing on respective storage devices of one or more remote systems or devices 32, such as the BCT database 40 of the server 36, are accessible by the device 12 via the one or more communication links 30. The controller 18 is arranged to facilitate user interaction with the one or more remote databases so that the remotely stored content is available as required.

[0250] It will be understood that any one of the described (multiple) databases may reside on any suitable storage device, which may include a solid-state drive, a hard disk drive, an optical drive, or a tape drive. The described (multiple) databases may reside on a single physical storage device, or may be distributed across multiple storage devices or modules.

[0251] As is well known to those skilled in the art, the device database 46 is coupled to the controller 18 and is in data communication with the controller 18 so that information and data can be read by and from the device database 46. Any suitable database structure may be used, and there may be one or more databases. In embodiments of the present invention, the device database 46 may be provided locally as a component of the device 12 (such as in the storage device 20) or remotely, such as on a remote server, electronic program instructions and any other data or information to be collected and / or presented.

[0252] Similarly, as is well known to those skilled in the art, both the RU and BCT databases 38 and 40 are coupled to the server 36 and are in data communication with the server 36, so that data can be read to and from the RU and BCT databases 38 and 40. Any suitable database structure may be used. Either or both of the RU and BCT databases 38 and 40 may be provided locally as a component of the server 36 (such as in a memory device), or remotely, such as on a remote server, and the server group software may also be provided remotely. In one embodiment, several computers may be set up in this manner to have a network client-server application. In the described embodiment, each of the RU and BCT databases 38 and 40 is stored internally in the memory device of the server 36 as a partition of a single database structure. In alternative embodiments of the present invention, there may be more or fewer databases.

[0253] Certain components of system 10 will now be described in further detail.

[0254] BCT Database 40

[0255] As will be described in further detail, the BCT database 40 comprises a unified large-capacity and diverse world database of human medical imaging, body composition, anthropometric measurements, participant characteristics and attributes, medical physiology, and epidemiological information collected simultaneously with each participant's human image. The BCT database 40 has a wealth of human intrinsic and extrinsic physical and health information.

[0256] The establishment of the BCT database 40 is a key component of the described embodiment of the invention. Unlike existing databases such as the world's largest and most recognized data sets (such as the National Health and Nutrition Examination Survey (NHANES), which collects some, but not all, of the aforementioned data types at different times), in embodiments of the present invention, multi-dimensional data types are collected simultaneously and in parallel for each participant who provides their physical details to the BCT database 40, while also collecting human images to form the basis of a unified BCT database 40 (in embodiments of the present invention, this can be done at different times or over a period of time to collect additional data). Reference will now be made to the accompanying drawings. Figure 4 and Figure 5 This is described in further detail.

[0257] In this embodiment, all human data collection is based on systematic, ethical and empirically validated, internationally recognized standards and data quality control procedures. This seeks to ensure that three important development requirements of the embodiment are achieved: i) high measurement fidelity between variables; ii) multi-level and secure data storage for large-scale and heterogeneous databases; and iii) accurate and repeatable predictive capabilities.

[0258] Embodiments of the present invention require the collection of physical details of a large heterogeneous sample of contributing human participants representing a global or world population so that CV, ML / AI methods and models can accurately estimate individual anthropometric and body composition. Participants who contribute details of their respective bodies to the BCT database 40 are stratified for a wide range of participant characteristics and attributes, including, for example, gender, age, race, fitness status, and health status. In this embodiment, a large sample of homogeneous cohorts in a heterogeneous, global population is required so that participant-standard deviations and covariances between unified measurements can be correctly modeled. The scale of these unified data collections is unique to embodiments of the present invention, advantageously providing a unified data collection protocol for collecting 2D, 3D, or 4D images while collecting medical physiology, participant characteristics and attributes, anthropometric and body composition, and epidemiological data from the world population.

[0259] This aspect of embodiments of the present invention is particularly advantageous in the medical and health, fitness, insurance, and government sectors, as other technologies in the broader medical technology field can only acquire accurate (102), assessable (101), and affordable (103) human body representation data when using expensive 3D scanning technology in a calibrated laboratory environment. In this regard, the embodiments of the present invention enable the use of the 3D scanning technology described above with reference to the present invention. Figure 1 All three aspects of the described ideal body composition estimation method.The fact that most, if not all, prior art techniques fail to achieve accuracy comparable to medical body composition scans motivates embodiments of the present invention.

[0260] In an embodiment of the present invention, for each participant contributing to the BCT database 40, six general measurements of their respective physical details are performed in parallel, which form the basis of the BCT database 40 (e.g. Figure 5 These details include:

[0261] i) participant characteristics and attributes (at step 501), including, for example, gender, age, race, health status, and medical history;

[0262] ii) participant anthropometrics (at step 502), including, for example, mass, height, waist, hips, chest, thigh, biceps, calf, and inseam, to name a few;

[0263] iii) including a representation of 2D, 3D, ... nD images or other body representation data (at step 503), including, for example, one or more of a photograph, a video, a depth image, a 3D scan or a 3D reconstructed point cloud or mesh;

[0264] iv) intrinsic and extrinsic parameters of the image capture device (at step 506), including, for example, one or more of focal length, 3D pose and orientation (e.g., gyroscope data), resolution, size and depth, field of view, rotation and translation relative to the coordinate system. In the case of medical imaging such as reference DEXA / DXA scanners, these parameters may include additional data such as data referenced to calibration patterns and phantoms, with scanner pixel sizes referenced to real-world measurements in, for example, centimeters or inches;

[0265] v) medical physiological measurements (at step 507), including, for example, one or more of heart rate, systolic and diastolic blood pressure, VO2 max, blood samples for, for example, blood glucose and blood lactate levels, body temperature, ventilation, and genome;

[0266] vi) medical imaging data (at step 508), including, for example, one or more of PET, CT, MRI, and DXA; and

[0267] vii) Epidemiological data (at step 509), including, for example, one or more of pre-diabetes, cardiovascular disease, osteoarthritis, mortality, fall incidence.

[0268] The measurements of the details recorded in the BCT database are said to be performed "in parallel" and "simultaneously". In the context of embodiments of the present invention, this generally refers to recording the data within a time period that is feasible for the data collected (preferably within a time period that is as short as possible) so that, for example, natural human processes such as drinking, eating, dehydration or feces do not affect the resulting measurements. For example, dehydration has been reported to affect the accuracy of body composition scanners, and in general, it is important to obtain the above body data within a short period of time to minimize or reduce any uncontrolled changes in body shape or weight, hence the terms simultaneously and measuring the data in parallel. Therefore, these expressions are not limited to the meaning of the same moment in real time, but rather the meaning of within a time period that is feasible for the data collected, so that natural human processes such as drinking, eating, dehydration or feces do not affect the resulting measurements.

[0269] In embodiments of the invention, additional data is collected at different time intervals to advantageously enable temporal modeling of the participant (human)(s) shapes and data collected over time to be created and / or developed, and to derive statistically meaningful diagnostic trackers, as well as to enable physically meaningful risk analysis and / or trends.

[0270] In the present embodiment, the BCT database 40 is a database that complies with the Health Insurance Portability and Accountability Act (HIPPA), and all data are collected, processed and stored in the BCT database 40 in accordance with the relevant guidelines outlined by the World Health Organization (WHO) in accordance with ethical standards and procedures for human research. In the present embodiment, all human data collected follows the ethical standards of national, local and institutional regulations and policies in the area where the human data is obtained. From a technical perspective, this advantageously allows the technology of the present embodiment to cluster and compare individuals with the global and local based on one or more criteria from the (being) collected human data.

[0271] In this embodiment, individual physical details of participating participants, including participant characteristics and attribute data such as gender, age, health status, medical history, and ethnicity, are collected and used for quality control of the data compiled within the BCT database 40. They are used for stratification and clustering of the data.

[0272] In this embodiment, as described above, anthropometric data such as segmented circumference and segmented length are recorded for each participant (at step 502). Each measurement follows the International Anthropometric Assessment Standard (ISAK). In this embodiment, each measurement in this regard is performed twice. If two measurements differ from each other by more than 2%, a third measurement is taken. The measurement closest to the third measurement is retained if it is within 2% of the third data point, and the other is discarded. In an embodiment, it is crucial to collect this data correctly because it is important for training and developing accurate and repeatable CV / ML / AI models and techniques. This advantageously allows estimates for the generation of body circumference measurements and 3D body shapes, and also allows external validation by experts in these estimated fields. Therefore, embodiments of the present invention advantageously combine anthropometrics in addition to 2D / 3D body shapes, medical physiology, and participant characteristic information for training ML / AI models. In addition, in additional tests, the measurements of qualified ISAK experts are subject to statistical tests, such as means differences, to determine any outliers or deviations, and score factors representing the confidence of certain measurements are assigned to the measurer. Such a process also facilitates the identification of robust and reliable data necessary for training, validating, and testing ML prediction and regression models.

[0273] Advantageously, in embodiments of the present invention, a representation in the form of a human image of the corresponding body of each participant contributing its body details is collected in parallel with other key data of details collected for the BCT database 40, particularly medical images. Although in the described embodiments, the process of capturing human 2D or 3D image data is implemented using a smartphone device (at step 503) as an affordable means, it should be understood that all processes and protocols are also applicable to human 2D, 3D or 4D image or body representation data collected by a wide range of devices, including, for example, 3D scanners, stereo, photography or video devices. These may include, but are not limited to, devices such as a) smart devices, b) tablets, c) PCs or laptops, d) web cameras, e) high and low resolution digital cameras, f) high and low resolution digital video recorders, g) high and low speed video (i.e., 50 Hz and 10,000 Hz), h) optoelectronic devices, i) infrared photography, j) thermal imaging and k) passive and active stereo motion capture systems. By 3D, it also means any 3D shape representation as well as any time-varying 2D representation. By 4D, it also means any time-varying 3D representation.

[0274] In this embodiment, during the offline phase, and when collecting 2D, 3D or 4D imaging, each participant is prepared and guided according to a protocol designed to achieve highly reliable photography and video recording, and a number of standards are followed. This includes controlling factors for the environment, such as lighting and background noise (e.g., (multiple) objects appearing in the same volume as the participant). All participants must wear uniform clothing. In an embodiment, this is form-fitting in nature and is made of materials that do not change the scanner data, allowing the imaging or body representation data to accurately reflect the body shape of the individual. It is important to note that in this embodiment, in the offline phase, these standards are important for accurate ground truth data, but in the online phase, individuals will be instructed to follow realistic and relaxed standards, and the invented novel CV, ML / AI methods will then handle the changing backgrounds and other challenging problems generated during the use of the system 10, which will be described in further detail herein.

[0275] During the offline phase, the focal length, orientation (pitch, roll and yaw), image resolution, depth and height of the 2D, 3D, nD imaging devices are also recorded. In fact, in embodiments, all intrinsic parameters of the device are usually recorded. These are important measures of the present embodiment because the data are required for translation and rotation or general projection, normalization and preprocessing of human images (at step 503) and for registration and alignment of medical images corresponding thereto (at step 510). This is another advantageous feature of embodiments of the present invention, in which the BCT database 40 registers human image data with medical image data for 2D and 3D segmentation of human image data.

[0276] In this embodiment, variables such as medical physiological measurements (at step 507) are also in detail measured in parallel with human imaging, medical imaging and body composition data. Medical physiological data include, but are not limited to, a) heart rate, b) systolic and diastolic blood pressure, c) VO2 max, d) blood samples for, for example, blood glucose, Hb1(Ac) and blood lactate levels, e) body temperature, f) ventilation and g) genome, and the medical physiological data of the participants' respective bodies are contributed to the BCT database 40. These parallel measurements again facilitate the embodiment of using medical physiological data, using human imaging, anthropometrics and epidemiological data in a unified module / system to predict an individual's body composition, pre-diabetes, chronic health and musculoskeletal diseases and mortality risk.

[0277] In this embodiment, it is an advantageous process that the medical imaging data is collected and registered / aligned in parallel with the human images, and in this embodiment, it is important that the two sets of data are registered to each other (for each respective participant body). From the medical imaging data (at step 508) are derived both: i) an image of the individual's full body skeletal data / body shape (at step 510) and ii) an image of the body composition data (at step 511). Importantly, in an embodiment, the medical imaging data contains both full body shape, skeletal and body composition data from a single calibrated medical imaging device. This uniquely and advantageously guides the estimation and segmentation procedures of joints and anatomical surface markers, as described in further detail below, which are used to estimate body composition from full body and segmented 2D images and 3D shapes of an individual.

[0278] In this embodiment, epidemiological data is prospectively obtained from each participant during data collection (cross-sectional) and at discrete time intervals (e.g., 1 year, 2 years, 4 years, 8 years, 16 years, etc.) after data collection. Epidemiological data of the same participant repeated over time advantageously allows the development of more accurate predictive health models, and will further correct errors that occur in the prediction equations currently used by health practitioners and researchers. Epidemiological data can be obtained from a variety of places, including, for example: 1) individual self-reports, 2) individual personal medical records, 3) government databases, etc., and 4) automatically detected data, for example, from a smartphone or via images and / or other processing. For self-reported data, in an embodiment, these will need to be verified by cross-reference with some type of medical record of the individual. In this embodiment, it is important that all epidemiological data are verified by recognized medical entities to seek to ensure that all predictive ML models and equations for prediabetes, chronic health diseases, musculoskeletal diseases, and mortality are robust and meet high levels of specificity and sensitivity. Epidemiological data may include, but are not limited to, diagnosis of prediabetes, diabetes, cardiovascular disease, cancer, mortality, fall events, significant loss of lean muscle tissue, development of osteoarthritis in joints, etc.

[0279] This is another advantageous feature of embodiments of the present invention. In this regard, most of the current prior art calculates body composition by fitting shapes and volumes to a 3D representation of an individual's body, and then uses a linear regression method with limited degrees of freedom to model the relationship between volume and shape and the body composition output calculated from an MRI or CT scan. There is no attempt to align the individual's anatomical registration obtained from a medical image (from step 510) to an individual's 2D or 3D representation, as in the embodiments. This highlights the limitations of previous methods in the ability to reliably and repeatably segment individual 2D and 3D shapes, because subject-specific anatomical information cannot be used to guide these methods. Having anatomically (e.g., using joints) guided data, shapes, and body parts in the methods of the embodiments also allows accurate, fair, and statistically meaningful comparisons to be made between one person and another, or to track and compare changes in a person over time.

[0280] Quality Assurance and New Medicine and Human film BCT Database 40 Implementation Method of Image Processing Technology

[0281] As will be described in further detail, quality assurance and pre-processing steps have been developed in connection with the capture phase of medical images and medical data for the BCT database 40. Figure 6a In particular, the quality assurance and pre-processing techniques developed for the BCT database 40 are related to the capture phase of human images, which can be in the form of images, videos or video frames, 2.xD images including depth images and 3D body scans for the BCT database ( Figure 6b ). Quality assurance and pre-processing steps have been developed in connection with the capture phase of anthropometric data for the BCT database 40. Additionally, quality assurance and pre-processing steps have been developed in connection with the capture phase of dynamic data for the BCT database 40.

[0282] Prior to acceptance into storage in the HIPAA-compliant BCT database 40, the collected data undergoes quality assurance protocols unique to embodiments of the present invention, as well as image processing techniques for human and medical images for identifying anatomical body surface landmarks and regions of interest, also unique to embodiments of the present invention. In the present embodiment, these data quality control procedures and image processing techniques are important for the proper registration of medical image data with human image data, and for the development of reliable ML / AI models to predict body composition and risk of chronic health and musculoskeletal diseases with high specificity and sensitivity.

[0283] With reference to the attached drawings Figure 6aAccording to the quality assurance protocol, all medical physiology, anthropometric and body composition details received before being accepted into the BCT database 40 (at step 601) are checked using a data validation algorithm. Therein, each medical physiology, anthropometric and body composition variable is compared with a Gaussian distribution of a global database of previously collected data (at step 603) before being accepted into the BCT database 40 (at step 602). The medical image validation algorithm then verifies the quality of the 2D and 3D medical image data (at step 605) (at step 604).

[0284] By using image processing technology, anatomical surface markers are identified from medical images, such as Figure 7 . Initially, each medical image is visually assessed by a trained researcher. During this visual inspection, specific anatomical surface markers from each medical image are digitized in two dimensions (x, y Cartesian coordinates) or three dimensions (x, y, z Cartesian coordinates). This depends on whether the medical image is collected in 2D (e.g., DXA) or in 3D (e.g., MRI or CT). The specific anatomical landmarks identified are the joint centers of the upper and lower extremities (i.e., wrists, elbows, shoulders, hips, knees, and ankles) (702) and the midpoint of the femoral neck. In addition, a line or plane bisecting the top of the iliac crest (i.e., pelvis) and the lower border of the mandible (i.e., chin), the level of the T12 / L1 vertebrae, and the lower border of the phalanges of the foot are recorded (701).

[0285] Once a sufficiently large sample of medical images with visually defined 2D or 3D coordinates (702) and vectors / planes (701) have been collated, an ML / AI model is developed to semi-automatically identify the process required to identify the joint center locations, vectors, and planes that define an individual's anatomy from the medical imaging data. Figure 8 In an embodiment, the input (801) variables of the ML / AI model are participant characteristic data (804), experimentally measured anthropometric data (803), and scaled 2D or 3D medical images (802). The output (806) of the ML / AI model is scaled 2D or 3D medical image data with user-defined 2D or 3D coordinates of the wrist, elbow, shoulder, hip, knee and ankle joints and the midpoint of the femoral neck (808). The output includes a vector or plane that bisects the top level of the iliac crest, the lower edge of the mandible, the level of the T12 / L1 vertebrae, and the lower edge of the foot (807). These outputs are stored in the BCT database 40.

[0286] With reference to the attached drawings Fig. 9 , and then using these participant-specific anatomically relevant joint centers and planes to segment or "cut" the entire body shape (901) into specific regions of interest. In an embodiment, these regions of interest include:

[0287] i) Arms: Pass through the plane below the chin, the center of the shoulder joint, the plane above the pelvis and the plane below the feet.

[0288] ii) Leg: The proximal portion of the segmentation will be defined using a plane passing from the superior lateral edge of the pelvis through the midpoint of the femoral neck. The distal portion of the segment will be the inferior edge of the foot.

[0289] iii) Head: The volume above the plane below the chin.

[0290] iv) Torso: This region will be defined as the entire body, minus the arms, legs and head (902).

[0291] v) Android (male) area: the volume between the upper pelvic plane and the parallel plane, 20% of the vertical length (y-axis) between the upper pelvic plane and the lower mandibular plane (i.e., chin) (903).

[0292] vi) Gynoid (female) region: the volume (904) between a plane (plane A) located 1.5 times the vertical length (Y axis) of the android region and a parallel plane (plane A) located 2 times the vertical length of the android region.

[0293] vii) Visceral Area: The visceral area is contained within the android area. The android area contains both visceral fat and subcutaneous fat, which forms an uneven layer of fat around the abdominal cavity. The volume of subcutaneous fat is calculated and then the visceral fat is subtracted from the total android fat in the android area. (905)

[0294] In this embodiment, it is important to reliably segment these segmentations and regions across participants because these shapes in 2D and 3D are used along with other input variables within the ML / AI model to predict an individual's segmentation and total body composition (i.e., total, segmentation, and regional lean mass and fat mass).

[0295] This anatomically guided approach used in embodiments of the present invention advantageously provides that each person's medical image in the BCT database 40 is mathematically defined using 2D or 3D coordinates (702), vectors, and / or planes (701) that use participant-specific anatomical information (i.e., joint centers and anatomical body surface landmarks) rather than simplified geometric shapes. In order for the human body geometry to be an ecologically valid representation of the human body and its associated medical images acquired using DEXA, CT, or MRI scans, it must conform to anatomical definitions and be able to represent the accurate physical body surface landmarks (e.g., joints) seen in the medical scans. In embodiments, body composition properties and values ​​not only rely solely on the standard geometry of the person (e.g., 3D shape scans), but also require anatomical data, such as valid joint centers.

[0296] The BCT database 40 of the present embodiment is unique in that it has rich and unique types of visual, medical and other clinical and physical statistical information data when compared to many existing databases such as the National Health and Nutrition Examination Survey (NHANES), which is the largest database in the United States in the world. In the present embodiment, BCT data are collected from various regions of the world and independently verified by experts in the field.

[0297] Learn and align body shape and composition with humans film CV, ML and AI techniques and models that are connected

[0298] The steps outlined and described above include:

[0299] i) the collection of large and heterogeneous databases (i.e., the BCT database 40 ) where human images are collected in parallel with medical images and other data;

[0300] ii) a high level of quality assurance of the data collected; and

[0301] iii) the use of precise anatomical joints and designated regions of interest to define medical and human images;

[0302] In an embodiment, it was found to be important to provide i) sufficient; ii) sufficiently accurate data input to train the necessary CV / ML / AI algorithms to be able to repeatedly and accurately predict an individual's body composition and anthropometry; and iii) the use of precise anatomical articulations and quality assurance protocols to allow for a robust and accurate unified representation of the data. Unified representation is an important process when using CV and ML in embodiments of the present invention. Simply put, a unified representation is a way of representing data or images using a hierarchy that is unique and identical for all participants. As a very simple example, if the images required by the ML model must be of size (M×N) and must be centered at the mid-hip position, then this means that any participant image of any image size needs to be normalized or preprocessed so that its size becomes (M×N) and the mid-hip position is centered.

[0303] The link or linking of this embodiment is an example of a relationship between body shape and composition and a human image. In other embodiments, additional and / or alternative relationships are possible. These may include being registered, and / or aligned, and / or matched to a unified representation of each participant's respective subject.

[0304] In this embodiment, there are four processes (which may include one or more different CV / ML / AI models) that have been developed offline to use machine intelligence to learn the relationships between the various components of the BCT database 40 and are able to estimate body composition and body circumference based on human images and relevant participant characteristics and attributes, as well as medical physiology, to estimate the details of the participants participating in and stored in the BCT database 40.

[0305] With reference to the attached drawings Figure 3 The first process (at step 301) of the CV / ML / AI model developed offline is a model that can and is operable to estimate object or user specific anatomical information and joints from human images. Data within the BCT database 40 is used to train the model. Figures 14a to 14d Both the medical images shown containing joints and body components and the human images collected simultaneously allow training CV / ML / AI models to estimate joints and anatomical surface landmarks from human images alone.

[0306] In particular, Figures 14a to 14d The image depicted in the figure shows the original medical image ( Fig.14a ), where the joints are Fig.14b are highlighted for machine learning training. These joints allow the ML / AI algorithm to define regions of interest, such as ( Fig.14c )and( Fig.14d) for skeleton and body composition images respectively, regardless of the person’s pose (i.e., pose invariant).

[0307] Fig.15a and Fig.15b The illustrated diagram shows how to use a CV / ML / AI model trained using data from the BCT database 40 to estimate specific anatomically relevant body surface landmarks and joint centers based on specific frontal and side profile human images.

[0308] This estimation of joint centers and anatomical body surface landmarks is an advantageous feature of the process of embodiments of the present invention, resulting in improved accuracy of body composition and body measurement estimates. It should also be noted that the developed CV / ML / AI model can estimate more joint centers than those shown in the figure.

[0309] The second process (at step 302) has a model developed to build on the previous discussion and in Fig. 9 The ROIs shown in use these anatomical body surface markers to estimate body part segmentation. In an embodiment of the present invention, these ROIs and parts of the body are needed and the system 10 uses these ROIs to be able to estimate various body composition estimates, such as whole body and partial lean mass, total and central fat measurements, and anthropometry, such as body circumference, area, volume, and total body mass.

[0310] This segmentation method facilitates this process. Prior to the present invention, body segmentation used to estimate body composition from 3D scanners was based on cutting the body into Fig.16 The specific areas shown. Anatomical landmarks and joint information are not determined and used as in the embodiments.

[0311] Via this advantageous segmentation method, the system 10 is operable and capable of estimating anatomical body landmarks and joints, and is therefore capable of segmenting the body in a manner similar to that used when segmenting medical images when actual joint information is available. Fig.16 and Fig.17 The difference between the two approaches can be seen in the images below, which show the difference between Fig.16 ) and use ( Fig.17 ) 3D segmentation of body parts using anatomical body landmarks and joint centers.

[0312] The third process (at step 303) involves processing of the data in the BCT database 40 to extract features and label image(s), 2D and 3D body shapes for the data in the BCT database 40, which are then used to train the CV / ML / AI model. Fig.18 An example of how these features can be prepared for specific human images is shown.

[0313] The fourth process (at step 304) involves using the extracted features, along with the image(s), 2D and 3D body shapes labeled for data in the BCT database 40, in combination with other medical physiology, participant characteristics and attributes to train an ML / AI model to produce estimates of body composition and body measurements.

[0314] By using the system 10 of the present embodiment, it was found that the modeling method from unseen participant data and preliminary results using only 2D images of the participants were as accurate as the results obtained using 3D scanners and linear regression based on the recently published method of Ng et al. (Ng et al. (2016). Clinical anthropometrics and body composition from 3D whole-body surface scans). The sample size of the inventors' preliminary results is also larger than the sample size used in the prior art, so their method is more extensive because it maintains the same accuracy over a larger sample size and only uses standard images.

[0315] As the size and diversity of the BCT database 40 increases, the processing and image feature extraction techniques of the system 10 result in a robust, improved, fine-tuned ML model with improved accuracy because the correlations between the different data are statistically meaningful and valid.

[0316] The device 12 has a controller 18 to control and operate the device 12 for receiving a signal from a human film Applications such as estimating shape and health risks use

[0317] Once the application is installed on the device 12, or an SDK or service storing and running the software of the present invention is called, the controller 18 is operable under the control of the analysis application to present a series of navigable electronic pages, screens and forms to the user 16 of the device 12 via the touch screen 42, allowing the input or capture of information and / or data, including data and / or information sensed by sensors via a set of sensors 44, such as images and video captured by a camera, instructions and commands related to the operation of the device 12 and system 10.

[0318] In the described embodiment, the server software suite of server 36 includes: a web server application, a registration and request application, an image processing application, a communication application, an invoicing / billing application, and a payment processing application.

[0319] As will be described in further detail, via the various applications of the server software suite, the server 36 is operable to perform functions and actions including: registering and sharing user data; data processing, including extracting, converting and merging data with data received via the application; and recording all real-time data passing through the application interface.

[0320] The web server application is operable to deliver content related to the system 10 to existing or potential users of the system 10 via a dedicated website such as a web page or other electronic page or screen. The website is operatively connected to data communication with the system 10 via a communication network, and a web browser is accessible via an Internet-enabled mobile communication device, such as a notebook computer or smart phone (including the device 12 in the embodiment). In the described embodiment, the means of data communication is through the Internet, however, other methods, such as direct connection, may be used in other embodiments of the present invention.

[0321] Content may include general information related to health and wellness, advertising and promotional or public relations information delivered through an appropriate forum or media or a combination of forums or media, including, for example, on the trademark YouTube. TM , Facebook TM and / or Twitter TM The services provided below.

[0322] The accessible web page includes an online registration page 110, which will be completed when the user first uses the system 10, and a request page 112. The website application is operable to enable a potential user of the system to manually register or record himself as a user, thereby creating a personal account, and requesting that the output from the system 10 be displayed via the display 22 of the device 12. In an embodiment, the output includes an estimate of the user's three-dimensional (3D) body shape and its associated body composition and health care risks. The output may include an estimate of the user's: shape, physical and ecological characteristics and / or at least one three-dimensional (3D) shape and its associated anthropometrics, body composition, and health care risks. The output user's body shape can be referred to as a second representation of the body, and can include a personalized subject-specific image of the user's body or a near-accurate image. The provided image can be referred to as an avatar. In this way, a visualization of an analysis via at least one of text, image, grid, 3D, video, icon, virtual reality, and graphics can be depicted in the output.

[0323] This is facilitated via the user completing through registration and request pages 110 and 112 and submitting to server 36 communications in the form of electronic registration and request forms including user registration and request information, respectively (at steps 201 and 202).

[0324] User registration information includes details of information and / or data relating to the user and his / her physical presence, including:

[0325] 1) User identification and contact details: Details that facilitate identification and communication with the user. These details may include the user's full private name, username when using the system 10, private home address, physical and / or email addresses for forwarding communications, contact telephone numbers, authentication information (such as passwords), and any other unique and / or relevant applicable identifying information. This information is used by the system 10 to communicate with the user, including communications and billing related to avatars created using the system 10.

[0326] 2) User Body Details: Information and / or data related to the user's body. In the described embodiment, this includes anthropometric data of the body, including sex / gender, height, weight, clothing size (e.g., small, medium, large, X-large, or XXL, to name a few examples), age / date of birth, and ethnic group. In alternative embodiments of the present invention, additional and / or alternative details related to and / or associated with the user's body may be requested.

[0327] 3) Billing and Payment Details: Details that facilitate billing and receiving payment from the debtor(s) responsible for paying for the user's use of system 10. Billing details may include physical and / or email addresses for forwarding communications, including, for example, billing notifications for processing and payment. Payment details may include details of a financial account, such as a credit card account of the debtor, stored and used to purchase items associated with actions performed via system 10, such as the performance of analysis and provision of estimates in embodiments. For example, in embodiments of the present invention, additional and / or alternative payment processing platforms may be used, including but not limited to the Paypal service.

[0328] The request information includes a first representation. As previously described, in an embodiment, the first representation includes a set of visual representations of the subject 14. Preferably, the visual representations in the set of visual representations include different views of the subject 14, and they are captured (in steps 203-205) where the subject 14 is located in front of a contrasting, substantially clutter / noise-free background. In particular, in the described embodiment, as a non-limiting example, the set of visual representations includes two photos of the subject 14, namely a first photo of a front view of the subject 14 and a second photo of a side view of the subject 14. In order to facilitate the capture and upload of the two photos, the user 16 is able to access the image capture screen 114 via the request page 112. The image capture screen allows the capture and viewing of photos before uploading the photos, and may include one or more sub-screens for guiding the user through the process. In the described embodiment, the device 12 may be operable via the controller 18 under the control of the imaging application to use data including orientation data generated via an internal gyroscope (of an orientation sensor that calculates the orientation of the device 12) to ensure that the image is taken in a vertical plane for improving its accuracy.

[0329] The above process includes: an image capture step for collecting images of a person; video, 2.xD or 3D images. In the described embodiment, this will include at least front and side photos, but may also include other images of a person such as video or burst photography.

[0330] This may be an image capture step such as that described in the published specification of international patent application with publication number WO 2016 / 086266 published in the name of MyFiziq Ltd or in its national phase patents / applications.

[0331] In alternative embodiments of the present invention, user registration and request information may include alternative or additional details, information and / or data.

[0332] All data and information collected via the applications of the server software suite (including the web server application and the registry application) is distributed within system 10 for use as described herein.

[0333] RU database 38 has a plurality of RU records. Each RU record includes a set of RU information related to an account of an RU of system 10, including registration and request information as described above, and other information associated with the RU, such as output created therefor.

[0334] The server 36 has a sensing component operable to sense or detect the receipt of communications including user registration and request information (sent via a dedicated website or other component as described herein). Upon sensing the receipt of such information, the server 36, via its processor, under the control of relevant applications of the server software set (including a database management module or application), is operable to generate, populate and manage records in the RU database 38 (as well as records in the BCT database 40), and perform actions as described herein based on the received data and information.

[0335] Potential users can also register or record themselves as users by providing user registration information via email, fax or other communication, such as Facebook TM or Twitter TM The social networking service, for example, is used to automatically capture and enter the RU database 38 through the action of software of the server software group or by a data entry operator or other employee of the administrator.

[0336] It should be noted that after successful registration, the RU may subsequently access the system 10 via an online access or "login" page, which provides access to the system 10 once the user enters appropriate identification and security authorization, such as their username and associated password.

[0337] The image processing application is operable to receive and process submitted user body details and the first representation of body 16 .

[0338] The process includes an ML model responsible for image inspection and processing steps, wherein the image(s) of a human being present in one type of a first representation of a body is first inspected for the presence of a person and its body elements within an optionally displayed guiding outline or image mask. Once the image(s) or representation(s) pass inspection, another ML model trained to distinguish between a person and other objects in the image(s) segments only the image(s) of the person from the background scene ( Fig.18 In embodiments of the invention, this may include initial image inspection and processing steps such as those described in the publication specification of international patent application with publication number WO 2016 / 086266 published in the name of MyFiziq Ltd, or in national phase patents / applications thereof, or as described in the disclosure of the invention. Figure 2 The steps described in the illustrated explanation are enhanced versions thereof.

[0339] The system 10 is then operable to implement a processing step to estimate joints from human images (or video, 2.xd or 3d images if received). This step uses the ML / AI model developed offline and described above (see step 301), which is constructed to estimate object or user specific anatomical information and joints from human images (see also Fig.15a , Fig.15b ).

[0340] The system 10 is then operable to implement processing steps to estimate body part segmentation from human images and establish 2D and 3D regions of interest using the joint and anatomical surface markers in the previous step and the CV / ML / AI model developed offline and described above (see step 302).

[0341] The system 10 is then operable to implement processing steps to extract features and label images, 2D and 3D body shapes using the ML / AI models that have been developed and described above (see step 303).

[0342] The system 10 is then operable to implement a processing step in which the ML / AI model(s) use the collected human images, features, landmarks, and 2D and 3D body shapes calculated in the previous steps to estimate body composition and body measurements as described above (see step 304).

[0343] The system 10 is then operable to implement steps wherein the estimated body composition and body shape are output and displayed to the user via the display 22, along with the user's trends and future predicted body composition based on previous (if any) and current estimates thereof.

[0344] The communication application is operable to enable communication between the server 36 and the device communicating therewith. Such communication includes the communication described herein, and may be any suitable type of communication, including email, pop-up notification, and SMS message, and may be encrypted for increased security.

[0345] The user is able to navigate, including advancing to and returning from generated electronic screens and pages, via execution of corresponding navigation interface element buttons provided on the generated electronic screens and pages. In particular, the navigation bar 116 has interface element buttons via which the user can control the system 10 to perform actions. In the described embodiment, this includes: taking a new image (with new analysis performed by the system / generating a new output estimate); and logging out / exiting the system 10.

[0346] The invoicing / billing application is operable to generate an invoice for each registered user including an amount payable based on their usage of the system 10 .

[0347] The payment processing application is operable to receive payment for each invoice.

[0348] In an embodiment of the present invention, one or more of the described, additional, and / or alternative operations performed by system 10 occur automatically without human intervention.

[0349] Health risk calculator based on estimated body composition and anthropometry

[0350] The following is an overview of how system 10 uses processed human images, relevant human features and estimated three-dimensional (3D) body shape, anthropometric and body composition data, as well as user-input participant characteristics and medical physiology to allow users to classify, assess and monitor their health care risks.

[0351] In an offline phase, body fat classifications were developed using body percent body fat (PBF) cut-offs from the BCT database 40, and data and visceral fat classifications from other public databases were developed using body visceral fat (VF) cut-offs from the BCT database 40. Data and cut-offs were stratified based on a combination of participant characteristics and attributes, medical physiology, and epidemiological information (e.g., sex, age, race, health status, fitness status, etc.).

[0352] In the offline phase, epidemiological data from peer-reviewed prospective literature were used to develop predictive risk equations, including a systematic review of the literature to provide sufficient data, and epidemiological data from the BCT database40 to develop risk equations for prediabetes, diabetes, cardiovascular disease, musculoskeletal disease, mortality, and falls.

[0353] Fig.10 An overview of predictive risk equations is described. A predicted health risk score (P-HRS) (1001) for an individual, such as an estimate of diabetes or cardiovascular disease, is a function of their participant characteristics and attributes (PA), body shape (BS), medical physiology (MP), and body composition (BC) (1002). A specific dependent variable in each of these categories will have an associated hazard ratio (HR) (1003) at a predefined confidence interval (CI) (i.e., 90%, 95%, 99%, etc.) (1004). The HR is preferred because it considers not only the probability of health and health events occurring, but also the timing of each event. If the user does not like HR, hazard ratios and odds ratios can also be used. The dependent variable (DV) (1005) and the corresponding HR (CI) depend on the health care risk that the individual is interested in assessing: among them, their risk of becoming prediabetic, developing a chronic disease or musculoskeletal disease, premature death, or fall event rate (i.e., 1 year, 5 years, 10 years, etc.). The HR (CI) for each DV was derived from epidemiological data stored in the BCT database 40 and peer-reviewed literature.

[0354] In an offline phase, a model is developed that uses machine intelligence to learn various statistical relationships between various components of the BCT database 40 to allow the model to predict medical physiology and epidemiology from human images, relevant participant characteristics and attributes, and estimated body composition and body circumference.

[0355] During the online phase, the display 22 is used to display the following information to the user:

[0356] i) Estimated body fat and estimated visceral fat are compared to body fat categories to show general categories of body fat, such as lean, normal, overweight, and obese; and visceral fat, such as low risk, normal risk, increased risk, high risk.

[0357] ii) for any combination of the participant's characteristics and attributes and medical physiology (e.g., gender, age, race, health status, fitness), comparing the estimated body composition output and / or the estimated circumference output to the normative population distribution contained in the BCT database 40 to show the user where they fall within the population distribution.

[0358] iii) An estimate of the individual's relative health risk score (RHRS) as a function of the participant's attributes, body type, medical physiology, and body composition. The specific variables used within these categories will depend on whether the individual is assessing their relative risk for prediabetes, chronic disease, musculoskeletal disease, mortality, or falls risk. If the individual's score places them at the low end of the health risk continuum, the individual's overall RHRS will be low; if their score places them at the upper end of the health risk continuum, the individual's overall RHRS will be high (e.g., if the individual's score is high, the individual will have a low RHRS). Fig.10 shown).

[0359] iv) An estimate of the user's increased risk of prediabetes calculated offline based on the predictive risk equation and / or ML / AI model.

[0360] v) Estimates of increased risk of chronic health diseases calculated offline based on predictive risk equations and / or ML / AI models.

[0361] vi) Estimates of increased risk of musculoskeletal disorders, mortality, and fall events calculated offline based on predictive risk equations and / or ML / AI models.

[0362] vii) A step of detecting sarcopenia indicated by the user or triggered by estimated low lean muscle mass relative to height.

[0363] In the online stage, the system 10 is operable to allow the user to track changes in any estimated body composition or body measurement relative to the most recent measurement and over time.

[0364] In the online phase, the ability to identify "at-risk" users, using estimates calculated from various predictive risk equations and body fat / visceral fat classifications, allows for early and tailored interventions to be recommended to the user throughout their lifetime to help improve the user's short-term and long-term healthcare risk profile.

[0365] The above and other features and advantages of embodiments of the present invention will now be further described with reference to the system 10 in use.

[0366] Interested persons register as users of the system via the registration process described above, resulting in them being provided with a user account.

[0367] Thereafter, the (now registered) user accesses and uses the system 10 as previously described to perform an analysis resulting in the generation of one or more outputs providing an estimate of their individual 3D body shape and its associated body composition and healthcare risks.

[0368] Over time, users can generate a sequence of such estimates showing changes in them. Through such frequent self-monitoring, users are able to assess their progress toward their personal health and wellness goals and are therefore more likely to achieve them.

[0369] It should be appreciated that the described embodiments of the present invention provide several advantages as previously highlighted and described herein.

[0370] In the context of seeking to provide an ideal body composition measurement tool, it can be seen that the described embodiments are affordable (103) and accessible (101) because the system 10 can be incorporated into a portable and easily accessible device, such as a smartphone, while also being accurate (102). It is the combination of the collection of BCT databases 40, various quality assurance and medical imaging / human image processing techniques, and the development of favorable CV, ML, and AI techniques that enables embodiments of the present invention to be both affordable and accessible while being accurate.

[0371] Embodiments of the present invention provide a database with an unlimited degree of individual information as it has actual 3D shapes, images, videos and medical images collected simultaneously, as well as many other properties.

[0372] Unlike the prior art, the method of the described embodiments is almost completely automated and does take into account the anatomical structure of the object in addition to using high-order models including machine learning models to characterize or describe the object with respect to its shape and other features.

[0373] Another important advantage of embodiments of the present invention is the registration and alignment of different data types and images captured by different sensors. Current sensing and imaging techniques, such as the work disclosed in International Patent Application Publication No. WO 2016 / 072926A1 and the literature surveyed therein, attempt to register or align an MRI image with another MRI image or a general medical image. In their work, the two MRIs were acquired by the same MRI machine, the same person, in the same body posture, during a single scan. The same is true when processing CT scans. Unlike the present invention, their method neither captures nor utilizes actual color images / photographs of people. In fact, the methods developed for this method cannot align a person's MRI with their color camera images, even if these images are available, and each person has a different posture and orientation in each image, which would make it impossible for the method to solve this problem. In addition, MRI scanners and cameras use completely different sensors, requiring different models to physically represent the process of sensing or capturing images. The work disclosed in WO2016 / 072926A1 aims to provide an enhanced method for clinicians processing whole-body MRI scans (thus, the word whole volume used in the specification of this application does not mean the actual volume in liters or similar units). In addition, their method is to provide better MRI images in the laboratory in order to calculate body composition data, such as bone structure and density, fat mass, lean mass and water, etc., to name just a few examples. The method is also to provide a better visualization of accurately aligned MRI images so that they can accurately track the same area or certain image clusters and make better assessments. The present invention is different from systems such as MRI systems and works differently. In the described embodiment, the present invention runs in a portable device such as a smart phone, and the invented underlying methods and algorithms are integrated and run in the device in an almost fully automated manner. They are based on novel and inventive methods derived from the registration of (i) object characteristic data (e.g., height, weight, age, nationality, etc.), (ii) MRI data including body composition data, and (iii) MRI images taken by a smart device camera or other camera and the object (person) image. All of these different types of data, or features extracted therefrom, are combined, aligned, and co-registered together to form a unique model of the object and allow AI and machine learning techniques to understand and learn the differences between subjects and improve predictions when comparing objects to previously modeled objects. Another key difference between the present invention and the work described in WO 2016 / 072926A1 is that they deform the skeleton during the alignment process. The described embodiments of the present invention do not alter the skeleton during alignment / registration as this goes against the physics of deformation of rigid elements such as bones and would affect the ecological significance of the human body.

[0374] In patent number US 8,588,495 B2, Gupta et al. describe a system developed and claimed to use medical image datasets for automatic diagnosis. They use generative and discriminative learning to create data maps of CT and MRI medical images, however, both are based on standard statistical analysis of the method used by Gupta et al. The atlas is created mainly using image intensity. Similar to the work disclosed in WO 2016 / 072926 A1, the work of Gupta et al. only involves the registration of medical images with medical images. However, the embodiments described in the present invention deal with the registration, alignment and matching of images of a person taken by a camera (e.g., a smartphone) and their medical images to create a unified representation. Complexity arises when processing and co-registering images from two completely different sensors (such as MRI / PET and a camera). In addition, the segmentation process in the described embodiments of the present invention involves segmenting the image of the person from the noisy background and aligning the segmented image with the person's related (multiple) medical images. In the above-mentioned embodiments of the present invention, another type of segmentation is designed and implemented to segment the 3D body shape into different parts. These processes differ from the segmentation process outlined in the invention of Gupta et al., in which segments of medical images are cropped using existing software (such as CAD for classifying diagnoses). Importantly, in the embodiments described in the present invention, the risk analysis or diagnosis process is accomplished via a portable smart device using only standard camera images that can be taken as frequently as needed, whereas the work of Gupta et al. requires actual MRI scans for automated diagnosis, which are expensive and cannot be acquired regularly due to the risk of repeated radiation exposure.

[0375] Those skilled in the art will appreciate that changes and modifications to the present invention described herein will be apparent without departing from the spirit and scope of the present invention. Changes and modifications apparent to those skilled in the art are considered to fall within the broad scope and range of the present invention described herein.

Claims

1. A portable device for analyzing a subject's body and analyzing a portion of the subject's body, the device comprising: Controller; a memory device storing electronic program instructions for operating the controller; as well as Input components; the input component comprises at least one sensor operable to capture at least one representation of the subject's body or the portion thereof, the at least one sensor comprising at least one camera, and the at least one representation of the subject's body or the portion thereof comprising a visual representation captured by the camera; wherein the controller is operable alone and / or in cooperation with one or more remote devices and is directed by the instructions of the electronic program to: receiving, via the input component, an input comprising the at least one representation of the subject's body or the portion thereof; processing the input to perform an analysis of the subject's body or the portion thereof and generating an output based on the analysis, the processing comprising estimating body composition, anthropometric variables and / or medical physiology using at least one of a machine learning (ML) and / or artificial intelligence (AI) algorithm trained using an ecologically and physically valid unified representation of the human body and its anatomical structures, said unified representation of the human body and its anatomical structures is a model of said human body and is created from a dataset of contributed details processed, prepared and quality assured using human anatomical body landmarks estimated using at least one computer vision CV, machine learning ML and / or artificial intelligence AI technique, the contributed details comprising a visual representation of the contributed details of at least one human body to the data set, and other corresponding data and / or information of the at least one human body including at least one medical image, registering, aligning, segmenting and matching other corresponding data and / or information to at least one visual representation of a human body contributing details to said dataset using anatomical body landmarks of said human body estimated using at least one computer vision CV, machine learning ML and / or artificial intelligence AI technique to create said unified representation of said human body and its anatomical structures; and transmitting said output, wherein said output comprises said analysis of said subject's body or a portion thereof; and The anatomical body surface landmarks include a plurality of joint centers selected to define a specific region of interest (ROI) in the subject's body.

2. The apparatus according to claim 1, wherein the at least one representation of the subject's body or the part thereof comprises a visual representation and / or a non-visual representation, the visual representation comprising an image, the non-visual representation referring to data that cannot be visualized.

3. The apparatus of claim 1 , wherein the anatomical body landmarks include joint centers of the upper and lower extremities, the midpoint of the femoral neck, a line or plane bisecting the top of the iliac crest and the lower border of the mandible, the level of the T12 / L1 vertebrae, and the lower border of the phalanges of the foot.

4. An apparatus according to claim 1, wherein the subject body or the part thereof being analyzed belongs to a category and / or group, and the data set includes details of multiple different human bodies or parts thereof belonging to the same and / or similar category and / or group as the subject body or the part thereof being analyzed, the details including data and / or information of each human body or part thereof in the multiple different human bodies, data and / or information associated with each human body or part thereof in the multiple different human bodies, and / or data and / or information related to each human body or part thereof in the multiple different human bodies.

5. An apparatus according to claim 4, wherein for each human of a plurality of human participants contributing details of their respective bodies to the dataset, the data and / or information comprises one or more human videos, photographs, full and / or partial body shape or surface scans, medical imaging, body composition, anthropometry, participant characteristics, participant attributes, medical physiology, and epidemiological information, which have been collected at the time of collection and are linked, and / or registered, and / or aligned, and / or matched to respective visual representations to create an ecologically and physically valid unified representation of each participant's respective said human body and its anatomical structure.

6. The apparatus of claim 5, wherein the collecting of the details occurs simultaneously and in parallel with collecting the respective visual representations.

7. Apparatus according to claim 6, wherein additional data is collected at different time intervals so as to be useful in developing temporal modelling of participant shapes and data collected over time, and deriving statistically meaningful diagnostic trackers, and enabling physically meaningful risk analysis.

8. The apparatus of claim 5, wherein the collection of the details comprises capturing details of each participant contributing to the data set according to quality assurance protocols and pre-processing procedures and steps.

9. The apparatus of claim 8, wherein the at least one representation of the subject's body or a portion of the subject's body comprises: A visual representation comprising an image and / or a non-visual representation representing that the data cannot be visualized, and wherein the processing comprises at least one of the following actions: Analyzing the collected data and / or information and images using multidimensional statistical techniques and / or machine learning techniques and / or artificial intelligence (AI) techniques, and excluding images of visual representations identified as outliers; Classify and cluster data and images into one or more of groups, regions, types, characteristics, and categories based on one or more labeling and / or annotation mechanisms of data types and / or associated outputs linked to data types; estimating salient features and / or body surface landmarks from any type of image of the subject's body, said salient features and / or body surface landmarks comprising anatomically relevant body surface landmarks and / or key points and / or joint centers and / or bone links; identifying and segmenting the subject's body from background clutter in the image; estimating body part segmentation from an image of the subject's body constrained by anatomical features extracted from medical imaging, and establishing a region of interest (ROI); as well as Image or overall shape features and labeled image segments and / or image clusters are extracted and their correspondence with body data is estimated.

10. The apparatus of claim 1 , comprising a display for displaying a user interface, wherein the controller is operable under the direction of the electronic program instructions to communicate the output by displaying the output via the display, the display depicting a visualization of the analysis via at least one of text, images, grids, 3D, video, icons, virtual reality, and graphics.

11. The apparatus of claim 1 , wherein the subject body is the body of an individual person and the output comprises estimates of the individual person's: shape, physical, and ecological characteristics and / or at least one three-dimensional 3D shape and its associated anthropometry, body composition, and health care risks.

12. The device according to claim 1, wherein the at least one representation of the subject's body is in the form of any type of numbers and / or text and / or data and / or images.

13. A method for analyzing a subject's body and analyzing a portion of the subject's body, the method comprising: storing electronic program instructions for controlling a controller of a portable device; as well as The electronic program instructions, operable alone and / or in cooperation with one or more remote devices, control the controller to: receiving input via an input component of a portable device, the input component comprising at least one sensor operable to capture at least one representation of the subject's body or the portion thereof, the at least one sensor comprising at least one camera, and the at least one representation of the subject's body or the portion thereof comprising a visual representation captured by the camera; as well as processing the input to perform an analysis of the subject's body or the portion thereof and generating an output based on the analysis, the processing comprising estimating body composition, anthropometric variables and / or medical physiology using at least one of a machine learning (ML) and / or artificial intelligence (AI) algorithm trained using an ecologically and physically valid unified representation of the human body and its anatomical structures, said unified representation of the human body and its anatomical structures is a model of said human body and is created from a dataset of contributed details processed, prepared and quality assured using human anatomical body landmarks estimated using at least one computer vision CV, machine learning ML and / or artificial intelligence AI technique, The contributed details include a visual representation of at least one human body contributing details to the data set, and other corresponding data and / or information of the at least one human body contributing details to the data set, including at least one medical image, using anatomical surface landmarks of the human body estimated using at least one computer vision CV, machine learning ML and / or artificial intelligence AI technology, registering, aligning, segmenting and matching the other corresponding data and / or information to the visual representation of at least one human body contributing details to the data set to create the unified representation of the human body and its anatomical structure, and The anatomical body surface landmarks include a plurality of joint centers selected to define a specific region of interest (ROI) in the subject's body.

14. The method according to claim 13, wherein: Said at least one representation of said subject's body or said part thereof comprises a visual representation including an image and / or a non-visual representation referring to data which cannot be visualized.

15. The method of claim 13, wherein the anatomical body landmarks include the joint centers of the upper and lower extremities, the midpoint of the femoral neck, the line or plane bisecting the top of the iliac crest and the lower border of the mandible, the level of the T12 / L1 vertebrae, and the lower border of the phalanges of the foot.

16. A method according to claim 13, wherein the subject body or the part thereof being analyzed belongs to a category and / or group, and the data set includes details of multiple different human bodies or parts thereof belonging to the same and / or similar category and / or group as the subject body or the part thereof being analyzed, the details including data and / or information of each human body or a part thereof in the multiple different human bodies, data and / or information associated with each human body or a part thereof in the multiple different human bodies, and / or data and / or information related to each human body or a part thereof in the multiple different human bodies.

17. A method according to claim 16, wherein for each human of a plurality of human participants who contribute details of their respective bodies to the dataset, the data and / or information includes one or more human videos, photographs, full and / or partial body shape or surface scans, medical imaging, body composition, anthropometry, participant characteristics, participant attributes, medical physiology, and epidemiological information, which have been collected at the time of collection and are linked, and / or registered, and / or aligned, and / or matched to respective visual representations to create an ecologically and physically valid unified representation of each participant's respective said human body and its anatomical structure.

18. The method of claim 17, wherein collecting the details occurs simultaneously and in parallel with collecting the respective visual representations.

19. The method of claim 18, wherein additional data is collected at different time intervals to enable development of temporal modeling of participant shapes and data collected over time and to derive statistically meaningful diagnostic trackers and enable physically meaningful risk analysis.

20. The method of claim 17, wherein the collecting of the details comprises capturing details of each participant contributing to the data set according to quality assurance protocols and pre-processing procedures and steps.

21. The method of claim 20, wherein the at least one representation of the subject's body or a portion of the subject's body comprises: A visual representation comprising an image and / or a non-visual representation representing that the data cannot be visualized, and wherein the processing comprises at least one of the following actions: Analyzing the collected data and / or information and images using multidimensional statistical techniques and / or machine learning techniques and / or artificial intelligence (AI) techniques, and excluding images of visual representations identified as outliers; Classify and cluster data and images into one or more of groups, regions, types, characteristics, and categories based on one or more labeling and / or annotation mechanisms of data types and / or associated outputs linked to data types; estimating salient features and / or body surface landmarks from any type of image of the subject's body, said salient features and / or body surface landmarks comprising anatomically relevant body surface landmarks and / or key points and / or joint centers and / or bone links; identifying and segmenting the subject's body from background clutter in the image; estimating body part segmentation from an image of the subject's body constrained by anatomical features extracted from medical imaging, and establishing a region of interest (ROI); as well as Image or overall shape features and labeled image segments and / or image clusters are extracted and their correspondence with body data is estimated.

22. The method of claim 13, comprising controlling the controller via the electronic program instructions to communicate the output by displaying the output via a display, the display depicting a visualization of the analysis via at least one of text, images, grids, 3D, video, icons, virtual reality, and graphics.

23. The method of claim 13, wherein the subject body or the portion thereof is the body of an individual person and the output comprises estimates of the individual person's: shape, physical, and ecological characteristics and / or at least one three-dimensional 3D shape and its associated anthropometry, body composition, and health care risks.

24. A non-transitory computer-readable storage medium having instructions stored thereon, which, when executed by a computing component, cause the computing component to: receiving input via an input component of the portable device, the input component comprising at least one sensor operable to capture at least one representation of a subject's body or a portion of the subject's body, the at least one sensor comprising at least one camera, and the at least one representation of the subject's body or the portion thereof comprising a visual representation captured by the camera; and processing the input to perform an analysis of the subject's body or the portion thereof and generating an output based on the analysis, the processing comprising using at least one of a machine learning (ML) and / or artificial intelligence (AI) algorithm to estimate body composition, anthropometric variables and / or medical physiology, wherein, The machine learning ML and / or artificial intelligence AI algorithms are trained using an ecologically and physically valid unified representation of the human body and its anatomy, said unified representation of the human body and its anatomical structures is a model of said human body and is created from a dataset of contributed details processed, prepared and quality assured using human anatomical body landmarks estimated using at least one computer vision CV, machine learning ML and / or artificial intelligence AI technique, The contributed details include a visual representation of at least one human body contributing details to the data set, and other corresponding data and / or information of the at least one human body contributing details to the data set, including at least one medical image, using anatomical surface landmarks of the human body estimated using at least one computer vision CV, machine learning ML and / or artificial intelligence AI technology, registering, aligning, segmenting and matching the other corresponding data and / or information to the visual representation of at least one human body contributing details to the data set to create the unified representation of the human body and its anatomical structure, and The anatomical body surface landmarks include a plurality of joint centers selected to define a specific region of interest (ROI) in the subject's body.

25. The non-transitory computer readable storage medium of claim 24, wherein the anatomical body landmarks include joint centers of upper and lower extremities, the midpoint of the femoral neck, a line or plane bisecting the top of the iliac crest and the lower border of the mandible, the level of the T12 / L1 vertebrae, and the lower borders of the phalanges of the foot.

Images