Human body figure management method based on multi-modal data fusion

Abstract

The invention discloses a human body figure management method based on multi-modal data fusion, and the method operates in at least two working modes, comprises a general mode and a high-precision mode, and comprises the steps: collecting the multi-modal data of a user, the multi-modal data at least comprises three-dimensional body model data, body composition physiological data and user portrait data containing user health taboo and athletic ability; determining a current working mode on the basis of analysis of the three-dimensional body model data, extracting three-dimensional morphological characteristics according to the current working mode, performing fusion analysis on the three-dimensional morphological characteristics and the body composition physiological data, and generating a quantitative defect list; and based on the quantized defect list and the user portrait data, querying a knowledge graph, and generating a personalized body-building diet scheme. The system robustness is improved, the resource efficiency is optimized, the reliability of the evaluation result is guaranteed, the generation rationality of the personalized scheme is enhanced, and a series of problems caused by data quality fluctuation and splitting are effectively solved.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of human body management, and particularly relates to a human body management method based on multi-modal data fusion. BACKGROUND

[0002] With the general improvement of national health consciousness and the rapid development of artificial intelligence and sensor technology, intelligent body and health management has become an important direction in the field of consumer electronics and digital health. Traditional body management methods mainly rely on simple tools such as body scales and tape measures combined with subjective feelings, lacking systematic, data-based and personalized guidance.

[0003] Subsequently, with the popularity of smartphone cameras, body shape evaluation technology based on 2D image analysis emerged, which identifies body contour key points through computer vision algorithms and estimates girth data, realizing the preliminary upgrade from weight management to body shape evaluation. At the same time, the rise of smart wearable devices such as wristbands and body fat scales enables the convenient acquisition of physiological data such as heart rate, body fat rate, and muscle mass, adding new dimensions to body evaluation.

[0004] To further improve the level of personalization and automation, the industry has begun to try to introduce more complex models. For example, some solutions use rule engines or simple machine learning models to match and generate preliminary solutions from a pre-set action library and recipe library based on user input targets such as weight loss and muscle gain and limited body data. More advanced research attempts to use large language models (LLM) to understand user needs through dialogue interaction and generate fitness suggestions in text form.

[0005] However, existing technologies highly depend on single data sources or low-precision data. The 2D image-based recognition method is severely affected by shooting angles, light, and clothing obstruction, with girth measurement errors usually exceeding 5%, and completely unable to obtain three-dimensional shape information of the body's side and back, resulting in low-quality three-dimensional reconstruction, insufficient key point recognition, or low confidence. In the face of such quality fluctuations, either force output unreliable results based on low-quality data, or directly report errors and interrupt the process, resulting in unavailable services, lack of adaptability to imperfect data scenarios, and serious impact on user experience and system practicality and reliability.

[0006] Even if three-dimensional scanning technology is introduced, it is mostly used to generate visual models, and the deep features of three-dimensional data are not effectively extracted and quantified. At the same time, visual shape data and physiological data from devices such as body fat scales often lack effective fusion analysis mechanisms.

[0007] Moreover, the generation logic of the existing scheme is mostly based on static defect-action mapping rules or broad user tags. The essence is to select from a limited scheme template library. This mode seriously ignores the individual multi-dimensional constraint conditions of the user and makes it impossible to predict and avoid individualized risks at the time of generation.

[0008] Therefore, the existing scheme still has a series of technical problems to be solved in realizing accurate, safe and sustainable personalized body management. SUMMARY

[0009] The present application provides a human body fitness management method based on multi-modal data fusion to solve the technical problems of poor evaluation reliability caused by data quality fluctuations and insufficient individualized adaptation caused by multi-source data fragmentation and scheme templating in existing fitness management technologies.

[0010] The technical solution adopted by the present application is: A human body fitness management method based on multi-modal data fusion, the method runs in at least two working modes, including a general mode based on a first order of magnitude key point and a high-precision mode based on more than a first order of magnitude key point, the method comprises: Collecting multi-modal data of the user, the multi-modal data at least includes three-dimensional body model data, body composition physiological data, and user portrait data containing user health contraindications and exercise capacity; Based on the analysis of the three-dimensional body model data, the current working mode is determined, and the three-dimensional shape features are extracted accordingly, and the three-dimensional shape features are fused and analyzed with the body composition physiological data to generate a quantitative defect list, wherein each item of the quantitative defect list at least includes type, location and severity quantization value; Based on the quantitative defect list and the user portrait data, a knowledge graph is queried to generate an individualized fitness and diet scheme.

[0011] The human body fitness management method based on multi-modal data fusion adopted in the present application also has the following additional technical features: Determining the current working mode, specifically comprising: Processing the three-dimensional body model data through a pre-trained three-dimensional key point detection model to identify key points; Judging whether the number of key points identified by the three-dimensional key point detection model and having a confidence higher than a first confidence threshold reaches a first number threshold; If it is reached, the high-precision mode is entered; if it is not reached, the general mode is executed.

[0012] After determining the current working mode, it further comprises: If the high-precision mode is entered, three-dimensional morphological features are extracted according to the identified key points with a confidence higher than the first confidence threshold, for generating a quantitative defect list; If the general mode is executed, core points are determined according to the identified key points with a confidence higher than the first confidence threshold, for generating a quantitative defect list.

[0013] The three-dimensional morphological features are extracted, specifically including: Based on the coordinates of the key points, the girth and angle geometric features of each part of the body are calculated to determine the core points according to the change rate of the geometric features; The surface of the three-dimensional body model data in the key point area is subjected to texture analysis to extract a texture feature vector for distinguishing the state of subcutaneous tissue.

[0014] The quantitative defect list is generated, specifically including: The geometric features are compared with a standardized body model database to generate a primary evaluation result based on size deviation; The texture feature vector is associated with the body composition physiological data for correction of the primary evaluation result to determine the type and severity of size deviation; The quantitative defect list containing the type, location and severity quantitative value is outputted; Wherein, under the same evaluation result, the size deviation value of the high-precision mode quantifying the primary evaluation result is smaller than that of the general mode.

[0015] The personalized fitness diet plan is generated, specifically including: The type and location in the quantitative defect list are taken as the first query condition; The user's health contraindications and exercise capacity in the user portrait data are taken as the second query condition to filter and match the exercise action entities in the knowledge graph; According to the severity quantitative value, the exercise action obtained after filtering and matching is configured with an initial training parameter, and the user is matched with a corresponding diet suggestion to obtain the personalized fitness diet plan.

[0016] After generating the personalized fitness diet plan, a safety verification step is further included: The risk level of the exercise action in the personalized fitness diet plan, the conflict degree of the diet suggestion with the user's health contraindications are evaluated to obtain a safety adaptation index of the personalized fitness diet plan; If the safety adaptation index is lower than a preset threshold, an artificial review process is automatically triggered.

[0017] After generating the personalized fitness diet plan, dynamic optimization is further included based on objective compliance data and subjective feedback data by using a reinforcement learning model, specifically: A composite reward signal is constructed, which is obtained by weighting sum of an objective reward based on body circumference or body composition change and a subjective reward based on user subjective score and execution punch card rate; The current parameters of the personalized fitness diet plan and the user's quantitative disadvantage list are taken as state inputs, the adjustment of the plan parameters is taken as an action space, the composite reward signal is used to train and update the strategy of the reinforcement learning model, so as to generate an optimized plan.

[0018] The application further discloses an electronic device, The electronic device comprises a processor and a memory, the memory stores a computer program, and the processor implements the method of any one of the embodiments when executing the computer program.

[0019] The application further discloses a computer readable storage medium, The computer readable storage medium stores a computer program, and the computer program is executed by a processor to implement the method of any one of the embodiments.

[0020] Due to the adoption of the above technical solutions, the application has the following beneficial effects: 1. In the application, the current working mode is determined based on the analysis of the three-dimensional body model data, and an elastic processing capability is provided. Regardless of the quality of the data, the subsequent evaluation and plan generation process can be started and completed by switching to the corresponding mode (general mode or high-precision mode). The problem of service interruption or reliability drop caused by data input quality fluctuation in the prior art is solved, and the user can obtain usable body management service under most real conditions, which significantly improves the practicability and user satisfaction of the system.

[0021] In addition, the data is analyzed first, and then the three-dimensional shape features are extracted. In the high-precision mode, more complex models can be called to perform deep feature mining on high-quality three-dimensional data to pursue fine evaluation, and in the general mode, based on the core data, more efficient and fault-tolerant analysis can be performed. This dynamic adaptation mechanism optimally matches the computing resources, time cost and possible analysis benefits. Under the premise of guaranteeing the basic service, the waste of resources and the deviation of results caused by excessive interpretation of low-quality data are avoided, and the value loss caused by insufficient mining of high-quality data is also avoided.

[0022] Moreover, the output personalized fitness diet plan, whose generation relies on two key inputs: the quantified deficiency list, and the user profile data. The knowledge graph, when querying and matching based on this list, can combine the quantified deficiency list to adjust the strategy of plan generation. This makes the final generated plan, its degree of personalization matches the characterization of the current data, and a balance is achieved between personalization and safety, feasibility, reducing the risk of overly aggressive or unrealistic plans due to data limitations.

[0023] In summary, by introducing a data quality adaptive working mode, and combining it with multi-modal data fusion analysis, knowledge graph query, the system robustness is improved, resource efficiency is optimized, evaluation result reliability is guaranteed, and personalized plan generation rationality is enhanced, effectively solving a series of problems caused by data quality fluctuations and fragmentation. BRIEF DESCRIPTION OF DRAWINGS

[0024] The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this application, illustrate embodiments of the application and together with the description serve to explain the application. In the drawings: Figure 1 FIG. 1 is a flowchart of a body shape management method based on multi-modal data fusion according to an embodiment of the present application. DETAILED DESCRIPTION

[0025] In order to more clearly illustrate the overall concept of the present application, the following will be described in detail with reference to the accompanying drawings.

[0026] In the following description, a large number of specific details are set forth in order to facilitate a thorough understanding of the application, but the application can also be implemented in other ways different from those described herein, therefore, the scope of protection of the application is not limited by the specific embodiments disclosed below.

[0027] As shown in FIG. 1, a body shape management method based on multi-modal data fusion, the method runs in at least two working modes, including a general mode based on a first order of magnitude key points and a high precision mode based on more than a first order of magnitude key points, the method comprises: Figure 1 S100: Collecting multi-modal data of the user, the multi-modal data at least includes three-dimensional body model data, body composition physiological data, and user profile data containing user health contraindications and exercise capacity.

[0028] ​The core purpose of this step is to build a comprehensive, high-quality, strongly associated and structured raw data base for all subsequent analysis, decision and optimization processes. Specifically, it aims to obtain three types of data with inherent logical connections at one time to provide the basis for pattern judgment, supply the original data for fusion analysis, lay the foundation for personalization and security, and solve the defects of data silos and uncontrollable data quality in existing technologies.

[0029] It should be noted that, in order to avoid invalid data caused by random shooting by users, an active guidance strategy is implemented through the interactive interface, including: Environment and dress code, prompting users to wear tight-fitting clothes and stand in front of a solid color background to maximize the contrast between body contour and background and reduce subsequent segmentation errors; Dynamic posture guidance and evaluation, real-time posture guidance is provided through the mobile phone screen (such as "please slightly open your arms" "please turn slowly"), and device sensors (gyroscope, accelerometer) or front camera are used to preliminarily judge the stability of user posture and lens height.

[0030] Preferably, a lightweight real-time posture evaluation model can be integrated to analyze the visibility and symmetry of body key points in the preview image in real time during user rotation and give immediate correction feedback (such as "please hold the phone higher").

[0031] Multi-angle data fusion reconstruction, guiding users to complete image or video collection of front, side and back, and then using structure from motion (SfM) or simultaneous localization and mapping (SLAM) technology in terminal or edge server to fuse and reconstruct multi-view 2D image sequences into three-dimensional mesh models with texture information. This strategy directly overcomes the problem of missing 2D image information and makes three-dimensional analysis possible.

[0032] While collecting three-dimensional data, a lightweight data quality pre-check model is run. This model can quickly evaluate the quality of image sequences before reconstruction, detect whether there are serious blurring, excessive occlusion, extremely uneven lighting or incomplete angle coverage, etc. If problems are found, the process is immediately interrupted and specific retake prompts are given to avoid wasting resources by transmitting invalid data to the cloud.

[0033] In addition, to solve the problem of data fragmentation, a synchronization and alignment strategy is implemented during the collection process. Timestamp synchronization: when initiating a complete evaluation, a unique session ID is generated. Users are required to complete three-dimensional scanning and body fat scale measurement within a short period of time (such as within 2 minutes), and all data obtained within this time window (three-dimensional model, body fat rate, muscle mass, etc.) are bound to the same session ID and stamped with a uniform timestamp. This ensures that different modal data reflects the user's body state at the same time.

[0034] Device data calibration interface: For body composition physiological data, a device calibration parameter library can be maintained. When accessing different brands or models of smart body fat scales, call the corresponding calibration coefficient to modify the original measurement value, reduce the system error introduced by device differences, and improve the reliability and consistency of multi-source data fusion.

[0035] The collection of user profile data focuses on structuring and dynamization, and hierarchical information input is performed: through structured forms or conversational interactions, users are guided to provide information in a hierarchical manner. For example: basic layer (age, gender, height), health layer (whether there are doctor-diagnosed contraindications such as hypertension, lumbar disc herniation), ability layer (self-assessed exercise experience level: beginner / intermediate / advanced), preference layer (dietary restrictions such as lactose intolerance, vegetarianism).

[0036] The user profile is designed to be dynamically updated. For example, when the user reports joint discomfort caused by a certain exercise in subsequent feedback, the system can suggest that the user update or supplement a related discomfort history record in their profile as a more detailed constraint condition for future solution generation.

[0037] This method is data-driven, and the accuracy of all subsequent intelligent decisions (pattern discrimination, defect analysis, solution generation) depends heavily on the quality and dimensionality of the input data. If the data collection is loose, unstructured, and unsynchronized, even the most advanced algorithms cannot produce reliable results. Therefore, high-quality, structured, and multi-modal data collection is essential to ensure that the solution is valid and effective.

[0038] S200: Based on the analysis of the three-dimensional body model data, determine the current working mode, and extract the three-dimensional shape features accordingly. Perform fusion analysis with the body composition physiological data to generate a quantitative defect list, where each defect includes at least type, location, and severity quantification value.

[0039] The core purpose of this step is to convert the collected raw multi-modal data into a computer-understandable, operable, and accurate diagnosis of the user's physique problems. The specific goals are divided into three progressive levels: quality assessment and mode adaptation, precise feature extraction and fusion, and generation of an operable quantitative diagnosis.

[0040] The mode determination engine based on quantitative indicators is not subjective estimation, but based on automated analysis of three-dimensional model data. A pre-trained three-dimensional key point detection model is called to process three-dimensional grid or point cloud data for key point recognition and confidence evaluation. The model not only outputs key point coordinates, but also outputs a confidence score for each point, reflecting the reliability of the point prediction.

[0041] According to the identified key points, determine whether to enter a high-precision mode or a general mode. If the three-dimensional data structure is complete and detailed, it can support fine analysis, and the system enters the high-precision mode. Otherwise, the data quality is insufficient, and to avoid misinterpretation, it automatically downgrades to the general mode. This strategy ensures the objectivity and consistency of decision-making.

[0042] The hierarchical feature extraction strategy bound to the mode is not fixed, but dynamically adjusts its granularity according to the mode. In the high-precision mode, fine extraction is based on full-quantity high-confidence key points; in the general mode, frame extraction is used for feature extraction by further screening key points.

[0043] The fusion analysis algorithm of multi-modal data association is the key to improving diagnostic accuracy, and its core is to establish the association logic of morphology and physiology data. The final output is designed as highly structured data, which is convenient for subsequent processing.

[0044] The standardized output format is that each disadvantage is a structure, which is forced to include: type (such as fat accumulation, muscle deficiency, abnormal posture), location (such as deep waist and abdomen, back of upper arm, upper back), and severity quantification value (such as "+2.3cm", "-15% muscle mass", and "7 degrees of forward tilt").

[0045] This step realizes the transformation from raw data to intelligent cognition. Without this step, it will not be able to cope with real-world data quality fluctuations, cannot realize joint interpretation of morphology and physiology, and cannot output accurate and measurable action guidelines.

[0046] Through the mode adaptive mechanism, the system has the ability to downgrade, and when faced with imperfect data, it no longer collapses, but provides the best possible service commensurate with the data quality, greatly expanding the applicable scenarios and user coverage.

[0047] According to the data quality, dynamically adjust the analysis depth, avoid wasting resources caused by performing complex calculations on low-quality data (high-precision mode), and avoid the value loss caused by insufficient analysis of high-quality data (general mode), and improve the overall operation efficiency.

[0048] S300: Based on the quantified disadvantage list and the user portrait data, query the knowledge graph to generate an individualized fitness and diet plan.

[0049] The core purpose of this step is to make intelligent decisions, combine accurate diagnosis (quantified disadvantage list) with individual constraints of users (user portrait data), and generate a safe, feasible, effective, and highly personalized action plan through structured knowledge reasoning.

[0050] The knowledge graph is used for scheme generation, and its structure is far beyond a simple database. The graph traversal is performed from the disadvantages as an entry to obtain candidate scheme elements, i.e., candidate movement actions, dietary ingredient targets, and taboo warnings related to the current disadvantages.

[0051] Real-time filtering and adaptation based on user portraits are used to realize personalization. For example, mandatory taboo filtering, ability and difficulty adaptation, and preference integration.

[0052] If this step is not performed, the precise quantitative disadvantage list cannot be automatically converted into an executable solution. This step reflects the transition from cognitive intelligence to decision intelligence and realizes user value. Through deep query in the knowledge graph based on the dual conditions of disadvantages and portraits, real personalization is realized, the shackles of templates are broken, and the executability of the scheme is improved.

[0053] As a preferred embodiment of the present application, the current working mode is determined, specifically including: The three-dimensional key point detection model is used to process the three-dimensional body model data to identify key points; It is judged whether the number of key points identified by the three-dimensional key point detection model and having a confidence higher than a first confidence threshold reaches a first number threshold; If yes, the high-precision mode is entered; if no, the general mode is executed.

[0054] The core purpose of the present embodiment is to establish an objective, quantitative, and automated decision-making mechanism to determine the depth of analysis that can be supported by the input three-dimensional body model data, thereby realizing adaptive switching of the system processing strategy. The direct goal is to solve a realistic and key technical contradiction: how to make full use of high-quality data to realize fine analysis while ensuring that the system service does not interrupt and the output is still reliable under the condition of low-quality data.

[0055] The pre-trained three-dimensional key point detection model used here is not primarily used for direct morphological measurement, but serves as a probe for data quality evaluation. The model is trained on a large amount of high-quality three-dimensional human data, and its ability to identify key points is strongly related to the integrity, clarity, and standardization of the input data.

[0056] The model outputs a confidence value for each identified key point, which comprehensively reflects the reliability of the point position prediction and indirectly maps the quality of the local three-dimensional data around the point (such as point cloud density, surface noise, and texture clarity). Therefore, the number of high-confidence key points becomes a robust and computable proxy indicator for the usability of the overall three-dimensional data.

[0057] A quality and completeness threshold is set, the first confidence threshold is used to filter out unreliable and uncertain points in the model recognition results, and only confident recognition results are retained. This avoids counting model low-confidence points into statistics, ensuring the reliability of the statistical basis.

[0058] The first quantity threshold defines the minimum amount of valid information required to enter the high-precision mode. This threshold is not arbitrarily set, and sufficient key points covering the main anatomical positions of the front, side, and back of the body are required to construct a dense measurement network for fine-grained assessment of body shape (such as distinguishing local fat and analyzing muscle lines).

[0059] The deterministic binary logic is executed, avoiding ambiguous judgments, easy to implement and verify, and ensuring consistency in user experience (the same quality of data will trigger the same mode).

[0060] The decision to enter the high-precision mode or execute the general mode actually sets a processing switch and parameter context for subsequent steps such as extracting three-dimensional morphological features.

[0061] In a data-driven AI system, the quality of input data determines the upper limit of output results. Without this step of pre-filtering and shunting data, the entire system will be forced to make compromises between using complex models for all data (high cost and high risk) and using simple models for all data (waste of high-quality data). Therefore, establishing an adaptive shunting mechanism based on the quality of the data itself is a logical necessity and technical prerequisite for building a robust, practical, and efficient body management system.

[0062] The system is degraded, greatly improving robustness and coverage, ensuring meaningful output under any data condition, reducing false results or service interruptions, ensuring the reliability of subsequent analysis results, and preventing low-quality data from polluting subsequent high-precision analysis pipelines. Moreover, it optimizes computing resource allocation and system response efficiency, accurately allocating computing resources (such as large neural networks and complex point cloud processing algorithms) to high-quality data worthy of processing. For general mode data, lightweight and fast algorithms are used, reducing average processing energy consumption and improving throughput and response speed.

[0063] As a preferred embodiment under the present embodiment, after determining the current working mode, it further includes: If the high-precision mode is entered, three-dimensional morphological features are extracted based on the identified key points with a confidence higher than the first confidence threshold, which are used for generating a quantitative defect list. If the general mode is executed, the key points identified and having a confidence higher than a first confidence threshold are used to extract three-dimensional morphological features, to determine core points for the generation of a quantitative list of defects.

[0064] The core purpose of the present embodiment is to prepare, on the basis of the completed work mode determination, input data that strictly matches the mode accuracy requirements and is reliable for the subsequent three-dimensional morphological feature extraction step. The specific goals include locking high-quality analysis materials for high-precision modes and constructing a robust analysis skeleton for general modes.

[0065] In the high-precision mode, the high-quality subset is directly inherited, and the set of key points filtered in the work mode determination stage and having a confidence higher than the first confidence threshold is directly used as the input for the subsequent three-dimensional morphological feature extraction. This operation itself is a quality filtering that discards all low-confidence and possibly incorrect predicted points.

[0066] Since the prerequisite for entering the high-precision mode is that the number of key points reaches the first number threshold, this means that these high-confidence key points are relatively complete and dense in the spatial distribution on the body surface, sufficient to support independent modeling of each part. Therefore, all reliable points are relied on and utilized to pursue richness and locality of feature extraction.

[0067] In addition, the core skeleton extraction in the general mode is not abandoned or randomly selected when the data is insufficient to support full analysis, but rather a set of intelligent degradation processing is performed.

[0068] The core points are determined, specifically a minimum subset of key points that can uniquely and stably define the main segments and basic proportion relationships of the human body. For example, it includes about 20-30 points with clear anatomical significance such as the top of the head, the root of the neck, the left and right shoulder peaks, the left and right anterior superior iliac spines, the left and right knees, and the left and right ankles. They form a simplified stick figure model of the human body.

[0069] Based on the mapping matching of anatomical prior, a list of core anatomical point names is maintained. The identified high-confidence key points are matched with the anatomical landmarks in the list (which can be through coordinate space relationship or semantic label). The points that match successfully are determined as core points.

[0070] In addition, based on the reasoning completion of geometric and topological relationships, when some core points cannot be directly identified due to data loss (such as one side of the anterior superior iliac spine being blocked), the system can estimate the most likely position of the missing point based on the identified symmetrical points, adjacent points, and the standard proportion model of the human body, and supplement it as a core point (but marked as a calculated point).

[0071] Perform consistency check, and perform basic biomechanics rationality check (such as leg length should not exceed twice the torso, left and right shoulder height difference should be within a reasonable range) on the determined / calculated core point set, and exclude obvious abnormal combinations.

[0072] Finally, a core point set with fewer points, clear meaning, relatively reliable position, and clear structural relationship is obtained. This set is the data basis for subsequent calculation of overall characteristics such as height, sitting height, torso and limb length ratio, and main girth (based on the measurement section defined by the core point).

[0073] The switching of the working mode must be accompanied by the adaptive conversion of the underlying data basis. Without this step, the high-precision mode may be forced to process data containing noise points, affecting its upper limit of precision; the general mode may lead to unclear or even contradictory features extracted due to the direct use of sparse and chaotic point sets. Therefore, this step ensures the logical consistency and quality matching between the mode and the feature.

[0074] Specifically, three-dimensional morphological features are extracted, specifically including: Based on the coordinates of the key points, the girths and angle geometric features of each part of the body are calculated to determine the core points according to the change rate of the geometric features; Perform texture analysis on the surface of the three-dimensional body model data in the key point region to extract texture feature vectors for distinguishing subcutaneous tissue states.

[0075] The core purpose of this embodiment is to extract two types of mathematical features that can quantitatively describe the user's physique from the screened or determined key points and corresponding three-dimensional model regions: macro-geometric structure features and micro-surface organization features. The purpose is to build a two-dimensional feature system that can simultaneously depict the body shape size and surface texture, providing rich and complementary information input for subsequent multi-modal fusion analysis.

[0076] The geometric feature calculation based on the key point network is not a direct measurement of the circumference of the three-dimensional model, but a measurement of the circumference based on anatomical knowledge and defined measurement sections between key points. For example, the waist circumference is formed by connecting the left and right waist points and the abdominal front and back points to form an elliptical or closed spline curve, and then calculating its circumference. The algorithm will automatically select the complexity of fitting and the point set according to the high-precision mode (point-intensive) or the general mode (core point sparse).

[0077] The angle calculation strategy forms a vector by connecting three or more key points, and calculates the included angle. For example, the spinal curvature can be evaluated by connecting the key points of the cervical vertebra, thoracic vertebra, lumbar vertebra, and sacrococcyx, and calculating the included angle sequence between the consecutive line segments.

[0078] Core point determination based on rate of change of geometric features (for general mode): this is a dynamic optimization process. In general mode, the system first tries to compute multiple possible geometric features (e.g. limb lengths defined differently) based on all high-confidence points. Then, the rate of change of these geometric feature values when a certain key point is removed or slightly perturbed is analyzed. Points with low rate of change, meaning that their position is not very sensitive to the overall geometry, are usually located in the middle of a limb or in a flat area, and are robust as measurement reference, and can be determined as core points; points with high rate of change, usually located at joints or in areas with sharp changes in shape, are sensitive to measurement results, and can be treated with caution or discarded when data is insufficient. This is essentially an automatic selection strategy based on data-driven sensitivity analysis.

[0079] Region-based surface analysis for texture feature extraction, region definition and preprocessing, a local surface region is defined on the 3D mesh model, centered at a key point (or a region defined by its connecting lines). The region is subjected to illumination normalization to eliminate texture artifacts caused by uneven lighting during scanning.

[0080] Multi-scale texture descriptor extraction: statistical features, calculate the statistics (mean, variance, skewness) of the color (gray level) of the pixels / vertices in the region, as well as the contrast, uniformity, correlation, etc. extracted by local binary pattern (LBP) or gray level co-occurrence matrix (GLCM), to describe roughness and regularity.

[0081] Spectral features, perform two-dimensional Fourier transform or wavelet transform on the region, analyze its frequency distribution, and high-frequency components often correspond to fine lines or small wrinkles on the skin.

[0082] Preferably, high-level features based on deep learning, use an intermediate layer (such as the fc7 layer of the VGG network) of a convolutional neural network (CNN) pre-trained on a large number of skin and subcutaneous tissue images, input the cropped local region image, and output a high-dimensional feature vector. The vector encodes the abstract patterns learned by the network that are related to fat shape, muscle texture, etc.

[0083] Perform feature vectorization, combine the scalar features extracted by the above-mentioned methods into a fixed-length texture feature vector, as the texture fingerprint of the body region.

[0084] To solve the problem of single evaluation dimension in the prior art, multiple aspects of information must be obtained in the feature extraction stage. Single circumference data cannot explain the causes, and single texture data cannot quantify the size. Therefore, simultaneously extracting geometric and texture features is an indispensable data prerequisite for realizing the fusion analysis of morphology + physiology in two dimensions.

[0085] The strategy of determining core points according to the rate of change of geometric features is a data-driven adaptive method that can intelligently filter out the most stable and reliable points from the current specific and imperfect data set at each processing time to construct the overall geometric features, which is a key technical guarantee to ensure the consistency and robustness of the output results of the general mode.

[0086] Preferably, a quantitative defect list is generated, specifically including: The geometric features are compared with a standardized body model database to generate a preliminary assessment result based on size deviation; The texture feature vector is associated with the body composition physiological data for correlation analysis to correct the preliminary assessment result and determine the type and severity of size deviation; The quantitative defect list containing the type, location and severity quantitative value is outputted; Under the same evaluation result, the size deviation value of the high-precision mode quantitatively evaluating the preliminary assessment result is smaller than that of the general mode.

[0087] The core purpose of the embodiment is to integrate and judge the morphological and physiological features extracted upstream and separated from each other in an orderly manner, and finally output a unified, clear and action-guiding body problem diagnosis report. It aims to solve the core defects of one-sided evaluation, cause misjudgment and output ambiguity in the prior art.

[0088] Objective primary evaluation based on group standards, wherein the standardized body model database is not a single standard, but a multi-level data set divided according to dimensions such as gender and age. The system selects the corresponding standard subset for comparison according to the user profile to ensure the fairness and reasonableness of the evaluation.

[0089] Each geometric feature of the user (such as waist circumference, waist-to-hip ratio, and spinal curvature angle) is compared with the corresponding mean value (or ideal range) in the database. Z-score (standard score) or percentage deviation is calculated. Set one or more abnormal thresholds, and features exceeding the threshold are marked as primary abnormalities and generate a preliminary size deviation conclusion (such as waist circumference exceeding +5.2%).

[0090] Cause correction and judgment of cross-modal data association, the texture feature vector is input into a special classification or regression model. The model is trained to map the texture mode to the estimation of subcutaneous tissue state, for example, outputting "fat accumulation possibility score", "skin tightness score" or "muscle line visibility level".

[0091] Logical correlation analysis with physiological data is performed to verify consistency: for example, if the initial assessment result is "excessive thigh circumference", texture analysis shows that the area has "obvious fat folds", and body composition data shows "high overall body fat percentage". If all three points are consistent, the judgment is strengthened to "fat-type excessive thigh circumference".

[0092] The system performs contradiction analysis and correction: For example, the initial assessment may show "excessive waist circumference," but texture analysis shows smooth abdominal skin with firm texture, while body composition data shows "normal body fat percentage and low core muscle mass." In this case, the system will correct the initial assessment's conclusion of "excessive fat" and instead determine that the abnormal circumference is caused by "lax core muscles and insufficient visceral support," and may quantify the "severity of laxity" based on muscle mass data.

[0093] The overall determination of the type and severity, and the final type of defect (fat type / muscle laxity type / body shape type, etc.), is determined by the results of the above correlation analysis. The severity quantification value is a comprehensive function, whose inputs typically include: the magnitude of the size deviation (from the initial assessment), the confidence level of the cause determination (from the correlation analysis), and the degree of deviation of relevant physiological data (such as the percentage of body fat percentage exceeding the standard).

[0094] It should be noted that the difference in the precision of the dimensional deviation values ​​(the deviation value in high-precision mode is smaller than that in general mode) is due to the fact that in high-precision mode, geometric features are calculated based on more and more accurate key points, resulting in smaller absolute errors in the measurement of circumference and angles. Therefore, the uncertainty range of the calculated deviation value relative to the standard is also narrower, meaning that the quantification of the dimensional deviation value is more accurate.

[0095] Furthermore, although the generated list of defects does not directly indicate the pattern, the "granularity" of its content naturally reflects the pattern differences. In high-precision mode, the location description of the list is more specific (e.g., "long head region of the triceps brachii on the back of the upper arm"), and the type classification is more granular (e.g., "superficial fat accumulation"); in general mode, the location of the list is more generalized (e.g., "upper limb"), and the type is more generalized (e.g., "high fat content").

[0096] By incorporating texture and physiological data for correlation, causal relationships can be inferred, providing the basis for personalized solutions and significantly improving the accuracy and professionalism of body assessment. Clearly defining the differences in quantitative precision across different modes reflects the honesty and rigor of this adaptive system. It clarifies the system's capabilities to users and reviewers: better input (triggering high-precision mode) will yield more accurate output, making the overall technical effectiveness of the solution predictable and verifiable.

[0097] As a preferred embodiment of the present invention, generating a personalized fitness diet plan specifically includes: The type and position in the list of quantified defects are used as the first query conditions; Filter and match the exercise action entities in the knowledge graph with the user health contraindications and exercise ability in the user portrait data as the second query condition; According to the severity quantization value, configure the initial training parameters for the exercise action obtained after filtering and matching, and match the corresponding dietary recommendations for the user to obtain the personalized fitness dietary plan.

[0098] The core purpose of the embodiment is to combine standardized diagnostic results (quantified defect list) with individual limitations of the user (user portrait data), and automatically synthesize a safe, feasible, effective, and highly customized action plan through a structured knowledge system (knowledge graph). The main purpose is to completely solve the drawbacks of the prior art, such as template-based solutions, static solutions, and ignoring individual differences.

[0099] Precise retrieval targeting defects, each item in the quantified defect list is analyzed to extract its type (such as "fat accumulation" and "muscle relaxation") and location (such as "deep waist and abdomen" and "upper arm back") as a composite query key. For example, generate the query semantics: "Find all exercise action entities that target the location: deep waist and abdomen and improve the type: fat accumulation".

[0100] Multi-hop association in the knowledge graph, the knowledge graph pre-constructs semantic relationship edges such as defect-exercise action and defect-dietary ingredient. Perform graph traversal, starting from the deep waist and abdomen fat accumulation node, and find all associated candidate actions (such as "Russian twist" and "mountain run") and dietary ingredient suggestions (such as "increase dietary fiber" and "control saturated fat intake") along the relationship edges.

[0101] Safety and feasibility filtering with the user as the criterion, achieving personalization and safety. Contraindication hard filtering (safety first): use the health contraindications in the user portrait data (such as "knee injury" and "high blood pressure") as an insurmountable negative rule set. After obtaining the candidate actions, immediately check whether each action in the knowledge graph is marked as "not recommended" or "prohibited" associated with these contraindications. For example, if the candidate action "deep squat" has a "contraindication" relationship with the "knee injury" node, it is immediately and permanently excluded from the plan.

[0102] Exercise ability soft adaptation (feasibility matching): use the user's exercise ability (such as "beginner") as a difficulty adapter. Each action entity in the knowledge graph has a difficulty level attribute. Preferably, select actions with difficulty matching the user's ability. For efficient but high-difficulty actions, call the pre-stored downgrade version relationship in the graph (such as "standard push-up" associated with "kneeling push-up"), and automatically replace it.

[0103] Degree-based parameterized assembly, converting qualitative matching results into quantitative executable plans. Movement parameter configuration: according to the severity quantification value of the corresponding disadvantage, configure "group number", "number of times", "intermittent time" and "frequency" for each selected movement through predefined mapping functions or rules. For example, when the severity of "excessive waist and abdominal fat" is "+3cm", configure "3 sets, 60 seconds each" for the matched "plank support"; if the severity is "+1.5cm", it may be configured as "3 sets, 40 seconds each".

[0104] Dietary recommendation quantification: According to the severity and the user's basal metabolic data, the dietary component recommendations are converted into specific quantitative targets. For example, for the "fat loss" type, according to the severity and body weight, the specific daily calorie gap and the gram requirements of protein, carbohydrates, and fat are calculated, and specific food combinations are recommended in combination with the user's dietary preferences (such as vegetarian).

[0105] Conflict resolution and overall assembly: When the user has multiple disadvantages, global plan optimization is required, such as merging training days, arranging the order of training different parts to avoid excessive fatigue, and ensuring that the total dietary goals are consistent in macronutrients.

[0106] The development of a fitness and diet plan involves knowledge from multiple fields such as anatomy, exercise physiology, nutrition, and individual medical contraindications. By constructing a knowledge graph and defining the query and filtering logic of this embodiment, the complex decision-making process of human experts is encoded into an automatically executable computer program, enabling large-scale and personalized services.

[0107] Specifically, after generating the personalized fitness and diet plan, a safety verification step is included: Assess the risk level of the movement actions in the personalized fitness and diet plan, the conflict degree of the dietary recommendations and the user's health contraindications, and obtain the safety fit index of the personalized fitness and diet plan; If the safety fit index is lower than the preset threshold, an automatic human review process is triggered.

[0108] The core purpose of this embodiment is to add a safety line and a man-machine collaborative decision-making node at the end of the entire automated plan generation process. It aims to solve a key contradiction: how to ensure safety in highly automated and personalized intelligent decision-making.

[0109] Multi-dimensional risk quantification evaluation model for movement action risk level assessment: Each movement action entity in the knowledge graph has pre-stored inherent risk level metadata (such as "low", "medium", "high") in addition to the associated contraindications. This level is determined based on the joint load, technical complexity, and injury probability statistics of the movement. During safety verification, the risk levels of all actions in the plan are extracted.

[0110] Diet vs. contraindication conflict degree quantification: The check here is more detailed than when it was generated. For example, the user has "diabetes", and the plan recommends "200 grams of carbohydrates per day". The system not only judges "there is carbohydrates", but also calculates the percentage of the recommended carbohydrate amount that exceeds the safe recommendation range according to the user's disease control target (such as the blood sugar control level in the user profile), as the conflict degree value.

[0111] Synthesis of the safety adaptation index: The index is the result of a composite function. For example: Safety adaptation index = f(maximum risk level of action, severity of dietary conflict, overall intensity of plan, user risk sensitivity factor), Where the user risk sensitivity factor can be dynamically adjusted according to the user's age and the complexity of the medical history. The function design can be a weighted sum or a rule engine (e.g. deduct 50 points directly if there is a "high risk" action; deduct 30 points if the dietary conflict exceeds 20%, etc.). Finally, a normalized score (such as 0-100 points) is output.

[0112] Threshold-based automatic grading decision and process switching, preset threshold setting: Set one or more safety thresholds (such as high safety line = 85 points, warning line = 60 points). Thresholds can be set uniformly at the system level, or fine-tuned according to different risk categories of user groups.

[0113] If the index ≥ high safety line: The plan is considered highly safe and is automatically approved and immediately issued to the user terminal.

[0114] If the warning line ≤ index < high safety line: The plan has moderate risk. The system can push a prominent safety prompt to the user at the same time (such as "the plan contains moderate intensity actions, please ensure action standards, and stop immediately if you feel unwell"), realizing informed execution.

[0115] If the index < warning line: The system automatically triggers the manual review process. The plan is suspended and stored in the pending database, and immediately notifies the health manager or professional coach in the background. At the same time, the user terminal displays "the plan is being reviewed by experts, please wait".

[0116] Manual review is not simply pass / fail, but an efficient collaborative link: The reviewer's interface will clearly show the specific reasons why the plan is marked as low safety index (such as "action A has high risk level and the user has related old injury history"), and can directly modify the plan or fill in the reasons for rejection in the system. The review conclusion and modification will be fed back to the system for subsequent model optimization learning.

[0117] Although the knowledge graph is comprehensive, it cannot cover all individual special cases, rare disease combinations, or cutting-edge medical discoveries. Automatically generated plans may have hidden risks due to limitations in the knowledge base or omissions of complex cross-forbidden combinations. This step serves as an independent, risk-quantified review layer, addressing such corner cases and ensuring the system's cautious behavior in unknown areas.

[0118] Specifically, after generating the personalized fitness diet plan, it further includes dynamic optimization based on objective compliance data and subjective feedback data using a reinforcement learning model, specifically: A composite reward signal is constructed, which is obtained by weighting the sum of objective rewards based on body circumference or body composition changes, and subjective rewards based on user subjective scores and execution punch-in rates; The current parameters of the personalized fitness diet plan and the user's quantitative disadvantage list are used as state inputs, the adjustment of the plan parameters is used as the action space, and the composite reward signal is used to train and update the strategy of the reinforcement learning model to generate an optimized plan.

[0119] The core purpose of this step is the self-evolution of the plan, and its core task is to break the rigid mode of traditional fitness plans being generated once and executed statically, achieving continuous learning and adaptive optimization of the system. The main purpose is to solve the core defects of existing technologies that the plan cannot respond to real-time feedback after being generated and lacks dynamic adjustment capability, upgrading individualization to dynamic adaptation.

[0120] A dual reward signal that integrates multiple dimensions is defined, and the objective reward is quantitatively designed based on periodic body re-measurement data (such as specific circumference change ΔC, body fat rate change ΔF). The reward function encourages positive physiological changes, for example: R_obj=α*(ΔC / target value)+β*(ΔF / target value). The key to its design lies in the solution of the sparsity and delay of rewards, which requires the combination of short-term behavior (such as completion rate) to set intermediate goals or the use of eligibility traces and other techniques.

[0121] Introduction and modeling of subjective rewards: This is the humanization core of this step. The system collects user ratings for "action difficulty" (1-5 points), "fatigue", "diet acceptance", and "punch-in rate" behavior data. For example: R_sub=γ*(difficulty rating inverse conversion)+δ*(punch-in rate). Subjective rewards convert user experiences that are difficult to quantify into model-optimizable goals, preventing the model from recommending plans that users cannot tolerate in pursuit of extreme physiological results.

[0122] Weighted fusion of composite rewards: R_total = w1 * R_obj + w2 * R_sub. The setting of weights w1 and w2 is a key strategy, which can be preset at the system level or dynamically adjusted according to user types (such as more objective for muscle builders and more subjective for beginners). This design ensures that the optimization direction is effective and friendly Pareto frontier.

[0123] Building a state and action space close to the application scenario, comprehensive characterization of the state space: the state S_t is defined as the combination of the current parameters of the personalized fitness diet plan and the latest quantified disadvantage list of the user. This definition enables the model to fully perceive the overall picture of the current plan and the current condition of the user, which is the basis for making reasonable adjustment decisions.

[0124] Fine operation design of action space: action A_t is defined as a series of fine-tuning operations on the parameters of the plan, rather than generating a new plan. This ensures the safety and continuity of optimization. For example: Exercise action parameter adjustment: increase / decrease the number of sets and times of a specific action (such as "add 1 to the number of sit-up sets"); Action replacement and upgrade / downgrade: according to the association in the knowledge graph, replace "plank" with "kneeling plank" (downgrade) or "dynamic plank" (upgrade); Diet parameter fine-tuning: increase the protein intake ratio from 1.2g / kg to 1.5g / kg, or replace the carbohydrate source for breakfast. This design makes the exploration process controllable and low-risk.

[0125] Safe online learning based on policy gradient, considering direct interaction with users, it is essential to adopt safe strategies. For example, use the Proximal Policy Optimization (PPO) algorithm, which limits the amplitude of each policy update to avoid causing harm or poor experience to users due to a single bad exploration leading to drastic changes in the plan. The model is trained and updated centrally in the cloud using a large amount of anonymized interaction data from users, forming a general strategy, and then saving individualized strategy network parameters for each user, achieving a balance between universality and specificity.

[0126] There are huge individual differences and time-varying nature in human response to training and diet, and any initial plan cannot be perfectly predicted. The trial-and-error learning framework provided by reinforcement learning is for the machine to find the optimal strategy autonomously through interaction data in an environment full of uncertainties.

[0127] The achievement of fitness results depends on long-term adherence. This step optimizes the subjective experience reward, which essentially incorporates the user's psychological feelings and adherence willingness into the system's optimization goal, enabling the system to actively tailor a more easily adhered plan for the user, which is an indispensable mechanism for improving user retention and long-term results.

[0128] The application also provides an electronic device, comprising a processor and a memory storing a computer program, the processor implementing the method of any one of the preceding claims when executing the computer program.

[0129] Therefore, any effect in the body figure management method based on multi-modal data fusion can be achieved, which will not be repeated here.

[0130] The application further provides a computer readable storage medium, having a computer program stored thereon, the computer program being executed by a processor to implement the method of any one of the preceding claims.

[0131] Therefore, any effect in the body figure management method based on multi-modal data fusion can be achieved, which will not be repeated here.

[0132] The application can be implemented by using or referring to the existing technology for the unmentioned parts.

[0133] Each of the embodiments in the specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment mainly describes the difference from other embodiments.

[0134] The above only describes the embodiments of the application and is not used to limit the application. The application can be variously changed and modified by those skilled in the art. Any modification, equivalent replacement, improvement, etc. within the spirit and principle of the application shall be included in the scope of the claims of the application.

Claims

1. A method for managing body shape based on multimodal data fusion, characterized in that, The method operates in at least two working modes, including a general mode based on a first order of magnitude of key points and a high-precision mode based on more than a first order of magnitude of key points. The method includes: Collect multimodal data of users, including at least three-dimensional body model data, body composition physiological data, and user profile data containing user health contraindications and exercise capabilities; Based on the analysis of the three-dimensional body model data, the current working mode is determined, and three-dimensional morphological features are extracted accordingly. These features are then fused and analyzed with the body composition physiological data to generate a list of quantitative defects. Each defect includes at least the type, location, and severity quantification value. Based on the quantified list of shortcomings and the user profile data, a knowledge graph is queried to generate a personalized fitness and diet plan.

2. The method according to claim 1, characterized in that, Determine the current work mode, specifically including: The 3D body model data is processed using a pre-trained 3D keypoint detection model to identify keypoints; Determine whether the number of key points identified by the 3D key point detection model with a confidence level higher than the first confidence threshold has reached the first quantity threshold; If the target is reached, then enter high-precision mode; otherwise, execute general mode.

3. The method according to claim 2, characterized in that, After determining the current work mode, it also includes: If high-precision mode is entered, three-dimensional morphological features are extracted based on the identified key points with a confidence level higher than the first confidence threshold, which are used to generate a quantitative list of defects. If the general mode is executed, the three-dimensional morphological features are extracted based on the identified key points with a confidence level higher than the first confidence threshold to determine the core points, which are used to generate the list of quantitative defects.

4. The method according to claim 3, characterized in that, Extracting three-dimensional morphological features, specifically including: Based on the coordinates of the key points, the circumference and angular geometric features of each part of the body are calculated, and the core point is determined according to the rate of change of the geometric features. Texture analysis is performed on the surface of the three-dimensional body model data of the key point region to extract texture feature vectors used to distinguish the state of subcutaneous tissue.

5. The method according to claim 4, characterized in that, Generate a list of quantitative drawbacks, specifically including: The geometric features are compared with a standardized body model database to generate a preliminary assessment result based on size deviation. The texture feature vector is correlated with the body composition physiological data to correct the initial assessment results and determine the type and severity of the size deviation. Output a list of the quantified defects that includes quantification values ​​for type, location, and severity. Among them, under the same evaluation results, the size deviation value of the high-precision mode quantization primary evaluation result is smaller than that of the general mode.

6. The method according to claim 1, characterized in that, Generate personalized fitness and diet plans, specifically including: The type and position in the list of quantified defects are used as the first query conditions; Using the user health contraindications and exercise ability in the user profile data as the second query conditions, the movement action entities in the knowledge graph are filtered and matched; Based on the severity quantification value, initial training parameters are configured for the filtered and matched exercise movements, and corresponding dietary suggestions are matched for the user to obtain the personalized fitness diet plan.

7. The method according to claim 6, characterized in that, After generating a personalized fitness and diet plan, a security verification step is also included: Assess the risk level of the exercise movements in the personalized fitness diet plan, the degree of conflict between the dietary recommendations and the user's health contraindications, and obtain the safety fit index of the personalized fitness diet plan; If the security compatibility index is lower than a preset threshold, a manual review process will be automatically triggered.

8. The method according to claim 1, characterized in that, After generating a personalized fitness and diet plan, the process also includes dynamic optimization using a reinforcement learning model based on objective compliance data and subjective feedback data. Specifically: A composite reward signal is constructed, which is obtained by weighted summation of objective rewards based on changes in body circumference or body composition and subjective rewards based on user subjective ratings and attendance rates. Using the current parameters of the personalized fitness diet plan and the user's quantified shortcomings list as state inputs, and the action space of adjusting the plan parameters, the reinforcement learning model is trained and the strategy is updated using the composite reward signal to generate an optimized plan.

9. An electronic device, characterized in that, It includes a processor and a memory, the memory storing a computer program, and the processor executing the computer program to implement the method of any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that, It stores a computer program thereon, which, when executed by a processor, implements the method of any one of claims 1 to 8.

Images