AI weight loss and body shaping detection method and system based on multi-modal data fusion

Abstract

The application relates to an AI weight loss and body shaping detection method and system based on multi-modal data fusion, a computer device and a storage medium. The method comprises the following steps: obtaining target user body fat data and a whole body image; extracting a body fat feature vector and a visual feature vector, and obtaining a fusion feature vector by adopting an attention mechanism; inputting the fusion feature vector into a multi-task learning network to perform body type classification, body state diagnosis, girth estimation and obesity classification tasks; obtaining user behavior, physiological and psychological data, inputting the data into a dynamic weight distribution model to obtain a dynamic distribution weight; taking weight loss speed, scheme compliance and psychological acceptance as optimization objectives, taking the weight as a constraint, and solving an optimal conditioning scheme to generate a personalized conditioning scheme; inputting the fusion feature vector into an SMPL parameter prediction network to obtain body type parameters and posture parameters, and inputting the parameters and target weight loss days into a generative adversarial network to obtain a predicted human body grid. The application realizes a dynamic closed loop of accurate detection and personalized conditioning.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to an AI-based weight loss and body shaping detection method, system, computer equipment, and storage medium based on multimodal data fusion. Background Technology

[0002] With the rapid development of artificial intelligence technology, techniques have emerged that utilize body fat scales and computer vision to analyze body composition and assess body shape. These technologies collect structured data such as body fat percentage and weight, or use cameras to capture full-body images for body shape recognition, providing users with posture assessments and weight loss guidance.

[0003] Traditional technologies for weight loss and body shaping detection typically employ either a single body fat scale measurement or a single image assessment method. Body fat scale-based methods only provide numerical indicators such as weight and body fat percentage, failing to reflect the user's body shape distribution characteristics and specific circumference information. While image-based assessment methods can capture external body shape, they lack support from internal physiological indicators such as body fat percentage and visceral fat levels. Existing technologies have attempted to apply multimodal data to health management, but most employ simple data stitching or feature connection methods, failing to address the heterogeneous fusion problem between structured body fat data and image features. Furthermore, existing solutions are mostly one-time static detection and solution generation, outputting fixed conditioning suggestions based solely on initial detection results, leading to a disconnect between the generated conditioning plan and the user's actual implementation.

[0004] Furthermore, the aforementioned methods lack a deep fusion mechanism for multimodal data and a dynamic feedback mechanism for user status, making it impossible to establish a complete closed loop from accurate detection to personalized solution generation and real-time adjustment. This results in a disconnect between detection and intervention, affecting the effectiveness and feasibility of the solution. Summary of the Invention

[0005] Therefore, it is necessary to provide an AI-based weight loss and body shaping detection method, system, computer equipment, and storage medium that can achieve accurate body shape detection and dynamically generate personalized conditioning plans through multimodal data fusion to address the above-mentioned technical problems.

[0006] Firstly, this application provides an AI-based weight loss and body shaping detection method based on multimodal data fusion. The method includes: The system acquires the target user's body fat data and full-body image, extracts body fat feature vectors from the body fat data, extracts visual feature vectors from the full-body image, and uses an attention mechanism to fuse the body fat feature vectors and the visual feature vectors to obtain a fused feature vector. The fused feature vector is input into a multi-task learning network to perform body shape classification, body posture diagnosis, circumference estimation, and obesity typology tasks, and outputs the corresponding classification results, diagnosis results, estimation results, and typology results; the loss function of the multi-task learning network includes a constraint term based on physiological laws; The behavioral data, physiological data, and psychological data of the target user during use are obtained, and the behavioral data, physiological data, and psychological data are input into a dynamic weight allocation model to obtain dynamically allocated weights. Using weight loss speed, treatment plan compliance, and psychological acceptance as optimization objectives, and the dynamically allocated weights as constraints, the NSGA-II multi-objective optimization algorithm is used to solve the candidate treatment plan set to obtain the optimal treatment plan, and a personalized treatment plan is generated based on the optimal treatment plan. The fused feature vector is input into the SMPL parameter prediction network to obtain body shape parameters and posture parameters; the body shape parameters, posture parameters, and target weight loss days are input into the generator of the generative adversarial network to obtain the predicted human body mesh; the loss function of the generative adversarial network includes a constraint term based on metabolic dynamics.

[0007] In one embodiment, the attention mechanism fuses the body fat feature vector and the visual feature vector in the following manner: The body fat feature vector is multiplied by a first learnable weight matrix to obtain a first mapped feature; the visual feature vector is multiplied by a second learnable weight matrix to obtain a second mapped feature. The first mapping feature and the second mapping feature are added together and then mapped using the Sigmoid function to obtain the fused gated vector; The fusion gating vector, the body fat feature vector, and the visual feature vector are concatenated to obtain the fusion feature vector; The first learnable weight matrix and the second learnable weight matrix are optimized by gradient descent.

[0008] In one embodiment, the loss function of the multi-task learning network is constructed according to the following formula: in, Let be the loss function for the i-th task. For the j-th physiological constraint term, This is the body fat feature vector. The visual feature vector, The constraint weights are determined through cross-validation. The number of the physiological constraints.

[0009] In one embodiment, the loss function of the generative adversarial network is constructed according to the following formula: in, For Wasserstein loss term, This is a metabolic constraint term. Loss item for maintaining identity, and These are the weighting coefficients determined through cross-validation.

[0010] In one embodiment, the metabolic constraint term is constructed according to the following formula: in, For the i-th circumference, the target weight loss days The change within, Let be the metabolic kinetic function for the i-th dimension. This represents the current body fat percentage. The basal metabolic rate at the current moment; the metabolic kinetic function Determined through regression optimization based on historical user data.

[0011] In one embodiment, the dynamic weight allocation model determines the dynamic weight allocation in the following manner: The behavior weight is calculated using the following formula: in, This refers to the frequency of the target user's QR code check-in over the past 7 days. The sensitivity coefficient is determined through regression optimization using user data. Used as the baseline clock-in frequency; Physiological weights are calculated using the following formula: The psychological weight is calculated by multiplying the emotional polarity and the stress index. The emotional polarity is obtained by NLP analysis of the target user's messages on the interactive interface, and the stress index is obtained by calculating heart rate variability physiological data or by obtaining it through user questionnaires when heart rate variability data is missing.

[0012] In one embodiment, the NSGA-II multi-objective optimization algorithm uses weight loss speed, program compliance, and psychological acceptance as optimization objectives, and the objective function for weight loss speed is constructed according to the following formula: The objective function for the compliance of the scheme is constructed according to the following formula: The objective function for psychological acceptance is constructed according to the following formula: The objective function for weight loss speed incorporates a time decay factor, which is used to increase the contribution weight of recent weight changes to the objective function value of weight loss speed. The action completion rate is the proportion of the number of prescribed actions actually performed by the user to the total number of planned actions.

[0013] Secondly, this application also provides an AI-based weight loss and body shaping detection system based on multimodal data fusion. The system includes: The feature fusion module is used to acquire the target user's body fat data and full-body image, extract body fat feature vectors from the body fat data, extract visual feature vectors from the full-body image, and fuse the body fat feature vectors and the visual feature vectors using an attention mechanism to obtain a fused feature vector; The multi-task learning module is used to input the fused feature vector into the multi-task learning network, perform body shape classification, body posture diagnosis, circumference estimation and obesity classification tasks, and output the corresponding classification results, diagnosis results, estimation results and classification results; the loss function of the multi-task learning network includes a constraint term based on physiological laws; The weight allocation module is used to acquire the target user's behavioral data, physiological data, and psychological data during use, and input the behavioral data, physiological data, and psychological data into the dynamic weight allocation model to obtain dynamically allocated weights; The multi-objective optimization module is used to obtain the optimal treatment plan by solving the candidate treatment plan set with the dynamic allocation weight as the optimization objective, the NSGA-II multi-objective optimization algorithm as the constraint condition, and the optimal treatment plan by using the weight loss speed, plan compliance and psychological acceptance as the optimization objective, and generating a personalized treatment plan based on the optimal treatment plan. The body simulation module is used to input the fused feature vector into the SMPL parameter prediction network to obtain body shape parameters and posture parameters; the body shape parameters, posture parameters, and target weight loss days are input into the generator of the generative adversarial network to obtain a predicted human body mesh; the loss function of the generative adversarial network includes a constraint term based on metabolic dynamics.

[0014] Thirdly, this application also provides a computer device. The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the aforementioned AI-based weight loss and body shaping detection method based on multimodal data fusion.

[0015] Fourthly, this application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, which, when executed by a processor, implements the aforementioned AI-based weight loss and body shaping detection method based on multimodal data fusion.

[0016] The aforementioned AI-based weight loss and body shaping detection method, system, computer equipment, and storage medium based on multimodal data fusion extracts body fat feature vectors and visual feature vectors from body fat data and full-body images respectively, and fuses them using an attention mechanism. This comprehensively utilizes the complementary information of structured body fat data and visual image data, overcoming the shortcomings of incomplete information in single-modal data, and improving the accuracy of multiple prediction tasks such as body shape classification, body posture diagnosis, circumference estimation, and obesity classification. By setting physiologically based constraints in the loss function of the multi-task learning network, it ensures that all output results conform to the laws of human anatomy and metabolic dynamics, avoiding prediction results that violate physiological common sense. By acquiring user behavior data, physiological data, and psychological data during use and inputting them into the system, the system can effectively address these issues. The state-weighted allocation model can dynamically adjust the weights of various factors according to the user's real-time status, making the treatment plan more closely aligned with the user's actual situation. By using weight loss speed, treatment plan compliance, and psychological acceptance as optimization objectives and employing the NSGA-II multi-objective optimization algorithm to solve for the optimal treatment plan, it achieves balanced optimization among multiple objectives, avoiding the imbalance caused by optimizing a single objective. By inputting the fused feature vector into the SMPL parameter prediction network to obtain body shape and posture parameters, and combining this with the target weight loss days input into the generator of the generative adversarial network to obtain the predicted human body mesh, and by setting a metabolic dynamics-based constraint term in the loss function of the generative adversarial network, it can generate human body shape prediction results that conform to the laws of metabolic dynamics, providing a reliable basis for detection and treatment plan. Attached Figure Description

[0017] Figure 1 This is an application environment diagram of an AI-based weight loss and body shaping detection method based on multimodal data fusion in one embodiment; Figure 2 This is a flowchart illustrating an AI-based weight loss and body shaping detection method based on multimodal data fusion in one embodiment. Figure 3 This is a flowchart illustrating the process of fusing body fat feature vectors and visual feature vectors in one embodiment; Figure 4 This is a structural block diagram of an AI-based weight loss and body shaping detection system based on multimodal data fusion in one embodiment; Figure 5 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0019] The AI-based weight loss and body shaping detection method based on multimodal data fusion provided in this application embodiment can be applied to, for example... Figure 1 In the application environment shown, terminal 102 communicates with server 104 via a network. A data storage system can store the data that server 104 needs to process. The data storage system can be integrated onto server 104 or placed on a cloud or other network server. Terminal 102 collects body fat data and full-body images of the target user. After preliminary processing of the collected data locally, terminal 102 uploads it to server 104. Server 104 extracts body fat feature vectors from the body fat data and visual feature vectors from the full-body images, and fuses them using an attention mechanism to obtain a fused feature vector. Then, it completes multiple tasks such as body type classification, posture diagnosis, circumference estimation, and obesity classification through a multi-task learning network. Simultaneously, based on behavioral, physiological, and psychological data, it generates personalized conditioning plans using a dynamic weight allocation model and the NSGA-II multi-objective optimization algorithm, and generates a predictive human body mesh through an SMPL parameter prediction network and a generative adversarial network. Finally, the generated personalized conditioning plan and predictive human body mesh are returned to terminal 102 for display to the user. Terminal 102 can be a large-screen smart all-in-one machine deployed in stores (integrating or connecting an 8-electrode body fat scale and a 1080P wide-angle camera, running a full-screen Android APK application on the device side for triggering body fat detection and full-body photography), a user's smartphone (used to access the customer-side H5 via WeChat QR code to view the complete test report, check in on body shape progress, and leave feedback), or a tablet or personal computer (used by store managers or headquarters administrators to access the management backend to view data dashboards and customer profiles). Server 104 can be implemented using a standalone server or a server cluster consisting of multiple servers.

[0020] It should be noted that the body fat data and full-body images involved in this application embodiment are sensitive personal information as defined by relevant laws and regulations. Therefore, in practical applications, the collection, storage, use, and processing of the data involved in this application embodiment all adhere to the principles of legality, legitimacy, necessity, and good faith, have a clear and reasonable processing purpose, are directly related to the processing purpose, adopt the method that minimizes the impact on individual rights, and are limited to the minimum scope of achieving the processing purpose. Before processing the aforementioned sensitive personal information, the individual shall be informed truthfully, accurately, and completely, in a prominent manner and in clear and understandable language, of the processor's processing purpose and method, the type of information and retention period, the method and procedure for the individual to exercise their rights, and obtain the individual's consent. At the same time, strict protection measures such as end-to-end encryption and de-identification shall be taken for the above data, and a personal information protection impact assessment shall be conducted before processing activities involving sensitive personal information. Individuals have the right to withdraw their consent and request the deletion of their personal information at any time, and data processors shall provide convenient methods for withdrawing consent and deletion.

[0021] Firstly, in the field of weight loss and body shaping detection, body fat detection and image assessment are usually performed separately: body fat scales provide numerical physiological indicators such as weight and body fat percentage, while cameras capture full-body images to record external body shape information. Traditional detection methods use these two independently, failing to fully explore the complementary correlation between body fat data and image features, resulting in insufficient accuracy in predictive tasks such as body shape classification and circumference estimation. Simultaneously, there is a lack of dynamic correlation between the detection results and subsequent treatment plans; after detection, only a fixed set of treatment suggestions is output, unable to be adjusted according to the user's real-time status during the process, leading to a disconnect between the plan and the user's actual situation. Therefore, in one embodiment, such as... Figure 2 As shown, an AI-based weight loss and body shaping detection method based on multimodal data fusion is provided, which is then applied to… Figure 1 Taking terminal 102 as an example, the explanation includes the following steps: S1: Obtain the target user's body fat data and full-body image, extract the body fat feature vector from the body fat data, extract the visual feature vector from the full-body image, and use an attention mechanism to fuse the body fat feature vector and the visual feature vector to obtain the fused feature vector; S2: Input the fused feature vector into the multi-task learning network to perform body shape classification, body posture diagnosis, circumference estimation and obesity typing tasks, and output the corresponding classification results, diagnosis results, estimation results and typing results; the loss function of the multi-task learning network includes a constraint term based on physiological laws; S3: Obtain behavioral, physiological, and psychological data of the target user during use, and input the behavioral, physiological, and psychological data into the dynamic weight allocation model to obtain the dynamic weight allocation; S4: Using weight loss speed, program compliance, and psychological acceptance as optimization objectives, and dynamic weight allocation as constraints, the NSGA-II multi-objective optimization algorithm is used to solve the candidate treatment program set to obtain the optimal treatment program, and a personalized treatment program is generated based on the optimal treatment program. S5: Input the fused feature vector into the SMPL parameter prediction network to obtain body shape parameters and posture parameters; input the body shape parameters, posture parameters, and target weight loss days into the generator of the generative adversarial network to obtain the predicted human body mesh; the loss function of the generative adversarial network includes a constraint term based on metabolic dynamics.

[0022] In the above steps, body fat data reflects physiological indicators such as fat content, muscle mass, and water content in the target user's body. Body fat data can be acquired using a body fat scale, for example, by applying a weak current to the body using an 8-electrode body fat scale and measuring the impedance values ​​of each segment to calculate indicators such as body fat percentage and muscle mass. Full-body images record the target user's external body contours and can be captured using a camera. Body fat data and full-body images describe the target user's physical condition from two dimensions: internal physiology and external morphology, respectively, and the two complement each other.

[0023] After obtaining body fat data, feature extraction is performed to obtain a body fat feature vector. The body fat feature vector is a numerical representation of the body fat data in feature space. Simultaneously, feature extraction is performed on the full-body image to obtain a visual feature vector. The visual feature vector is a numerical representation of the full-body image in feature space, reflecting the external body shape characteristics of the target user. The body fat feature vector and visual feature vector originate from different modalities of data sources and possess different data characteristics; that is, the body fat feature vector is structured numerical data, while the visual feature vector is image feature data.

[0024] After obtaining the body fat feature vector and visual feature vector separately, an attention mechanism is used to fuse them. The attention mechanism dynamically assigns fusion weights based on the importance of each feature vector, ensuring the fusion result highlights information more critical to the current task. The fused feature vector obtained after fusion via the attention mechanism contains both the internal physiological information reflected by the body fat data and the external body shape information reflected by the full-body image, achieving complementarity and enhancement of multimodal information.

[0025] The fused feature vector is input into a multi-task learning network, which simultaneously performs four tasks: body type classification, posture diagnosis, circumference estimation, and obesity typology. The body type classification task identifies the target user's body type. The posture diagnosis task detects whether the target user has posture problems. The circumference estimation task predicts the circumference values ​​of various body parts of the target user. The obesity typology task determines the type of obesity the target user has. These four tasks share the same fused feature vector as input; through multi-task learning, the tasks can mutually reinforce each other and collectively improve prediction accuracy.

[0026] During the training of a multi-task learning network, its loss function includes a constraint term based on physiological principles. This physiological constraint term is used to ensure that the output results of each task conform to the laws of human anatomy and metabolic dynamics, avoiding prediction results that violate common physiological sense.

[0027] During the target user's use of the program, behavioral, physiological, and psychological data are continuously acquired. Behavioral data reflects the target user's adherence to and participation in the treatment plan. This data can be obtained by recording the frequency of scanning QR codes to view test reports and the completion rate of actions. Physiological data reflects the target user's current physical condition. This data can be obtained through periodic measurements using a body fat scale, including current body fat percentage, target body fat percentage, and maximum allowable body fat percentage. Psychological data reflects the target user's emotional tendencies and stress levels. This data can be obtained through NLP analysis of the target user's comments on the interactive interface to determine emotional polarity. These three types of data are then input into a dynamic weighting model, which outputs dynamically assigned weights based on the target user's real-time status. The dynamically assigned weights reflect the relative importance of the behavioral, physiological, and psychological dimensions in the current state.

[0028] Using weight loss speed, protocol compliance, and psychological acceptability as three optimization objectives, and dynamically allocated weights as constraints for each objective, the NSGA-II (Non-dominated Sorting Genetic Algorithm II) multi-objective optimization algorithm was employed to solve the set of candidate conditioning solutions. The NSGA-II multi-objective optimization algorithm is a Pareto optimal solution-based multi-objective evolutionary algorithm that can optimize multiple conflicting objectives in parallel without pre-setting the weights of each objective, resulting in a set of Pareto optimal solutions. From this Pareto optimal solution set, based on the current dynamically allocated weights (i.e., the relative importance of the three objectives—behavioral, physiological, and psychological—in the current state), the candidate solution that best fits the current weight distribution is selected as the optimal conditioning solution.

[0029] A personalized treatment plan is generated based on the optimal treatment plan. This personalized treatment plan is specifically tailored to the target user's current state and optimization goals, and includes specific content in multiple aspects such as dietary adjustments, exercise plans, and behavioral interventions.

[0030] The fused feature vectors are input into the SMPL parameter prediction network, which outputs the body shape and pose parameters of the SMPL model. SMPL (Skinned Multi-Person Linear Model) is a parameterized human body model that controls the shape and posture of the human body mesh through body shape and pose parameters. The aforementioned body shape parameters, pose parameters, and the target number of weight loss days are input into the generator of a generative adversarial network (GAN), which outputs a predicted human body mesh. This predicted human body mesh represents the predicted body shape of the target user after the target number of weight loss days. During the training of the GAN, its loss function includes a metabolic kinetics-based constraint term. This metabolic constraint term is used to ensure that the rate of change of the girth of the generated human body mesh conforms to metabolic kinetics, ensuring that the prediction results are physiologically reasonable.

[0031] Through the above methods, body fat data and full-body images are fused using an attention mechanism, comprehensively utilizing the complementary information of structured body fat data and visual image data. This overcomes the incompleteness of information in single-modality data and improves the accuracy of multiple prediction tasks such as body shape classification, body posture diagnosis, circumference estimation, and obesity classification. The physiological constraints of the multi-task learning network ensure that the output results conform to the laws of human anatomy and metabolic dynamics, avoiding prediction results that violate physiological common sense. The dynamic weight allocation model dynamically adjusts the weights of each factor according to the user's real-time status, making the plan adjustment more in line with the user's actual situation. The NSGA-II multi-objective optimization algorithm achieves a balanced optimization among the three objectives of weight loss speed, plan compliance, and psychological acceptance, avoiding plan imbalance caused by single-objective optimization. The combination of SMPL parameter prediction and metabolic constraint GAN makes the generated predicted human body grid conform to the laws of metabolic dynamics, providing a visual basis for detection and conditioning plans, thus establishing a complete closed loop from detection to personalized plan generation and real-time adjustment.

[0032] When fusing body fat feature vectors and visual feature vectors using an attention mechanism, it is necessary to determine a specific fusion method to ensure that the fusion result effectively reflects the importance differences between the two modalities. Traditional methods typically employ simple concatenation or weighted averaging, which makes it difficult to dynamically adjust the contribution level of each modality based on the input data. Therefore, in one embodiment, such as... Figure 3 As shown, the attention mechanism fuses body fat feature vectors and visual feature vectors in the following manner: The first learningable weight matrix is ​​used to perform matrix multiplication on the body fat feature vector to obtain the first mapping feature; the second learningable weight matrix is ​​used to perform matrix multiplication on the visual feature vector to obtain the second mapping feature. The first mapping feature and the second mapping feature are added together and then mapped using the Sigmoid function to obtain the fused gated vector; The fused gating vector, body fat feature vector, and visual feature vector are concatenated to obtain the fused feature vector; The first and second learnable weight matrices are optimized using gradient descent.

[0033] In the above steps, the body fat feature vector and the visual feature vector are first linearly transformed by a first learnable weight matrix and a second learnable weight matrix, respectively. The dimension of the first learnable weight matrix matches the dimension of the body fat feature vector, so that the result of the matrix multiplication (i.e., the first mapped feature) has the same dimension as the body fat feature vector. Specifically, if the body fat feature vector is... Then the first learnable weight matrix The first mapping feature is Similarly, if the visual feature vector is Then the second learnable weight matrix The second mapping feature is The first mapping feature and the second mapping feature are in the same position. In the feature space, subsequent addition operations can be performed.

[0034] Add the first mapping feature and the second mapping feature element by element, that is This yields a comprehensive feature. This comprehensive feature integrates information from both body fat and visual modalities. The comprehensive feature is then input into a Sigmoid function for element-wise mapping. The expression for the Sigmoid function is: It compresses the value of each element into an open interval between 0 and 1, resulting in a fusion gating vector. Each element in the fusion gating vector corresponds to a gating value between 0 and 1. These gating values ​​are used to control the degree to which features of each dimension in the body fat feature vector and visual feature vector are preserved in the final fusion result.

[0035] The fusion gate vector gg is combined with the original body fat feature vector. and the original visual feature vector The features are concatenated to obtain the fused feature vector. , where ⊕⊕ represents the vector concatenation operation. The dimension of this fused feature vector is . The gating part originates from the synthesis of features from two modalities, while the feature part retains the original information of both modalities. This allows the subsequent multi-task learning network to acquire interaction information between modalities while preserving the original features of each modality. (This is from an embodiment of the application.) The value of is subject to the actual network design; for example, it can be 64 dimensions. This application does not limit this value.

[0036] During model training, the first and second learnable weight matrices are used as network parameters and updated using the gradient descent optimization algorithm. In each iteration, the parameters are adjusted along the direction of gradient descent based on the gradient of the total loss function with respect to each parameter. and The value of gradually reduces the total loss of the model on the training data. The specific implementation of gradient descent optimization can employ stochastic gradient descent, the Adam optimizer, or other variants; this application does not limit this implementation. In this way, the first and second learnable weight matrices can adaptively learn the optimal mapping during training, thereby achieving the optimal fusion of body fat feature vectors and visual feature vectors.

[0037] Through the aforementioned learnable attention gating mechanism, the model can dynamically calculate the fusion gating vector based on the input data, achieving adaptive weighted fusion of body fat features and visual features. This allows the fused feature vector to highlight information that is more critical to the current prediction task, thereby improving the prediction accuracy of subsequent multi-task learning networks.

[0038] During the training of a multi-task learning network, a suitable loss function needs to be constructed to guide the optimization of network parameters. To enable collaborative learning among tasks while simultaneously being constrained by physiological laws, traditional loss functions simply perform a weighted sum of the losses for each task, failing to ensure that the output conforms to physiological common sense. Therefore, in one embodiment, the loss function of the multi-task learning network is constructed according to the following formula: in, Let be the loss function for the i-th task, where i ranges from 1 to 19, indicating that there are a total of 19 tasks. Let j be the j-th physiological constraint term, where j takes values ​​from 1 to... , This represents the number of physiological constraints. The constraint weights are determined through cross-validation, and their specific values ​​are determined through cross-validation.

[0039] The loss function described above consists of two parts: the first part is the sum of the loss functions for each task. The second part is the sum of all physiological constraints. With constraint weights The product of.

[0040] The first part consists of 19 tasks: a body shape classification task with 4 body shape categories (pear-shaped, apple-shaped, H-shaped, X-shaped); a posture diagnosis task with 4 posture problem diagnoses (hunchback, anterior pelvic tilt, false hip width, O / X-shaped legs); a circumference estimation task with 6 circumference estimations (waist, abdomen, hips, thigh, calf, and arm circumference); and an obesity classification task with 5 obesity types (fat obesity, edema obesity, low metabolism obesity, muscle laxity obesity, and postural deformity obesity), totaling 4+4+6+5=19 tasks. Each The loss function can be adapted to the task type. For example, cross-entropy loss can be used for classification tasks (body shape classification, body shape diagnosis, obesity classification), and mean squared error loss or mean absolute error loss can be used for regression tasks (circumference estimation). This application does not limit the specific loss function.

[0041] In Part Two, each physiological constraint is a function of the body fat feature vector and the visual feature vector. The specific form of the physiological constraint is determined according to the physiological laws it constrains. For example, one physiological constraint can ensure that the estimated waist-to-hip ratio is within a reasonable physiological range (waist-to-hip ratio should be greater than 0.7 and less than 1.0); another physiological constraint can ensure that the relationship between body fat percentage and body mass index conforms to the statistical laws of the normal population, that is, body fat percentage should not deviate from the normal range with changes in body mass index; yet another physiological constraint can ensure that the estimated circumferences satisfy the human body proportion relationship (thigh circumference should be greater than calf circumference, waist circumference should be smaller than hip circumference). The value of each physiological constraint can be the square or absolute value of the deviation between the predicted value and the reasonable physiological range. When the predicted value is within the reasonable range... The value is 0 when the predicted value deviates from the reasonable range. The value is positive.

[0042] Constraint weights It is a scalar coefficient used to control the proportion of physiological constraints in the total loss. The specific value is determined through cross-validation, the process of which is as follows: the training data is divided into K folds (e.g., K=5 or K=10), and on each fold, different values ​​are used... The model is trained using selected values, and its predictive performance is evaluated on the corresponding validation set. The value that minimizes the average prediction error for each task on the validation set is chosen. The value serves as the final constraint weight.

[0043] Using the multi-task learning loss function that includes physiological constraints, the total loss is... It can drive the model to achieve good predictive performance on various tasks, and ensure that the model output conforms to the laws of human anatomy and metabolic dynamics, thus avoiding unreasonable prediction results that may be caused by single-task optimization.

[0044] During the training of generative adversarial networks (GANs), a suitable loss function needs to be constructed to ensure that the generated human body mesh is both realistic and conforms to physiological laws. Traditional GANs only focus on the realism of the generated images and cannot guarantee the physiological rationality of the generated results. Therefore, in one embodiment, the loss function of the GAN is constructed according to the following formula: in, For Wasserstein loss term, This is a metabolic constraint term. Loss item for maintaining identity, and These are the weighting coefficients determined through cross-validation.

[0045] The loss function described above consists of three terms. The first term... The expression is ,in The data is based on real human body grids and follows a real data distribution. ; The conditional variables for the input generator (including body shape parameters, posture parameters, and target weight loss days) follow a prior distribution. ; For generator, The predicted human body mesh output by the generator; For discriminator, This represents the discriminator's classification score for the input sample. The Wasserstein loss minimizes the Wasserstein distance (also known as the bulldozer distance) between the real and generated data distributions, allowing the generator to learn a distribution that gradually approximates the real data distribution. Compared to traditional generative adversarial network losses, the Wasserstein loss provides smoother gradients, effectively mitigating training instability and mode collapse problems.

[0046] Second item The girth changes of the generated human body mesh used to constrain the laws of metabolic kinetics, the third term The identity preservation loss term is used to ensure that the generated predicted human body mesh is consistent with the input target user in terms of identity features. Specifically, it can be the mean square error or perceptual loss between the predicted human body mesh and the target user's current human body mesh. By minimizing this loss, the generator is driven to maintain the individual identity features of the target user during the process of changing body shape, so as to avoid the loss of the target user's individual features in the generated result.

[0047] Weighting coefficient The weighting coefficient μ controls the proportion of the metabolic constraint term in the total loss, while the weighting coefficient μ controls the proportion of the identity preservation loss term in the total loss. and The specific values ​​were determined through cross-validation. The specific process was as follows: the training data was divided into several folds, and the results were then... and The model is trained with different combinations of values, and the combination of values ​​that generates the best quality on the validation set is selected as the final weight coefficient. The evaluation index of generation quality can be the FID score (Fréchet Inception Distance) between the generated result and the real image, the user's rating of the similarity between the generated result and the target user, etc., and this application embodiment does not limit the specific metrics used.

[0048] By using the generative adversarial network loss function that includes Wasserstein loss, metabolic constraint term, and identity preservation loss, the generator is constrained by realism (Wasserstein loss) and physiological rationality (metabolic constraint term) during training, while maintaining the individual identity characteristics of the target user (identity preservation loss), thus generating human grid prediction results that are both realistic and in line with physiological laws, while preserving individual identity.

[0049] In the loss function of generative adversarial networks (GANs), the specific construction method of the metabolic constraint term directly affects whether the generated human body mesh conforms to metabolic dynamics. Therefore, in one embodiment, the metabolic constraint term Cmetab is constructed according to the following formula: The value of i ranges from 1 to 6, corresponding to six body circumference indicators: waist circumference, abdominal circumference, hip circumference, thigh circumference, calf circumference, and arm circumference. For the i-th circumference, the target weight loss days The change within the body, expressed in centimeters. Target weight loss days. The value can be 30 days, 60 days, or 90 days. This represents the current body fat percentage, expressed as a percentage (e.g., 25% is written as 0.25 in the formula). This is the basal metabolic rate at the current moment, expressed in kilocalories per day (e.g., 1500 kilocalories / day). Let be the metabolic kinetic function for the i-th circumference. This function takes the current body fat percentage and basal metabolic rate as inputs and outputs the theoretical maximum permissible rate of change of the i-th circumference per unit time (day), in centimeters per day.

[0050] This represents the actual predicted rate of change for the i-th circumference, expressed in centimeters per day. This actual predicted rate of change is then compared with the metabolic kinetic function. The output is compared with the theoretical maximum permissible rate of change. If the actual predicted rate of change is less than or equal to the theoretical maximum permissible rate of change, i.e. ,but A value of 0 indicates that the rate of change in this circumference has not exceeded the physiologically permissible upper limit, and no metabolic constraint loss occurs. If the actual predicted rate of change exceeds the theoretical maximum permissible rate of change, i.e. ,but The value representing this difference indicates that the rate of change in this dimensionality exceeds the physiologically permissible upper limit, resulting in a metabolic constraint loss proportional to this excess. The metabolic constraint losses for all six dimensions are summed to obtain the total metabolic constraint term. The larger the value of this metabolic constraint term, the less the dimensional changes of the generated human body mesh conform to metabolic kinetics.

[0051] The specific function form is determined through regression optimization based on historical user data. Specifically, a large amount of historical user data on body fat percentage, basal metabolic rate, and changes in various body measurements during the actual weight loss process are collected. The functional relationship between body fat percentage, basal metabolic rate, and the rate of change of each body measurement is fitted using linear or nonlinear regression methods.

[0052] During the training of a generative adversarial network, by minimizing the inclusion The total loss function drives the generator to adjust its output so that the rate of change of the girth of the generated predicted human body mesh gradually approaches the range that conforms to the laws of metabolic dynamics.

[0053] In the metabolic constraint terms, waist circumference, as a core indicator reflecting the degree of abdominal fat accumulation, has a significant impact on the constraint effect due to the specific form of its metabolic kinetic function. Therefore, in one embodiment, the metabolic kinetic function of waist circumference is calculated according to the following formula: In the above formula, fwaist = fwaist This represents the theoretical maximum permissible rate of change in waist circumference per unit time (day), expressed in centimeters per day. The coefficient for body fat percentage is -0.3, indicating that a higher body fat percentage results in a greater theoretical maximum permissible rate of change in waist circumference. This is because users with higher body fat percentages typically achieve faster waist reduction in the early stages of weight loss. The coefficient for basal metabolic rate is 0.1, indicating that a higher basal metabolic rate also results in a greater theoretical maximum permissible rate of change in waist circumference. This is because users with higher basal metabolic rates burn energy faster, and the theoretical upper limit for waist circumference change is correspondingly higher. Through the above linear combination, It can dynamically calculate the theoretical maximum allowable rate of change in waist circumference per unit time based on the target user's current body fat percentage and basal metabolic rate.

[0054] The regression coefficients (-0.3 and 0.1) in the above formula were determined through regression analysis of a large amount of historical user weight loss data, and can fit the statistical relationship between body fat percentage, basal metabolic rate, and the actual rate of change in waist circumference well. It should be noted that the specific values ​​of the above regression coefficients are obtained based on fitting a specific historical dataset. In practical applications, regression analysis can be performed again based on the actual collected historical user data to obtain regression coefficients applicable to the target user group. This application embodiment does not limit this. Furthermore, the metabolic kinetic functions for other circumferences (waist circumference, hip circumference, thigh circumference, calf circumference, and arm circumference) can adopt a form similar to that of waist circumference, with their regression coefficients fitted based on the corresponding historical data. This application embodiment does not specifically limit this.

[0055] In the dynamic weight allocation process, behavioral weights, physiological weights, and psychological weights need to be calculated separately to reflect the real-time state of the target user across different dimensions. Traditional solutions typically assign fixed weights to each dimension, failing to reflect individual user differences and state changes. Therefore, in one embodiment, the dynamic weight allocation model determines the dynamically allocated weights in the following manner: The behavior weight is calculated using the following formula: in, This refers to the frequency of QR code scanning and check-in by the target users over the past 7 days. The sensitivity coefficient is determined through regression optimization using user data. The benchmark attendance frequency.

[0056] Physiological weights are calculated using the following formula: The psychological weight is calculated by multiplying the emotional polarity and the stress index. The emotional polarity is obtained by NLP analysis of the target users' messages in the interactive interface, while the stress index is calculated by heart rate variability physiological data or obtained by user questionnaires when heart rate variability data is missing.

[0057] The above behavior weight formula uses the Sigmoid function form. The unit is times / day, reflecting the average activity level of the target user in viewing test reports and participating in treatment plans in the past week. This refers to a reference value for judging the frequency of user check-ins, measured in times per day. It is preset based on historical store data or industry standards, for example, set to 0.5 times per day (i.e., 3.5 check-ins per week). Greater than At that time, the index part When the value is less than 1, the behavior weight approaches 1; when Less than When the index is greater than 1, the behavioral weight approaches 0. Used to control the sensitivity of behavioral weights to changes in attendance frequency. The larger the value, the steeper the transition of the behavior weight around the baseline check-in frequency. The value of is determined through user data regression optimization: A large amount of historical user behavior data and its corresponding weight loss effect are collected. Using weight loss effect as the dependent variable and check-in frequency as the independent variable, the optimal value is fitted using logistic regression or other regression analysis methods. value.

[0058] In the above physiological weighting formula, the current body fat percentage is the target user's current measured body fat percentage (expressed as a percentage), the target body fat percentage is the target body fat percentage the target user hopes to achieve (expressed as a percentage), and the maximum allowable body fat percentage is the upper limit of body fat percentage determined based on the target user's age and gender (expressed as a percentage). For male users, the maximum allowable body fat percentage is usually set at 25%; for female users, it is usually set at 32%. The physiological weight reflects the degree of deviation between the target user's current body fat percentage and the target body fat percentage. The greater the deviation, the higher the physiological weight, indicating that more emphasis needs to be placed on improving physiological indicators during solution optimization.

[0059] In the aforementioned psychological weight calculation method, emotional polarity is obtained through natural language processing analysis of text messages entered by target users in the interactive interface. Specifically, text messages entered by target users in the interactive interface are collected, and a pre-trained language model is used to classify the sentiment of the message text, outputting an emotional polarity value between -1 and +1, where +1 represents the most positive sentiment, -1 represents the most negative sentiment, and 0 represents neutral sentiment. The stress index is calculated using heart rate variability (HRV) physiological data: HRV refers to the small differences between successive heartbeat cycles, usually quantified by indicators such as the root mean square (RMS) of the difference between adjacent RR intervals (the time interval between two consecutive R wave peaks in an electrocardiogram) or high-frequency power. When the human body is under stress, HRV usually decreases. Specifically, by collecting the target user's electrocardiogram signal, extracting the RR interval sequence, calculating the RMS value of the difference between adjacent RR intervals, and then mapping the RMS value of the difference between adjacent RR intervals to a stress index between 0 and 1 using a formula. In the absence of heart rate variability data, a stress index was obtained through user questionnaires. Users completed a standardized stress perception questionnaire, and the scores were mapped to a stress index ranging from 0 to 1. The psychological weight was the product of emotional polarity and the stress index: when emotional polarity was positive and the stress index was low, the psychological weight was positive and small; when emotional polarity was negative and the stress index was high, the psychological weight was negative and large in absolute value.

[0060] By calculating the behavioral weight, physiological weight, and psychological weight separately, the dynamic weight allocation model can dynamically allocate weights according to the real-time status of the target user, thus achieving adaptive adjustment of the weights in the three dimensions of behavior, physiology, and psychology.

[0061] In the NSGA-II multi-objective optimization algorithm, three objective functions need to be constructed separately: weight loss speed, protocol adherence, and psychological acceptance, to quantitatively evaluate the performance of different candidate treatment plans. Traditional single-objective optimization only focuses on weight loss effects, ignoring the feasibility of the plan and the user's psychological acceptance. Therefore, in one embodiment, the NSGA-II multi-objective optimization algorithm uses weight loss speed, protocol adherence, and psychological acceptance as optimization objectives. The objective function for weight loss speed is constructed as follows: The objective function for protocol compliance is constructed as follows: The objective function for psychological acceptance is constructed as follows: The objective function for weight loss speed incorporates a time decay factor, which increases the weight of recent weight changes in the objective function value of weight loss speed. The action completion rate is the proportion of the number of prescribed actions actually performed by the user to the total number of planned actions.

[0062] In the aforementioned weight loss rate target function, the current weight is the target user's current weight measurement (in kilograms), and the predicted target weight is the predicted weight value (in kilograms) that can be achieved after implementing the candidate conditioning program. The predicted target weight can be calculated using any existing metabolic equivalent method: based on the daily dietary calorie intake and exercise calorie expenditure in the candidate program, combined with the target user's basal metabolic rate, it is calculated through an energy balance equation, where 1 kilogram of adipose tissue is approximately equal to 7700 kcal of energy equivalent. The larger the value (in kilograms), the greater the expected weight loss and the faster the weight loss rate of the candidate treatment plan.

[0063] In the above compliance objective function, the check-in days refer to the actual number of days the target user checked in during the program's implementation period (in days), and the total number of days refers to the total number of days during the program's implementation period (in days). The ratio of these two reflects the overall participation frequency of the target user in the program. Action completion rate is the proportion of the number of prescribed actions actually performed by the user to the total number of planned actions. Specifically, the treatment plan specifies several actions that the user needs to perform (such as specific exercise actions or dietary record actions). The total number of planned actions is the total number of actions that should be completed as specified in the plan, and the actual number of prescribed actions performed is the number of actions actually completed by the user. The formula for calculating action completion rate is: Action completion rate equals the actual number of prescribed actions performed divided by the total number of planned actions. It is a dimensionless ratio, with a value between 0 and 1. The larger the value, the more likely the candidate treatment plan is to be actually implemented by the target user, and the higher the plan compliance.

[0064] In the aforementioned psychological acceptance objective function, the happiness score is the target user's overall subjective evaluation score of their current quality of life (in points), obtained through a standardized happiness scale. The solution satisfaction questionnaire score is the target user's satisfaction evaluation score of the current treatment plan (in points), obtained through a custom satisfaction questionnaire. The sum of the two ratings (in points) indicates that the higher the value, the more psychologically acceptable the candidate treatment plan is to the target user.

[0065] Secondly, the time decay factor gives higher weight to recent weight changes than earlier weight changes when calculating the rate of weight loss. This allows the optimization algorithm to focus more on the actual weight loss effect of candidate solutions at the current stage, enabling the solution to adapt more promptly to the user's current state. The specific form of the time decay factor can adopt any time-weighted method in the prior art, such as exponential decay or linear decay, and this application embodiment does not limit this. The population size and number of iterations of the NSGA-II multi-objective optimization algorithm can be set according to actual computing resources and accuracy requirements, for example, a population size of 100 and a number of iterations of 50, and this application embodiment does not limit this.

[0066] By constructing the above three objective functions, the NSGA-II multi-objective optimization algorithm can simultaneously optimize weight reduction effect, feasibility of the solution, and user psychological acceptance, achieving a balance between multiple objectives and avoiding the imbalance of the solution caused by single objective optimization.

[0067] During the data acquisition phase, the quality of body fat data and full-body images directly affects the accuracy of subsequent feature extraction and fusion. Therefore, in one embodiment, body fat data is acquired using an 8-electrode body fat scale. The scale outputs data via an RS232 / UART serial port with a baud rate of 115200, 8 data bits, 1 stop bit, no parity bit, ASCII plaintext data format, and a sampling frequency of 10Hz. The 8-electrode body fat scale applies a weak current (typically 50kHz to 500kHz AC, with a current intensity less than 1 mA) to the body through eight electrodes (feet and hands), measuring the impedance values ​​of different segments of the body to calculate indicators such as body fat percentage and muscle mass for the limbs and trunk. Compared to a 4-electrode body fat scale that only uses the feet, the 8-electrode scale can measure the segmental impedance of the upper limbs, lower limbs, and trunk separately, thus offering higher measurement accuracy. RS232 / UART serial port is a universal serial communication interface standard. The body fat scale and receiving device communicate via this serial port. The baud rate of 115200 indicates 115200 bits per second, 8 data bits indicate each data frame contains 8 data bits, 1 stop bit indicates each data frame contains 1 stop bit, and no parity bit indicates no parity check. These serial port parameters ensure stable data transmission between the body fat scale and the receiving device. The data format is ASCII plaintext, meaning the output data is presented in readable ASCII character form for easy parsing and processing. The sampling frequency is 10Hz, meaning the body fat scale collects 10 sets of data per second. Within a stable detection time of approximately 5 seconds, about 50 sets of data can be obtained. By averaging multiple sets of data, the stability and accuracy of the measurement results can be improved.

[0068] Full-body images are captured using a 1080P wide-angle, distortion-free camera. The camera's detection optical axis has an angle of 0° to 90° with the horizontal plane. The 1080P resolution (1920×1080 pixels) provides sufficient image detail for subsequent human keypoint detection and body shape analysis. The wide-angle, distortion-free lens can capture complete full-body images at a relatively short shooting distance (typically 1.5 to 2.5 meters), while the distortion-free design ensures that the proportions of different parts of the body in the image accurately reflect reality, avoiding body shape distortion caused by lens distortion. The 0° to 90° angle between the detection optical axis and the horizontal plane means that the camera can shoot horizontally (0° angle), overhead (90° angle), or at any angle in between. This angle range allows the camera to be flexibly positioned according to actual site conditions, such as front-mounted (approximately horizontal shooting) or top-mounted (approximately overhead shooting).

[0069] With the specific parameter configuration of the 8-electrode body fat scale and 1080P camera described above, body fat data and full-body images of the target user can be collected stably and accurately, providing a high-quality data foundation for subsequent feature extraction and fusion. In practical applications, other models or parameter configurations of the body fat scale and camera can be selected, as long as the corresponding data acquisition function can be achieved. This application embodiment does not limit this.

[0070] When extracting feature vectors from body fat data and full-body images, an appropriate network structure is needed to ensure the quality and efficiency of feature extraction. Considering the different characteristics of body fat data and image data, in one embodiment, the body fat feature vector is extracted through a body fat branch network, which is a multilayer perceptron (MLP). This network sequentially includes an input layer, a 128-dimensional fully connected layer, and a first output layer. The first output layer outputs a 64-dimensional body fat feature vector, and each layer uses the ReLU activation function. A multilayer perceptron is a feedforward neural network consisting of an input layer, several hidden layers, and an output layer. In this embodiment, the input layer receives 10 body fat data indicators as input. The 128-dimensional fully connected layer maps the 10-dimensional data from the input layer to a 128-dimensional space through a linear transformation. Specifically, this is implemented as follows: ,in The weight matrix is ​​128×10. The bias vector is 128-dimensional, and then nonlinearity is introduced by the ReLU activation function. The ReLU activation function is defined as follows: When the input is positive, the output equals the input; when the input is negative, the output is 0. This approach offers advantages such as simple computation and mitigating gradient vanishing. The first output layer further maps the 128-dimensional features to 64 dimensions through a linear transformation. Output a 64-dimensional body fat feature vector. .

[0071] Visual feature vectors are extracted using a visual branch network, specifically a ResNet-18 network. The output of the ResNet-18 network is mapped to a 64-dimensional visual feature vector via a fully connected layer. ResNet-18 is a residual network containing 17 convolutional layers and 1 fully connected layer (a total of 18 layers). Its residual connection structure effectively mitigates the vanishing gradient problem during deep network training. The ResNet-18 network takes a 1080P full-body image as input. After feature extraction through convolutional blocks and pooling layers, it outputs a feature representation (typically 512-dimensional). This feature representation is further transformed into a 64-dimensional visual feature vector through a fully connected layer. .

[0072] Through the aforementioned body fat branch network and visual branch network, body fat data and full-body images are mapped to feature vectors of the same dimension (64-dimensional), which facilitates subsequent attention mechanism fusion.

[0073] When predicting SMPL model parameters from fused feature vectors, an appropriate network structure is needed to ensure the prediction accuracy of body shape and pose parameters. The body shape and pose parameters of the SMPL model have different semantic meanings and dimensions, making separate predictions more reasonable. Therefore, in one embodiment, the SMPL parameter prediction network includes a body shape prediction branch and a pose parameter prediction branch. The body shape prediction branch sequentially comprises an input layer, a 128-dimensional fully connected layer, a 64-dimensional fully connected layer, a 32-dimensional fully connected layer, and a 10-dimensional output layer, employing the ReLU activation function to output 10-dimensional body shape parameters. The attitude parameter prediction branch consists of an input layer, a 128-dimensional fully connected layer, a 64-dimensional fully connected layer, a 32-dimensional fully connected layer, and a 72-dimensional output layer, using the ReLU activation function to output 72-dimensional attitude parameters. .

[0074] SMPL (Skinned Multi-Person Linear Model) is a parametric human body model that uses body shape parameters. and attitude parameters Control the shape and posture of the human body mesh. Body type parameters. This is a 10-dimensional vector, with each dimension corresponding to the first 10 principal component directions in the human body shape principal component analysis, controlling overall body shape characteristics such as fatness / thinness, height, limb proportions, and trunk proportions. Posture parameters. It is a 72-dimensional vector containing the rotation angles of 24 joints (each joint has 3 rotational degrees of freedom), where joint 0 is the root joint (pelvis) and the remaining 23 joints are sub-joints. The rotation angles of each joint are represented using the axis-angle method.

[0075] The body shape parameter prediction branch and the pose parameter prediction branch share the same input, namely the fused feature vector, but have different network parameters and output dimensions. Both branches use a fully connected network structure: the input layer receives the fused feature vector, which is then passed through fully connected layers of 128, 64, and 32 dimensions for feature transformation and dimensionality reduction, and finally outputs 10-dimensional body shape parameters respectively. and 72-dimensional attitude parameters Each layer employs the ReLU activation function to introduce nonlinearity. Through this dual-branch structure, the SMPL parameter prediction network can extract body shape-related information and pose-related information separately from the fused feature vector, achieving independent prediction of body shape parameters and pose parameters.

[0076] When generating predicted human body meshes from SMPL parameters, an appropriate generative adversarial network (GAN) structure is needed to ensure the generated meshes are of high quality and realistic. Therefore, in one embodiment, the generator is based on the StyleGAN architecture, sequentially comprising an input layer, a 128-dimensional fully connected layer, a 256-dimensional fully connected layer, a 512-dimensional fully connected layer, a 1024-dimensional fully connected layer, and a generator output layer. The generator output layer outputs the predicted human body mesh; each layer uses the Leaky ReLU activation function. StyleGAN (Style Generative Adversarial Network) is an improved GAN architecture that, by introducing a style modulation mechanism, can generate images with high realism and controllability. In this embodiment, the generator input includes the body shape parameters of the SMPL model. Attitude parameters and target number of days to lose weight (1-dimensional scalar), with a total of 83 dimensions of input. After receiving the above input, the input layer sequentially passes through fully connected layers of 128, 256, 512, and 1024 dimensions for progressive feature transformation and dimensionality upscaling. Finally, the generator output layer outputs the predicted human body mesh. The predicted human body mesh is a 3D mesh model containing 6890 vertices and 13776 triangular faces, where the coordinates of each vertex are derived from 3D spatial coordinates. This indicates that each layer uses the Leaky ReLU activation function, and the expression for Leaky ReLU is: ,in It is a small positive number (usually 0.01 or 0.2), which allows a small non-zero gradient when the input is negative, thus avoiding the neuron death problem that ReLU may cause.

[0077] The discriminator of the generative adversarial network (GAN) consists of an input layer, a 512-dimensional fully connected layer, a 256-dimensional fully connected layer, a 128-dimensional fully connected layer, and a discriminator output layer. Each layer uses the Leaky ReLU activation function. The input to the discriminator is a human body mesh (including real human body meshes and predicted human body meshes generated by the generator). This mesh undergoes progressive feature extraction and dimensionality reduction through the 512-dimensional, 256-dimensional, and 128-dimensional fully connected layers. Finally, the discriminator output layer outputs a discrimination score. This score is a scalar value representing the probability that the input human body mesh is a real human body mesh; the closer the value is to 1, the more realistic the discriminator considers the input. By continuously distinguishing between real and generated human body meshes, the discriminator forces the generator to continuously improve its generation quality, thus forming adversarial training.

[0078] It should be understood that although the steps in the flowcharts of the above embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the above embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages in other steps.

[0079] Secondly, based on the same inventive concept, this application also provides a system for implementing the AI-based weight loss and body shaping detection method based on multimodal data fusion described above. The solution provided by this system is similar to the implementation described in the above method; therefore, the specific limitations in one or more system embodiments provided below can be found in the limitations of the AI-based weight loss and body shaping detection method based on multimodal data fusion described above, and will not be repeated here.

[0080] In one embodiment, such as Figure 4 As shown, the system includes: a feature fusion module, a multi-task learning module, a weight allocation module, a multi-objective optimization module, and a body size simulation module. Among them: The feature fusion module is used to acquire the target user's body fat data and full-body image, extract body fat feature vectors from the body fat data, extract visual feature vectors from the full-body image, and use an attention mechanism to fuse the body fat feature vectors and visual feature vectors to obtain a fused feature vector. The multi-task learning module is used to input the fused feature vector into the multi-task learning network to perform body shape classification, body posture diagnosis, circumference estimation, and obesity typing tasks, and output the corresponding classification results, diagnosis results, estimation results, and typing results; the loss function of the multi-task learning network includes a constraint term based on physiological laws; The weight allocation module is used to acquire the behavioral, physiological, and psychological data of the target user during use, and input the behavioral, physiological, and psychological data into the dynamic weight allocation model to obtain the dynamically allocated weights. The multi-objective optimization module is used to obtain the optimal treatment plan by solving the candidate treatment plan set with the optimization objectives of weight loss speed, plan compliance and psychological acceptance, and with dynamic weight allocation as the constraint. The module then generates a personalized treatment plan based on the optimal treatment plan. The body simulation module is used to input the fused feature vector into the SMPL parameter prediction network to obtain body shape parameters and posture parameters; the body shape parameters, posture parameters, and target weight loss days are input into the generator of the generative adversarial network to obtain the predicted human body mesh; the loss function of the generative adversarial network includes a constraint term based on metabolic dynamics.

[0081] In one embodiment, the feature fusion module includes a fusion processing unit. Specifically, the feature fusion module performs matrix multiplication on the body fat feature vector using a first learnable weight matrix to obtain a first mapped feature; and performs matrix multiplication on the visual feature vector using a second learnable weight matrix to obtain a second mapped feature. The feature fusion module then adds the first and second mapped features and maps them using a sigmoid function to obtain a fusion gate vector. Finally, the feature fusion module concatenates the fusion gate vector, the body fat feature vector, and the visual feature vector to obtain a fusion feature vector. The feature fusion module also optimizes the first and second learnable weight matrices using gradient descent.

[0082] In one embodiment, the multi-task learning module includes a network loss function construction unit. This unit is used to construct the loss function of the multi-task learning network according to the following formula: in, Let be the loss function for the i-th task. For the j-th physiological constraint term, This is a body fat feature vector. For visual feature vectors, The constraint weights are determined through cross-validation. This represents the number of physiological constraints.

[0083] In one embodiment, the body size simulation module includes an adversarial network loss function construction unit. This unit is used to construct the loss function for generating the adversarial network according to the following formula: in, For Wasserstein loss term, This is a metabolic constraint term. Loss item for maintaining identity, and These are the weighting coefficients determined through cross-validation.

[0084] In one embodiment, the adversarial network loss function construction unit includes a metabolic constraint term construction subunit. This metabolic constraint term construction subunit is used to construct metabolic constraint terms according to the following formula: in, For the i-th circumference, the target weight loss days The change within, Let be the metabolic kinetic function for the i-th dimension. This represents the current body fat percentage. Basal metabolic rate at the current moment; metabolic kinetic function Determined through regression optimization based on historical user data.

[0085] In one embodiment, the weight allocation module includes a dynamic weight allocation model module. The dynamic weight allocation model module is used to determine the dynamically allocated weights in the following manner: Behavioral weights are calculated using the following formula: in, This refers to the frequency of QR code scanning and check-in by the target users over the past 7 days. The sensitivity coefficient is determined through regression optimization using user data. Used as the baseline clock-in frequency; Physiological weights are calculated using the following formula: The psychological weight is calculated as the product of emotional polarity and stress index. Emotional polarity is obtained by NLP analysis of the target users' messages in the interactive interface, and stress index is obtained by heart rate variability physiological data or by user questionnaires when heart rate variability data is missing.

[0086] In one embodiment, the NSGA-II multi-objective optimization algorithm uses weight loss speed, program compliance, and psychological acceptability as optimization objectives. The multi-objective optimization module includes an optimization objective construction unit. This optimization objective construction unit is used to construct the objective function for weight loss speed according to the following formula: Optimize the target building units to construct the objective function for scheme compliance according to the following formula: The optimized target building unit is used to construct the objective function of psychological acceptability according to the following formula: The objective function for weight loss speed incorporates a time decay factor, which increases the weight of recent weight changes in the objective function value of weight loss speed. The action completion rate is the proportion of the number of prescribed actions actually performed by the user to the total number of planned actions.

[0087] The modules in the aforementioned AI-based weight loss and body shaping detection system based on multimodal data fusion can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the computer device's memory as software, so that the processor can call and execute the corresponding operations of each module.

[0088] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 5 As shown, the computer device includes a processor, memory, and a network interface connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database stores all data required for system operation. The network interface communicates with external terminals via a network connection. When executed by the processor, the computer program implements an AI-based weight loss and body shaping detection method based on multimodal data fusion.

[0089] Those skilled in the art will understand that Figure 5 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0090] Thirdly, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of any of the methods described in the above method embodiments.

[0091] Fourthly, a computer-readable storage medium is provided, on which a computer program is stored, which, when executed by a processor, implements the steps of any of the methods described in the above method embodiments.

[0092] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0093] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0094] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. An AI-based weight loss and body shaping detection method based on multimodal data fusion, characterized in that, The method includes: The system acquires the target user's body fat data and full-body image, extracts body fat feature vectors from the body fat data, extracts visual feature vectors from the full-body image, and uses an attention mechanism to fuse the body fat feature vectors and the visual feature vectors to obtain a fused feature vector. The fused feature vector is input into a multi-task learning network to perform body shape classification, body posture diagnosis, circumference estimation, and obesity typology tasks, and outputs the corresponding classification results, diagnosis results, estimation results, and typology results; the loss function of the multi-task learning network includes a constraint term based on physiological laws; The behavioral data, physiological data, and psychological data of the target user during use are obtained, and the behavioral data, physiological data, and psychological data are input into a dynamic weight allocation model to obtain dynamically allocated weights. Using weight loss speed, treatment plan compliance, and psychological acceptance as optimization objectives, and the dynamically allocated weights as constraints, the NSGA-II multi-objective optimization algorithm is used to solve the candidate treatment plan set to obtain the optimal treatment plan, and a personalized treatment plan is generated based on the optimal treatment plan. The fused feature vector is input into the SMPL parameter prediction network to obtain body shape parameters and posture parameters; the body shape parameters, posture parameters, and target weight loss days are input into the generator of the generative adversarial network to obtain the predicted human body mesh; the loss function of the generative adversarial network includes a constraint term based on metabolic dynamics.

2. The method according to claim 1, characterized in that, The attention mechanism fuses the body fat feature vector and the visual feature vector in the following manner: The body fat feature vector is multiplied by a first learnable weight matrix to obtain a first mapped feature; the visual feature vector is multiplied by a second learnable weight matrix to obtain a second mapped feature. The first mapping feature and the second mapping feature are added together and then mapped using the Sigmoid function to obtain the fused gated vector; The fusion gating vector, the body fat feature vector, and the visual feature vector are concatenated to obtain the fusion feature vector; The first learnable weight matrix and the second learnable weight matrix are optimized by gradient descent.

3. The method according to claim 1, characterized in that, The loss function of the multi-task learning network is constructed according to the following formula: in, Let be the loss function for the i-th task. For the j-th physiological constraint term, This is the body fat feature vector. The visual feature vector, The constraint weights are determined through cross-validation. The number of the physiological constraints.

4. The method according to claim 1, characterized in that, The loss function of the generative adversarial network is constructed according to the following formula: in, For Wasserstein loss term, This is a metabolic constraint term. Loss item for maintaining identity, and These are the weighting coefficients determined through cross-validation.

5. The method according to claim 4, characterized in that, The metabolic constraint term is constructed according to the following formula: in, For the i-th circumference, the target weight loss days The change within, Let be the metabolic kinetic function for the i-th dimension. This represents the current body fat percentage. The basal metabolic rate at the current moment; the metabolic kinetic function Determined through regression optimization based on historical user data.

6. The method according to claim 1, characterized in that, The dynamic weight allocation model determines the dynamic weight allocation in the following manner: The behavior weight is calculated using the following formula: in, This refers to the frequency of the target user's QR code check-in over the past 7 days. The sensitivity coefficient is determined through regression optimization using user data. Used as the baseline clock-in frequency; Physiological weights are calculated using the following formula: The psychological weight is calculated by multiplying the emotional polarity and the stress index. The emotional polarity is obtained by NLP analysis of the target user's messages on the interactive interface, and the stress index is obtained by calculating heart rate variability physiological data or by obtaining it through user questionnaires when heart rate variability data is missing.

7. The method according to claim 1, characterized in that, The NSGA-II multi-objective optimization algorithm uses weight loss speed, program compliance, and psychological acceptance as optimization objectives. The objective function for weight loss speed is constructed according to the following formula: The objective function for the compliance of the scheme is constructed according to the following formula: The objective function for psychological acceptance is constructed according to the following formula: The objective function for weight loss speed incorporates a time decay factor, which is used to increase the contribution weight of recent weight changes to the objective function value of weight loss speed. The action completion rate is the proportion of the number of prescribed actions actually performed by the user to the total number of planned actions.

8. An AI-based weight loss and body shaping detection system based on multimodal data fusion, characterized in that, The system includes: The feature fusion module is used to acquire the target user's body fat data and full-body image, extract body fat feature vectors from the body fat data, extract visual feature vectors from the full-body image, and fuse the body fat feature vectors and the visual feature vectors using an attention mechanism to obtain a fused feature vector; The multi-task learning module is used to input the fused feature vector into the multi-task learning network, perform body shape classification, body posture diagnosis, circumference estimation and obesity classification tasks, and output the corresponding classification results, diagnosis results, estimation results and classification results; the loss function of the multi-task learning network includes a constraint term based on physiological laws; The weight allocation module is used to acquire the target user's behavioral data, physiological data, and psychological data during use, and input the behavioral data, physiological data, and psychological data into the dynamic weight allocation model to obtain dynamically allocated weights; The multi-objective optimization module is used to obtain the optimal treatment plan by solving the candidate treatment plan set with the dynamic allocation weight as the optimization objective, the NSGA-II multi-objective optimization algorithm as the constraint condition, and the optimal treatment plan by using the weight loss speed, plan compliance and psychological acceptance as the optimization objective, and generating a personalized treatment plan based on the optimal treatment plan. The body simulation module is used to input the fused feature vector into the SMPL parameter prediction network to obtain body shape parameters and posture parameters; the body shape parameters, posture parameters, and target weight loss days are input into the generator of the generative adversarial network to obtain a predicted human body mesh; the loss function of the generative adversarial network includes a constraint term based on metabolic dynamics.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the AI ​​weight loss and body shaping detection method based on multimodal data fusion as described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the AI ​​weight loss and body shaping detection method based on multimodal data fusion as described in any one of claims 1 to 7.

Images