A health condition digitalization integrated method and system applied to a smart mirror

Abstract

This invention relates to the field of data integration technology, and more particularly to a method and system for digitally integrating health status using a smart mirror. The method includes the following steps: acquiring a user's facial image and full-body motion image using a camera embedded in the smart mirror; enhancing the details of the user's facial image to generate an enhanced facial image; performing facial structure analysis on the enhanced facial image to generate facial structure data; performing micro-expression recognition on the enhanced facial image based on the facial structure data to generate user emotion data; performing user visual feature analysis on the enhanced facial image to generate user visual feature data; performing pupil morphology change analysis on the enhanced facial image based on the user visual feature data to generate pupil morphology feature data; and performing optical difference sensitivity analysis on the pupil morphology feature data to generate user eye physiological signal data. This invention achieves efficient and accurate digital integration of health status.

Description

Technical Field

[0001] This invention relates to the field of data integration technology, and in particular to a method and system for digitally integrating health status data for use in smart mirrors. Background Technology

[0002] Smart mirrors are an emerging type of intelligent health device that combines mirrors and smart technology to provide a digitally integrated approach to personal health status. As people's demand for health monitoring and management increases, smart mirrors have become a convenient and practical tool. By integrating health data, they can monitor users' health status in real time and provide personalized health advice. Traditional methods of digitally integrating health status often rely on off-the-ground sensor devices or observation by medical professionals, which suffer from low efficiency and poor accuracy. To meet the needs of modern health monitoring, an intelligent method for digitally integrating health status applied to smart mirrors is needed. Summary of the Invention

[0003] To address the aforementioned technical problems, this invention proposes a method and system for digitally integrating health status data in smart mirrors, comprising the following steps:

[0004] Step S1: Acquire user facial images and full-body motion images using the embedded camera in the smart mirror; enhance the details of the user's facial images to generate enhanced facial images; perform facial structure analysis on the enhanced facial images to generate facial structure data; perform micro-expression recognition on the enhanced facial images based on the facial structure data to generate user emotion data.

[0005] Step S2: Perform user visual feature analysis on the enhanced facial image to generate user visual feature data; perform pupil morphology change analysis on the enhanced facial image based on the user visual feature data to generate pupil morphology feature data; perform optical difference sensitivity analysis on the pupil morphology feature data to generate user eye physiological signal data.

[0006] Step S3: Perform facial fine skin analysis on the enhanced facial image based on user emotion data to generate facial fine skin feature data; construct a user facial profile based on user eye physiological signal data and facial fine skin feature data to build a digital facial model;

[0007] Step S4: Perform dynamic feature recognition on the full-body motion image to generate user dynamic feature data; perform muscle group morphology and structure analysis on the full-body motion image based on the user dynamic feature data to generate muscle group morphology and structure data; perform muscle health status analysis on the muscle group morphology and structure data to generate muscle health status data.

[0008] Step S5: Perform 3D skeleton reconstruction on the full-body motion image to generate a 3D skeleton model of the user; perform skeletal structure feature analysis on the 3D skeleton model of the user to generate skeletal structure feature data; perform abnormal site analysis on the skeletal structure feature data to generate abnormal skeletal structure data.

[0009] Step S6: Use deep learning algorithms to perform real-time fusion analysis on muscle health status data and skeletal abnormality structure data to generate dynamic musculoskeletal data for users; perform potential risk trend analysis on the dynamic musculoskeletal data to generate musculoskeletal risk trend data; use recurrent convolution algorithms to perform holographic visual modeling on the musculoskeletal risk trend data and digital facial model to construct a dynamic holographic visual model to perform digital integration of health status.

[0010] This invention provides a comprehensive observation of a user's appearance and dynamic posture by acquiring facial and full-body motion images. Detail enhancement improves the clarity and visibility of facial images. Facial structure analysis extracts facial feature information, such as facial contours and eye positions. Micro-expression recognition captures the user's emotional state through the analysis of subtle facial movements. User visual feature data provides detailed information about the user's facial appearance, such as skin condition and facial features. Pupil morphology change analysis reveals the user's attention level and emotional state. Pupil morphology feature data and ocular physiological signal data provide quantitative information about the user's visual perception and attention level. Facial fine skin analysis provides information about the user's skin health and texture. The construction of the user's facial profile is based on emotional data, ocular physiological signal data, and facial fine skin feature data, comprehensively describing the user's facial features and emotional state. The digital facial model serves as a visual representation of the user's facial features, providing a foundation for subsequent health status analysis. Dynamic features... This system identifies and analyzes user postures and movement patterns, providing information about user activity status and posture. Muscle group morphology analysis reveals the distribution, shape, and relationships of user muscles, providing information on muscle structure. Muscle health data assesses the health of user muscles, including indicators such as muscle strength and flexibility. User 3D skeleton reconstruction provides a 3D representation of user skeletal posture and structure. Skeletal structure feature analysis extracts key skeletal features, such as bone length and angles. Skeletal abnormality structure data identifies abnormal parts of the skeletal structure, such as fractures and deformities, providing information on skeletal health. Real-time fusion analysis combines muscle health data and skeletal abnormality structure data to comprehensively assess the user's musculoskeletal health, providing comprehensive musculoskeletal analysis results. Potential risk trend analysis predicts the trend of user's musculoskeletal health, helping to detect health problems early and take corresponding measures. Dynamic holographic visual models combine musculoskeletal risk trend data with digital facial models to provide intuitive visualization, facilitating doctors and users to understand and analyze health conditions.

[0011] Preferably, step S1 includes the following steps:

[0012] Step S11: Acquire the user's facial image and full-body motion image using the embedded camera in the smart mirror;

[0013] Step S12: Enhance the details of the user's facial image to generate a detail-enhanced facial image;

[0014] Step S13: Perform facial feature node recognition on the detail-enhanced facial image to generate facial feature node position data;

[0015] Step S14: Perform facial structure analysis on the detail-enhanced facial image based on the facial feature node position data to generate facial structure data;

[0016] Step S15: Perform micro-expression recognition on the enhanced facial image based on facial structure data to generate user micro-expression data;

[0017] Step S16: Perform emotional fluctuation analysis on the user's micro-expression data to generate user emotional data.

[0018] This invention acquires user facial images and full-body motion images for comprehensive observation of the user's appearance and dynamic posture. Detail enhancement improves the visibility of facial images by enhancing image detail and clarity. This enhanced facial image detail contributes to the accuracy and reliability of subsequent tasks such as facial feature node recognition, facial structure analysis, and micro-expression recognition. Facial feature node recognition accurately locates facial features such as the eyes, nose, and mouth. The location data of these facial features provides crucial information for subsequent facial structure analysis and micro-expression recognition. Facial structure analysis extracts facial feature information, such as facial contours, eye positions, and mouth shapes. This facial structure data provides a quantitative description of the user's facial morphology and features, laying the foundation for subsequent micro-expression recognition and emotion fluctuation analysis. Micro-expression recognition captures subtle changes in the user's expressions and emotions by analyzing minute facial movements. The user's micro-expression data provides a quantitative description of the user's emotional state. Emotion fluctuation analysis identifies and analyzes the trends and patterns of the user's emotional changes. The user's emotional data provides a comprehensive assessment of the user's emotional state, helping to understand the user's emotional preferences, emotional stress, and other information.

[0019] Preferably, the specific steps of step S2 are as follows:

[0020] Step S21: Perform user visual feature analysis on the detail-enhanced facial image to generate user visual feature data;

[0021] Step S22: Perform eye trajectory optical flow tracking on the detail-enhanced facial image based on the user's visual feature data to generate user eye trajectory data;

[0022] Step S23: Perform gaze focus identification on the user's eye trajectory data to generate the eye gaze focus;

[0023] Step S24: Analyze pupil morphology changes in the detail-enhanced facial image based on eye gaze focus to generate pupil morphology feature data;

[0024] Step S25: Perform optical difference sensitivity analysis on the pupil morphology feature data to generate user eye physiological signal data.

[0025] This invention extracts various visual features from facial images through user visual feature analysis, such as skin condition, facial symmetry, and facial proportions. Eye trajectory optical flow tracking captures the user's eye movement trajectory in the image, including changes in the fixation point and the direction of eye movement. User eye trajectory data provides a description of the user's attention focus and eye movement patterns. Fixation focus identification determines the user's fixation point in the facial image, i.e., the location where the user's current attention is focused. Eye fixation focus provides a description of the user's points of interest and areas of focus. Pupil morphology change analysis identifies the size, shape, and dynamic changes of the pupil, reflecting the user's physiological and psychological state in different situations. Pupil morphology feature data provides a quantitative description of the user's emotions, cognitive load, and cognitive activity level, which can be used in fields such as emotion recognition and cognitive research. Optical difference sensitivity analysis infers the user's physiological state and circadian rhythm by analyzing the pupil's sensitivity to changes in light. User eye physiological signal data provides a quantitative description of the user's physiological changes, fatigue level, and emotional fluctuations, which can be used for applications such as health monitoring and fatigue assessment.

[0026] Preferably, the specific steps of step S24 are as follows:

[0027] Step S241: Perform pupil scaling analysis on the detail-enhanced facial image based on eye gaze focus to generate pupil scaling data;

[0028] Step S242: Perform curve fitting on the pupil dilation data to generate a pupil dilation curve;

[0029] Step S243: Perform edge contour evolution analysis on the detail-enhanced facial image based on the pupil scaling curve to generate pupil edge contour change data;

[0030] Step S244: Perform non-circularity structure calculation on the pupil edge contour change data to generate pupil contour non-circularity parameters;

[0031] Step S245: Analyze the dynamic change characteristics of the pupil by using the non-circularity parameter of the pupil contour to generate the dynamic pupil scaling pattern;

[0032] Step S246: Analyze the pupil morphology changes in the detail-enhanced facial image based on the dynamic scaling rules of the pupil to generate pupil morphology feature data.

[0033] This invention measures pupil diameter changes through pupil dilation analysis, adjusting pupil size based on the position and distance of the focal point. Pupil dilation data provides a quantitative description of the user's visual attention. Curve fitting transforms discrete pupil dilation data into continuous dilation curves, better reflecting the trend of pupil size changes. These curves provide a quantitative description of dynamic pupil changes, used to analyze features such as the rate and magnitude of pupil change. Edge contour evolution analysis captures morphological changes in the pupil edge, including the shape, curvature, and sharpness of the edge. Pupil edge contour change data provides a quantitative description of pupil contour changes, including non-circular structures. The calculation measures the irregularity of the pupil outline, reflecting the degree of variation in pupil shape. The non-circularity parameter of the pupil outline provides a quantitative description of the pupil shape, which is used for the analysis of pupil shape changes and the study of individual differences. The dynamic change feature analysis of the pupil links the non-circularity parameter of the pupil outline with the change characteristics of the scaling curve, revealing the dynamic law of pupil shape and scaling. The dynamic scaling law of the pupil provides a quantitative description of the changes in pupil shape and scaling. The pupil shape change analysis further analyzes the changes in pupil shape, such as changes in shape and outline, based on the dynamic scaling law of the pupil. The pupil shape feature data provides a quantitative description of the user's pupil shape.

[0034] Preferably, the specific steps of step S25 are as follows:

[0035] Step S251: Expose the user to multi-frequency light to obtain pupil light response data;

[0036] Step S252: Perform multi-frequency light response rate analysis on the user's pupillary light response data to generate multi-frequency light intensity response characteristic curves;

[0037] Step S253: Detect pupil dilation saturation time based on the multi-frequency light intensity response characteristic curve to generate dilation saturation time data;

[0038] Step S254: Perform multi-frequency optical inertial strain analysis on the pupil morphology feature data to generate multi-frequency pupil morphological qualitative change data;

[0039] Step S255: Analyze the pupillary light loss recovery characteristics of the multi-frequency pupillary morphological qualitative change data using the scaling saturation time data to generate pupillary recovery data;

[0040] Step S256: Perform photosensitive analysis on the pupil restorative data to generate pupil photosensitivity data;

[0041] Step S257: Perform eye physiological health analysis on pupil morphological feature data based on pupil photosensitivity data to generate user eye physiological signal data.

[0042] This invention stimulates the user's pupillary light response to multi-frequency light illumination, acquiring pupillary response data to light of different frequencies. This pupillary light response data provides a quantitative description of the functional state of the user's visual system. Multi-frequency light response rate analysis analyzes the pupillary response rate to different frequencies of light, revealing the frequency characteristics of the pupillary light response. Multi-frequency light intensity response characteristic curves provide a quantitative description of the frequency characteristics of the pupillary light response, helping to analyze the user's visual sensitivity and the health of the visual system. Pupil dilation saturation time detection, through multi-frequency light intensity response characteristic curves, analyzes the time required for the pupil to reach saturation under varying light intensity. The dilation saturation time data provides the temporal characteristics of the pupillary light response, assessing the pupil's ability to adjust to different light intensities and the sensitivity of the visual system. Multi-frequency light inertia strain analysis analyzes the morphological changes of the pupil under different frequencies of light stimulation, reflecting the pupil's response to light. Adaptability analysis includes: multi-frequency pupil morphological change data providing a quantitative description of pupil morphological changes, assessing the user's visual adaptability and eye health; pupil apoptosis recovery characteristic analysis using scaling saturation time data to analyze the time and rate at which the pupil recovers from a saturated state to a baseline state; pupil restorativity data providing a quantitative description of the pupil illumination recovery process, assessing the user's pupil recovery ability and visual system health; light difference sensitivity analysis analyzing the pupil's sensitivity to changes in light intensity, reflecting the pupil's ability to adjust for light differences; pupil photosensitivity data providing a quantitative description of the pupil's sensitivity to changes in light intensity, assessing the user's visual adaptability and eye health; eye physiological health analysis using pupil photosensitivity data to comprehensively assess the user's pupil morphological characteristics and photosensitivity to infer the eye's physiological condition; and user eye physiological signal data providing a quantitative description of eye health and visual system functional status.

[0043] Preferably, the specific steps of step S3 are as follows:

[0044] Step S31: Based on user emotion data, perform skin texture recognition on the detail-enhanced facial image to generate skin texture data;

[0045] Step S32: Perform skin texture analysis on the skin texture data to generate facial skin texture data;

[0046] Step S33: Perform facial pigment distribution analysis on the enhanced facial image to generate pigment uniformity data;

[0047] Step S34: Perform skin feature analysis on facial skin texture data and pigmentation evenness data to generate fine facial skin feature data;

[0048] Step S35: Construct a user facial profile based on the user's eye physiological signal data and facial fine skin feature data to build a digital facial model.

[0049] This invention provides clearer and more accurate facial images through detail enhancement, which aids in subsequent skin texture recognition. Skin texture data is obtained by analyzing texture features in the facial image to acquire the user's skin texture information for skin analysis and health assessment. Skin texture analysis, through feature extraction and analysis of skin texture data, assesses the texture characteristics of the user's facial skin, such as smoothness and roughness. Facial skin texture data provides a quantitative description of the user's skin texture characteristics, determining skin health and the degree of skin aging. Facial pigment distribution analysis extracts pigment distribution features from the detail-enhanced facial image, assessing the uniformity and distribution of pigment on the face. The system provides a quantitative description of the evenness of facial pigmentation distribution, identifying skin problems such as pigmentation and age spots. Facial skin feature analysis comprehensively considers multiple characteristics, including skin texture and pigmentation evenness, to provide a detailed assessment of the user's facial skin. Facial fine skin feature data provides a comprehensive description of facial skin condition, including skin texture, pigmentation distribution, and skin tone evenness, assessing skin health and beauty needs. The user facial profile construction combines eye physiological signal data with facial fine skin feature data to generate a personalized facial feature description for the user. The digital facial model provides a visual representation of the user's facial features for applications such as health status analysis, beauty consultation, and virtual makeup.

[0050] Preferably, the specific steps of step S4 are as follows:

[0051] Step S41: Perform dynamic feature recognition on the full-body motion image to generate user dynamic feature data;

[0052] Step S42: Perform fine-grained muscle tissue image segmentation on the whole-body motion image based on the user's dynamic feature data to generate a muscle tissue image;

[0053] Step S43: Quantize muscle mass in the muscle tissue image to generate muscle mass data;

[0054] Step S44: Perform muscle group distribution analysis on the muscle mass data to generate muscle group distribution data;

[0055] Step S45: Perform muscle group morphology and structure analysis on muscle tissue images using muscle group distribution data to generate muscle group morphology and structure data.

[0056] Step S46: Analyze the muscle group morphology and structure data to generate muscle health status data.

[0057] This invention extracts user motion features, such as amplitude, speed, and frequency, from full-body motion images through dynamic feature recognition. The user's dynamic feature data provides a quantitative description of their movement behavior, assessing their athletic ability and posture accuracy. Fine-grained muscle tissue image segmentation, guided by the user's dynamic feature data, separates muscle tissue from other tissues in the full-body motion images. The muscle tissue images provide a visual representation of the user's muscle tissue, laying the foundation for subsequent muscle mass and muscle group analysis. Muscle mass quantification assesses the user's muscle mass by analyzing indicators such as muscle density and area in the muscle tissue images. The muscle mass data provides a quantitative description of the user's muscle mass, determining the degree of muscle development and muscle health. Muscle group distribution analysis further analyzes muscle mass... The statistical analysis of quantitative data assesses the distribution of different muscle groups in the user's body. Muscle group distribution data provides the relative proportions and distribution of different muscle groups, allowing understanding of the user's muscle development and muscle balance. Muscle group morphology and structure analysis, combined with muscle group distribution data, extracts morphological and structural features of muscle groups from muscle tissue images, such as shape and connection methods. Muscle group morphology and structure data provides a quantitative description of muscle group morphology, assessing coordination between muscle groups and changes in muscle structure. Muscle health status analysis comprehensively considers multiple characteristics such as muscle mass, muscle group distribution, and morphology and structure to provide a detailed assessment of the user's muscle health status. Muscle health status data provides a quantitative description of muscle status and health problems, and can be used to assess muscle development level, muscle imbalance, sports injuries, etc.

[0058] Preferably, the specific steps of step S5 are as follows:

[0059] Step S51: Use computer vision technology to perform joint point cloud recognition on the whole-body motion image to generate joint point cloud data;

[0060] Step S52: Perform user 3D skeleton reconstruction on the joint point cloud data to generate a user 3D skeleton model;

[0061] Step S53: Perform skeletal structure symmetry evaluation on the user's 3D skeletal model to generate skeletal symmetry data;

[0062] Step S54: Perform skeletal structural feature analysis on the skeletal symmetry data to generate skeletal structural feature data;

[0063] Step S55: Analyze the abnormal parts of the skeletal structure feature data to generate abnormal skeletal structure data.

[0064] This invention extracts the 3D coordinate information of key joints from full-body motion images through joint point cloud recognition, forming joint point cloud data. This data provides an accurate description of the user's posture and joint movements, used to analyze the user's posture, movement smoothness, etc. Through 3D skeleton reconstruction, the user's 3D skeletal structure is restored based on the joint point cloud data. The generated 3D skeletal model provides the foundation for subsequent skeletal structure analysis, assessing the user's skeletal posture and proportions. Skeletal structure symmetry assessment analyzes whether there are symmetry differences in the user's skeletal structure, such as the length and angle of the left and right limbs. The generated skeletal symmetry data provides a quantitative description of the symmetry of the user's skeletal structure, assessing the existence of skeletal asymmetry. Skeletal structure feature analysis further extracts features of the user's skeletal structure based on the skeletal symmetry data, such as bone length and angles. The generated skeletal structure feature data provides a more detailed quantitative description of the user's skeletal structure, used to assess the morphological characteristics and changes of the skeletal structure. Abnormal site analysis detects and identifies abnormal sites in the user's skeletal structure by comparing the skeletal structure feature data with normal reference ranges. The abnormal skeletal structure data provides the location and description of abnormal sites in the user's skeletal structure, helping to discover and diagnose abnormal skeletal conditions.

[0065] Preferably, step S6 consists of the following steps:

[0066] Step S61: Use deep learning algorithms to perform real-time fusion analysis on muscle health data and abnormal skeletal structure data to generate dynamic musculoskeletal data for users.

[0067] Step S62: Perform implicit association analysis on the user's dynamic musculoskeletal data to generate dynamic musculoskeletal association data;

[0068] Step S63: Perform potential risk analysis on the user's dynamic musculoskeletal data based on the dynamic musculoskeletal correlation data to obtain potential musculoskeletal risk data;

[0069] Step S64: Perform risk trend prediction on the musculoskeletal potential risk data to generate musculoskeletal risk trend data;

[0070] Step S65: Use the recurrent convolution algorithm to perform holographic visual modeling on musculoskeletal risk trend data and digital facial model to construct a dynamic holographic visual model for performing digital integration of health status.

[0071] This invention utilizes deep learning algorithms to fuse and analyze muscle health data and skeletal abnormality data, thereby obtaining dynamic musculoskeletal data for the user. This dynamic musculoskeletal data comprehensively considers both muscle health and skeletal abnormalities, providing a more complete picture of the user's musculoskeletal status. Implicit correlation analysis explores the potential relationships between the user's dynamic musculoskeletal data, revealing the interaction between muscles and bones. The generated dynamic musculoskeletal correlation data provides a description of the overall operation of the user's musculoskeletal system, helping to understand muscle and bone coordination and balance. Potential risk analysis assesses potential risks and problems in the user's musculoskeletal system based on the dynamic musculoskeletal correlation data. The generated dynamic musculoskeletal potential risk data provides a risk assessment of the user's musculoskeletal health, helping to identify and prevent potential problems early. Risk trend prediction analyzes the development trend of the user's musculoskeletal health based on the potential risk data and predicts future risks. The generated musculoskeletal risk trend data provides long-term predictions and trend analysis of the user's musculoskeletal health, helping to develop personalized prevention and rehabilitation plans. A recurrent convolution algorithm performs holographic visual modeling on the musculoskeletal risk trend data and digital facial model, achieving digital integration of the user's health status. The constructed dynamic holographic visual model provides a visual representation of the user's musculoskeletal health status.

[0072] This specification provides a digital health status integration system for smart mirrors, comprising:

[0073] The facial structure analysis module acquires user facial images and full-body motion images using the embedded camera in the smart mirror; it enhances the details of the user's facial images to generate enhanced facial images; it performs facial structure analysis on the enhanced facial images to generate facial structure data; and it performs micro-expression recognition on the enhanced facial images based on the facial structure data to generate user emotion data.

[0074] The pupil morphology feature module performs user visual feature analysis on the enhanced facial image to generate user visual feature data; based on the user visual feature data, it performs pupil morphology change analysis on the enhanced facial image to generate pupil morphology feature data; and it performs optical difference sensitivity analysis on the pupil morphology feature data to generate user eye physiological signal data.

[0075] The skin feature module performs subtle facial skin analysis on enhanced facial images based on user emotion data to generate subtle facial skin feature data; it also constructs a user facial profile based on user eye physiological signal data and subtle facial skin feature data to build a digital facial model.

[0076] The muscle morphology and structure module performs dynamic feature recognition on full-body motion images to generate user dynamic feature data; analyzes the muscle morphology and structure of the full-body motion images based on the user dynamic feature data to generate muscle morphology and structure data; and analyzes the muscle health status of the muscle morphology and structure data to generate muscle health status data.

[0077] The 3D skeleton model module reconstructs the user's 3D skeleton from the full-body motion image to generate a user 3D skeleton model; it analyzes the skeletal structure features of the user 3D skeleton model to generate skeletal structure feature data; and it analyzes abnormal parts of the skeletal structure feature data to generate abnormal skeletal structure data.

[0078] The holographic visual model module uses deep learning algorithms to perform real-time fusion analysis of muscle health status data and skeletal abnormality structure data to generate dynamic musculoskeletal data for users; it performs potential risk trend analysis on the dynamic musculoskeletal data to generate musculoskeletal risk trend data; and it uses a recurrent convolution algorithm to perform holographic visual modeling on the musculoskeletal risk trend data and digital facial model to construct a dynamic holographic visual model to perform digital integration of health status.

[0079] This invention utilizes a facial structure analysis module to enhance facial images and recognize micro-expressions, generating user facial structure and emotion data to help assess facial muscle activity, expressions, and emotional states. A pupil morphology feature module analyzes enhanced facial images and pupil morphology changes to generate user visual feature data and pupil morphology feature data, used to assess visual attention, pupillary response, and eye health. A skin feature module analyzes subtle facial skin details to generate detailed facial skin feature data, revealing the user's skin condition, such as texture, pigmentation, and pattern, to assess skin health and overall health. A muscle group morphology and structure module identifies muscle groups through full-body motion images and dynamic features. The system generates user dynamic feature data and muscle group morphology data to assess the user's posture, movements, and muscle usage, helping to understand muscle health and athletic performance. The 3D skeletal model module generates skeletal structure feature data and abnormal skeletal structure data through user 3D skeletal reconstruction and skeletal structure feature analysis, assessing the user's skeletal health, the impact of posture on the skeleton, and existing skeletal abnormalities. The holographic vision model module uses deep learning algorithms to fuse and analyze muscle health data and abnormal skeletal structure data, generating user dynamic musculoskeletal data and musculoskeletal risk trend data. Comprehensive analysis provides a more comprehensive musculoskeletal health assessment and risk prediction, helping to detect potential health problems early. Attached Figure Description

[0080] Figure 1 This is a flowchart illustrating the steps of a method for digitally integrating health status into a smart mirror according to the present invention.

[0081] Figure 2 This is a flowchart illustrating the detailed implementation steps of step S1.

[0082] Figure 3 This is a flowchart illustrating the detailed implementation steps of step S2.

[0083] Figure 4 This is a flowchart illustrating the detailed implementation steps of step S3. Detailed Implementation

[0084] It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of the invention.

[0085] This application provides a method and system for digitally integrating health status data for smart mirrors. The executing entities of the method and system for digitally integrating health status data for smart mirrors include, but are not limited to, mechanical equipment, data processing platforms, cloud server nodes, network upload devices, etc., which can be considered general computing nodes in this application. The data processing platform includes, but is not limited to, at least one of an audio-visual management system, an information management system, and a cloud data management system.

[0086] Please see Figures 1 to 4 This invention provides a method for digitally integrating health status data for use in smart mirrors, the method comprising the following steps:

[0087] Step S1: Acquire user facial images and full-body motion images using the embedded camera in the smart mirror; enhance the details of the user's facial images to generate enhanced facial images; perform facial structure analysis on the enhanced facial images to generate facial structure data; perform micro-expression recognition on the enhanced facial images based on the facial structure data to generate user emotion data.

[0088] Step S2: Perform user visual feature analysis on the enhanced facial image to generate user visual feature data; perform pupil morphology change analysis on the enhanced facial image based on the user visual feature data to generate pupil morphology feature data; perform optical difference sensitivity analysis on the pupil morphology feature data to generate user eye physiological signal data.

[0089] Step S3: Perform facial fine skin analysis on the enhanced facial image based on user emotion data to generate facial fine skin feature data; construct a user facial profile based on user eye physiological signal data and facial fine skin feature data to build a digital facial model;

[0090] Step S4: Perform dynamic feature recognition on the full-body motion image to generate user dynamic feature data; perform muscle group morphology and structure analysis on the full-body motion image based on the user dynamic feature data to generate muscle group morphology and structure data; perform muscle health status analysis on the muscle group morphology and structure data to generate muscle health status data.

[0091] Step S5: Perform 3D skeleton reconstruction on the full-body motion image to generate a 3D skeleton model of the user; perform skeletal structure feature analysis on the 3D skeleton model of the user to generate skeletal structure feature data; perform abnormal site analysis on the skeletal structure feature data to generate abnormal skeletal structure data.

[0092] Step S6: Use deep learning algorithms to perform real-time fusion analysis on muscle health status data and skeletal abnormality structure data to generate dynamic musculoskeletal data for users; perform potential risk trend analysis on the dynamic musculoskeletal data to generate musculoskeletal risk trend data; use recurrent convolution algorithms to perform holographic visual modeling on the musculoskeletal risk trend data and digital facial model to construct a dynamic holographic visual model to perform digital integration of health status.

[0093] This invention provides a comprehensive observation of a user's appearance and dynamic posture by acquiring facial and full-body motion images. Detail enhancement improves the clarity and visibility of facial images. Facial structure analysis extracts facial feature information, such as facial contours and eye positions. Micro-expression recognition captures the user's emotional state through the analysis of subtle facial movements. User visual feature data provides detailed information about the user's facial appearance, such as skin condition and facial features. Pupil morphology change analysis reveals the user's attention level and emotional state. Pupil morphology feature data and ocular physiological signal data provide quantitative information about the user's visual perception and attention level. Facial fine skin analysis provides information about the user's skin health and texture. The construction of the user's facial profile is based on emotional data, ocular physiological signal data, and facial fine skin feature data, comprehensively describing the user's facial features and emotional state. The digital facial model serves as a visual representation of the user's facial features, providing a foundation for subsequent health status analysis. Dynamic features... This system identifies and analyzes user postures and movement patterns, providing information about user activity status and posture. Muscle group morphology analysis reveals the distribution, shape, and relationships of user muscles, providing information on muscle structure. Muscle health data assesses the health of user muscles, including indicators such as muscle strength and flexibility. User 3D skeleton reconstruction provides a 3D representation of user skeletal posture and structure. Skeletal structure feature analysis extracts key skeletal features, such as bone length and angles. Skeletal abnormality structure data identifies abnormal parts of the skeletal structure, such as fractures and deformities, providing information on skeletal health. Real-time fusion analysis combines muscle health data and skeletal abnormality structure data to comprehensively assess the user's musculoskeletal health, providing comprehensive musculoskeletal analysis results. Potential risk trend analysis predicts the trend of user's musculoskeletal health, helping to detect health problems early and take corresponding measures. Dynamic holographic visual models combine musculoskeletal risk trend data with digital facial models to provide intuitive visualization, facilitating doctors and users to understand and analyze health conditions.

[0094] In this embodiment of the invention, reference is made to Figure 1 The diagram below illustrates the steps of a method for digitally integrating health status data in a smart mirror according to the present invention. In this embodiment, the steps of the method for digitally integrating health status data in a smart mirror include:

[0095] Step S1: Acquire user facial images and full-body motion images using the embedded camera in the smart mirror; enhance the details of the user's facial images to generate enhanced facial images; perform facial structure analysis on the enhanced facial images to generate facial structure data; perform micro-expression recognition on the enhanced facial images based on the facial structure data to generate user emotion data.

[0096] In this embodiment, the smart mirror is embedded with a camera that acquires facial images and full-body motion images of the user. The camera captures the user's facial expressions and body movements. Image processing algorithms, such as contrast enhancement and edge sharpening, are used to enhance the details of the acquired facial images to improve their clarity and detail visibility. Computer vision technology and face recognition algorithms are used to detect facial feature points and contour lines. Facial structure analysis is performed on the enhanced facial images to understand the shape and position information of the facial structures. Based on the facial structure data, machine learning and deep learning technologies are used to train a model to detect subtle changes in facial expressions. Micro-expression recognition is performed on the enhanced facial images to infer the user's emotional state.

[0097] Step S2: Perform user visual feature analysis on the enhanced facial image to generate user visual feature data; perform pupil morphology change analysis on the enhanced facial image based on the user visual feature data to generate pupil morphology feature data; perform optical difference sensitivity analysis on the pupil morphology feature data to generate user eye physiological signal data.

[0098] In this embodiment, computer vision technology and image processing algorithms are used to perform user visual feature analysis on the enhanced facial image to detect and extract facial features such as eyes, eyebrows, and lips to generate user visual feature data. Based on the user visual feature data, image processing and computer vision technology are used to perform pupil morphology change analysis on the enhanced facial image to detect and track pupil morphology changes in different image frames to generate pupil morphology feature data. The pupil morphology feature data is then subjected to light difference sensitivity analysis to generate user eye physiological signal data. By analyzing the pupil's response to changes in light, such as the speed and range of pupil contraction and dilation, the user's eye physiological state, such as visual attention and fatigue level, can be inferred.

[0099] Step S3: Perform facial fine skin analysis on the enhanced facial image based on user emotion data to generate facial fine skin feature data; construct a user facial profile based on user eye physiological signal data and facial fine skin feature data to build a digital facial model;

[0100] In this embodiment, based on user emotion data, image processing and computer vision technologies are used to perform facial fine skin analysis on the enhanced facial images, detecting and analyzing subtle changes in facial skin, such as skin texture, pigmentation, wrinkles, etc., to generate facial fine skin feature data. Based on the user's eye physiological signal data and facial fine skin feature data, the eye physiological signal data and facial skin feature data are combined to construct a user facial profile, thereby building a digital facial model. The digital facial model is a comprehensive representation that includes the user's emotional state, eye physiological signals, and facial skin features, used to describe the user's facial features and state.

[0101] Step S4: Perform dynamic feature recognition on the full-body motion image to generate user dynamic feature data; perform muscle group morphology and structure analysis on the full-body motion image based on the user dynamic feature data to generate muscle group morphology and structure data; perform muscle health status analysis on the muscle group morphology and structure data to generate muscle health status data.

[0102] In this embodiment, computer vision and motion analysis technologies are used to perform dynamic feature recognition on full-body motion images, detect and analyze the user's actions, postures, and movement patterns to generate the user's dynamic feature data, such as gait, postural stability, and range of motion. Based on the user's dynamic feature data, muscle group morphology analysis is performed on the full-body motion images to detect and track the morphology of muscle groups, such as their location, size, and symmetry, to generate muscle group morphology data. Pattern recognition and machine learning technologies are used to analyze the muscle health status of the muscle group morphology data, comparing and analyzing it with health status indicators to infer the user's muscle health status, such as muscle balance and muscle strength.

[0103] Step S5: Perform 3D skeleton reconstruction on the full-body motion image to generate a 3D skeleton model of the user; perform skeletal structure feature analysis on the 3D skeleton model of the user to generate skeletal structure feature data; perform abnormal site analysis on the skeletal structure feature data to generate abnormal skeletal structure data.

[0104] In this embodiment, a three-dimensional skeleton reconstruction of the user is performed on the whole-body motion image. Deep learning and computer vision technologies are used, combined with multi-view images or depth sensor data, to reconstruct the user's skeleton in three dimensions to generate a three-dimensional skeleton model of the user. Skeletal structural feature analysis is performed on the user's three-dimensional skeleton model. Computer graphics and geometric analysis techniques are used to detect and analyze the structural features of the skeleton, such as bone length, joint angles, and bone proportions, to generate skeleton structural feature data. Abnormal parts are analyzed on the skeleton structural feature data. By comparing the skeleton structural feature data with the normal reference range, abnormal parts of the skeleton structure, such as bone deformities and asymmetries, are detected and identified to generate abnormal skeleton structural data.

[0105] Step S6: Use deep learning algorithms to perform real-time fusion analysis on muscle health status data and skeletal abnormality structure data to generate dynamic musculoskeletal data for users; perform potential risk trend analysis on the dynamic musculoskeletal data to generate musculoskeletal risk trend data; use recurrent convolution algorithms to perform holographic visual modeling on the musculoskeletal risk trend data and digital facial model to construct a dynamic holographic visual model to perform digital integration of health status.

[0106] In this embodiment, deep learning algorithms are used to extract and fuse features from two data sources. Real-time fusion analysis is performed on muscle health data and skeletal abnormality data to obtain a comprehensive musculoskeletal feature representation. Convolutional Neural Networks (CNNs) or other deep learning models are used for processing. Based on the fusion analysis results, dynamic musculoskeletal data of the user is generated, including information on muscle health and skeletal abnormalities, to describe the user's musculoskeletal state. Potential risk trend analysis is performed on the user's dynamic musculoskeletal data. Machine learning and time series analysis techniques are used to detect and analyze musculoskeletal risk trends, such as muscle degeneration and bone deformation, to generate musculoskeletal risk trend data. A recurrent convolution algorithm is used to extract and fuse features from the musculoskeletal risk trend data and the digital facial model. Holographic visual modeling is then performed on the musculoskeletal risk trend data and the digital facial model to construct a dynamic holographic visual model for comprehensively displaying the user's health status.

[0107] In this embodiment, reference Figure 2 The diagram below illustrates the detailed implementation steps of step S1. In this embodiment, the detailed implementation steps of step S1 include:

[0108] Step S11: Acquire the user's facial image and full-body motion image using the embedded camera in the smart mirror;

[0109] Step S12: Enhance the details of the user's facial image to generate a detail-enhanced facial image;

[0110] Step S13: Perform facial feature node recognition on the detail-enhanced facial image to generate facial feature node position data;

[0111] Step S14: Perform facial structure analysis on the detail-enhanced facial image based on the facial feature node position data to generate facial structure data;

[0112] Step S15: Perform micro-expression recognition on the enhanced facial image based on facial structure data to generate user micro-expression data;

[0113] Step S16: Perform emotional fluctuation analysis on the user's micro-expression data to generate user emotional data.

[0114] In this embodiment, the smart mirror has an embedded camera that acquires the user's facial image and full-body motion image. Image processing techniques, such as sharpening and contrast enhancement, are used to enhance the details of the acquired facial image, highlighting facial details and improving image clarity and visibility. Computer vision and machine learning techniques are used to identify facial features in the enhanced facial image, such as facial landmark detection algorithms, to detect and identify the positions of facial features like the eyes, nose, and mouth. Based on the facial feature location data, facial structure analysis, such as feature extraction, is performed on the enhanced facial image. It analyzes facial structural features, such as face shape and proportions, using methods like shape analysis to generate corresponding facial structure data. Based on this data, machine learning and computer vision techniques are used to enhance the details of facial images and perform micro-expression recognition, such as facial expression recognition algorithms, to analyze and identify minute and instantaneous changes in facial expressions and generate corresponding micro-expression data. Furthermore, it analyzes user micro-expression data for emotional fluctuations, using sentiment analysis and time series analysis techniques, such as machine learning models or signal processing algorithms, to detect and analyze emotional changes in micro-expression data, such as happiness, sadness, and anger, and generate corresponding emotional data.

[0115] In this embodiment, reference Figure 3 The diagram below illustrates the detailed implementation steps of step S2. In this embodiment, the detailed implementation steps of step S2 include:

[0116] Step S21: Perform user visual feature analysis on the detail-enhanced facial image to generate user visual feature data;

[0117] Step S22: Perform eye trajectory optical flow tracking on the detail-enhanced facial image based on the user's visual feature data to generate user eye trajectory data;

[0118] Step S23: Perform gaze focus identification on the user's eye trajectory data to generate the eye gaze focus;

[0119] Step S24: Analyze pupil morphology changes in the detail-enhanced facial image based on eye gaze focus to generate pupil morphology feature data;

[0120] Step S25: Perform optical difference sensitivity analysis on the pupil morphology feature data to generate user eye physiological signal data.

[0121] In this embodiment, computer vision and image processing technologies, such as feature extraction and facial recognition algorithms, are used to perform user visual feature analysis on the enhanced facial image. Visual features, such as facial expressions and skin condition, are extracted from the facial image, and corresponding user visual feature data is generated. Based on this user visual feature data, an optical flow algorithm is used to perform eye trajectory optical flow tracking on the enhanced facial image to track the movement trajectory of the eyes in the facial image and generate corresponding eye trajectory data. Fixation focus identification is then performed on the user's eye trajectory data, and machine learning and computer vision technologies, such as fixation point detection algorithms, are used to analyze the eye trajectory. The system uses data to determine the user's gaze focus position and generates corresponding eye gaze focus data. Based on the eye gaze focus, it performs pupil morphology change analysis on the enhanced facial image. Using image processing and computer vision techniques, such as pupil detection and morphology analysis algorithms, it analyzes the morphological changes of the pupil under different gaze focuses, generating corresponding pupil morphology feature data. It then performs light difference sensitivity analysis on the pupil morphology feature data and uses optical and image processing techniques, such as grayscale change detection algorithms, to analyze the brightness changes in the pupil morphology feature data in order to obtain the user's ocular physiological signal data, such as the pupil's response to light and the degree of pupil constriction.

[0122] In this embodiment, the specific steps of step S24 are as follows:

[0123] Step S241: Perform pupil scaling analysis on the detail-enhanced facial image based on eye gaze focus to generate pupil scaling data;

[0124] Step S242: Perform curve fitting on the pupil dilation data to generate a pupil dilation curve;

[0125] Step S243: Perform edge contour evolution analysis on the detail-enhanced facial image based on the pupil scaling curve to generate pupil edge contour change data;

[0126] Step S244: Perform non-circularity structure calculation on the pupil edge contour change data to generate pupil contour non-circularity parameters;

[0127] Step S245: Analyze the dynamic change characteristics of the pupil by using the non-circularity parameter of the pupil contour to generate the dynamic pupil scaling pattern;

[0128] Step S246: Analyze the pupil morphology changes in the detail-enhanced facial image based on the dynamic scaling rules of the pupil to generate pupil morphology feature data.

[0129] In this embodiment, pupil scaling analysis is performed on the detail-enhanced facial image based on the eye's gaze focus. The pupil position is determined according to the gaze focus. The pupil size is measured, and the scaling ratio is calculated by comparing the pupil's diameter or area with a reference size, generating corresponding pupil scaling data. Curve fitting is performed on the pupil scaling data by fitting it to a suitable mathematical model to obtain a pupil scaling curve. Common fitting methods include polynomial fitting and Gaussian fitting. The fitting process aims to find the best-fit curve to reflect the overall trend of the pupil scaling data. Based on the pupil scaling curve, edge contour evolution analysis is performed on the detail-enhanced facial image. This is achieved by calculating the contour changes of the pupil edge under different scaling levels. For each scaling level, image processing and edge detection techniques are used to extract the pupil's edge contour and calculate the degree of contour change to generate the corresponding pupil. The edge contour change data of the pupil is used to perform non-circularity structure calculations. The non-circularity of the pupil is measured by calculating the shape deviation of the contour. The eccentricity or roundness index of the contour is calculated to evaluate the approximate roundness of the pupil. Through these calculations, pupil contour non-circularity parameters are generated. Based on the pupil contour non-circularity parameters, the pupil zoom curve is analyzed to perform dynamic change feature analysis. This is achieved by comparing the relationship between the pupil zoom curve and the non-circularity parameters. By analyzing the shape of the curve and the changing trend of the non-circularity parameters, the dynamic scaling law of the pupil at different scaling levels is revealed, generating pupil dynamic scaling law. Based on the pupil dynamic scaling law, the pupil morphology change analysis is performed on the detail-enhanced facial image. By adjusting the magnification or reduction degree of the detail-enhanced facial image according to the changes in the pupil zoom curve, the morphological changes of the pupil at different scaling levels are simulated, generating corresponding pupil morphological feature data.

[0130] In this embodiment, the specific steps of step S25 are as follows:

[0131] Step S251: Expose the user to multi-frequency light to obtain pupil light response data;

[0132] Step S252: Perform multi-frequency light response rate analysis on the user's pupillary light response data to generate multi-frequency light intensity response characteristic curves;

[0133] Step S253: Detect pupil dilation saturation time based on the multi-frequency light intensity response characteristic curve to generate dilation saturation time data;

[0134] Step S254: Perform multi-frequency optical inertial strain analysis on the pupil morphology feature data to generate multi-frequency pupil morphological qualitative change data;

[0135] Step S255: Analyze the pupillary light loss recovery characteristics of the multi-frequency pupillary morphological qualitative change data using the scaling saturation time data to generate pupillary recovery data;

[0136] Step S256: Perform photosensitive analysis on the pupil restorative data to generate pupil photosensitivity data;

[0137] Step S257: Perform eye physiological health analysis on pupil morphological feature data based on pupil photosensitivity data to generate user eye physiological signal data.

[0138] In this embodiment, the user is irradiated with multi-frequency light using light sources of different frequencies and intensities. The light of different frequencies gradually increases in intensity, stimulating the pupil's light response. By calculating the pupil dilation rate under different light frequencies, multi-frequency light response rate analysis is performed on the user's pupillary light response data. The pupillary response rate under different light frequencies is determined by measuring changes in pupil diameter or area, generating a multi-frequency light intensity response characteristic curve. Based on this curve, pupil dilation saturation time is detected by analyzing the trend of the light intensity response curve. The pupil dilation saturation time is determined by detecting the saturation point or dilation rate trend of the light intensity response curve, generating dilation saturation time data. Multi-frequency optical inertia strain analysis is then performed on the pupillary morphological data, comparing the pupillary morphological characteristics under different light frequencies. This system achieves its goal by analyzing indicators such as pupil dilation and contraction amplitude and morphological change rate to determine the degree of morphological changes in the pupil under multi-frequency light exposure, generating multi-frequency morphological change data of the pupil. It then analyzes the pupil's light loss recovery characteristics by comparing the pupil's recovery process after light stimulation ends, determining the pupil's light loss recovery characteristics by analyzing indicators such as pupil morphological recovery rate and recovery amplitude. Finally, it performs light difference sensitivity analysis on the pupil's recovery data by comparing the pupil's recovery rate and recovery amplitude under different light intensity differences. By measuring the pupil's recovery characteristics under different light intensity differences, it assesses the pupil's sensitivity to light intensity differences. Based on the pupil photosensitivity data, it performs ocular physiological health analysis on the pupil morphological characteristic data by comparing the relationship between the pupil morphological characteristic data and known ocular physiological health indicators. Finally, it assesses the user's ocular physiological health status by analyzing indicators such as pupil size, reaction speed and amplitude to light exposure.

[0139] In this embodiment, reference Figure 4 The diagram below illustrates the detailed implementation steps of step S3. In this embodiment, the detailed implementation steps of step S3 include:

[0140] Step S31: Based on user emotion data, perform skin texture recognition on the detail-enhanced facial image to generate skin texture data;

[0141] Step S32: Perform skin texture analysis on the skin texture data to generate facial skin texture data;

[0142] Step S33: Perform facial pigment distribution analysis on the enhanced facial image to generate pigment uniformity data;

[0143] Step S34: Perform skin feature analysis on facial skin texture data and pigmentation evenness data to generate fine facial skin feature data;

[0144] Step S35: Construct a user facial profile based on the user's eye physiological signal data and facial fine skin feature data to build a digital facial model.

[0145] In this embodiment, based on user emotion data, skin texture recognition is performed on the enhanced facial image using image analysis and machine learning algorithms to identify skin texture features in the facial image. Based on the skin texture recognition results, skin texture data of the facial image is generated, including information such as skin texture type, density, and granularity. Skin texture analysis is then performed on the skin texture data using image processing and texture analysis algorithms to evaluate skin texture characteristics. Based on the skin texture analysis results, facial skin texture data is generated, including information such as skin smoothness, roughness, and fineness. Image processing and color analysis techniques are then used to further refine the enhanced facial image. Facial pigment distribution analysis is performed to assess the distribution of pigments in facial images. Based on the results of the facial pigment distribution analysis, pigment uniformity data is generated, including information such as the uniformity of facial pigmentation and the distribution of spots. Skin feature analysis is performed on facial skin texture data and pigment uniformity data. By comprehensively considering factors such as skin texture and pigment uniformity, the subtle skin features of the face are evaluated. Based on the user's eye physiological signal data and subtle facial skin feature data, a user facial profile is constructed. The user's digital facial model is generated by comprehensively analyzing the eye physiological signal and skin feature data. The digital facial model includes information such as the user's facial features, emotional state, skin texture, and pigment distribution.

[0146] In this embodiment, step S4 includes the following steps:

[0147] Step S41: Perform dynamic feature recognition on the full-body motion image to generate user dynamic feature data;

[0148] Step S42: Perform fine-grained muscle tissue image segmentation on the whole-body motion image based on the user's dynamic feature data to generate a muscle tissue image;

[0149] Step S43: Quantize muscle mass in the muscle tissue image to generate muscle mass data;

[0150] Step S44: Perform muscle group distribution analysis on the muscle mass data to generate muscle group distribution data;

[0151] Step S45: Perform muscle group morphology and structure analysis on muscle tissue images using muscle group distribution data to generate muscle group morphology and structure data.

[0152] Step S46: Analyze the muscle group morphology and structure data to generate muscle health status data.

[0153] In this embodiment, computer vision and motion analysis algorithms are used to perform dynamic feature recognition on full-body motion images. Joint positions, skeletal postures, and motion trajectories are analyzed and extracted. Dynamic feature data of the user, including the speed, amplitude, and rhythm of movements, is extracted from the full-body motion images. Fine-grained image segmentation of muscle tissue is performed on the full-body motion images. Through image processing and segmentation algorithms, muscle tissue in the image is separated from other parts. The segmented muscle tissue image contains the user's muscle tissue information for subsequent analysis and quantification. Image processing and computer vision techniques are used to perform quantitative analysis of muscle mass in the muscle tissue image, including measuring and calculating muscle size, shape, and density. The analysis results generate muscle mass data, including muscle mass values, proportions, and distribution. Based on this data, muscle group distribution analysis is performed to statistically assess the muscle mass and distribution of different muscle groups. Then, based on this distribution data, muscle tissue images are analyzed for morphological structure, including the shape, connectivity, and hierarchical structure of muscle groups. This generates muscle morphological structure data, including characteristic descriptions and morphological relationships. Finally, based on this data, muscle health status analysis is conducted, assessing the normality, symmetry, and balance of muscle morphology. This generates muscle health status data, including assessment results and abnormal indicators.

[0154] In this embodiment, the specific steps of step S5 are as follows:

[0155] Step S51: Use computer vision technology to perform joint point cloud recognition on the whole-body motion image to generate joint point cloud data;

[0156] Step S52: Perform user 3D skeleton reconstruction on the joint point cloud data to generate a user 3D skeleton model;

[0157] Step S53: Perform skeletal structure symmetry evaluation on the user's 3D skeletal model to generate skeletal symmetry data;

[0158] Step S54: Perform skeletal structural feature analysis on the skeletal symmetry data to generate skeletal structural feature data;

[0159] Step S55: Analyze the abnormal parts of the skeletal structure feature data to generate abnormal skeletal structure data.

[0160] In this embodiment, computer vision algorithms are used to process full-body motion images to extract joint information. Joints typically represent the positions of joints in the human body, such as shoulders, elbows, and knees. The extracted joint information is converted into joint point cloud data, which is point cloud data composed of the three-dimensional coordinates of joints, with each joint corresponding to a single point. Based on the joint point cloud data, a 3D reconstruction algorithm is used to reconstruct the user's skeletal structure. By inferring the continuity and posture of the bones from the spatial relationships between joints, a 3D skeletal model of the user is generated based on the reconstruction results. The skeletal model is a model composed of the three-dimensional representation of the bones, containing information such as the position and posture of the bones, and symmetry is used for evaluation. The algorithm performs symmetry analysis on the user's 3D skeletal model, including comparing the relative positions and symmetry between various joints, generating skeletal symmetry data, including symmetry evaluation results of the skeletal structure, such as symmetry index and symmetry ratio. Based on the skeletal symmetry data, it performs skeletal structural feature analysis, including measuring and calculating the length, angle, and proportion of bones, generating skeletal structural feature data, including feature descriptions of the skeletal structure, such as bone length ratio and angle range. Based on the skeletal structural feature data, it performs abnormal site analysis, comparing the skeletal structural features with the normal range, detecting abnormal sites, and generating skeletal abnormal structure data, including information such as the location, type, and degree of the abnormal sites.

[0161] In this embodiment, the specific steps of step S6 are as follows:

[0162] Step S61: Use deep learning algorithms to perform real-time fusion analysis on muscle health data and abnormal skeletal structure data to generate dynamic musculoskeletal data for users.

[0163] Step S62: Perform implicit association analysis on the user's dynamic musculoskeletal data to generate dynamic musculoskeletal association data;

[0164] Step S63: Perform potential risk analysis on the user's dynamic musculoskeletal data based on the dynamic musculoskeletal correlation data to obtain potential musculoskeletal risk data;

[0165] Step S64: Perform risk trend prediction on the musculoskeletal potential risk data to generate musculoskeletal risk trend data;

[0166] Step S65: Use the recurrent convolution algorithm to perform holographic visual modeling on musculoskeletal risk trend data and digital facial model to construct a dynamic holographic visual model for performing digital integration of health status.

[0167] In this embodiment, a deep learning algorithm is used to fuse and analyze muscle health status data and skeletal abnormality data. By inputting both types of data into a deep learning model for training and prediction, dynamic musculoskeletal data of the user is generated. Based on this dynamic musculoskeletal data, data mining techniques, such as association rule mining or cluster analysis, are used to perform implicit association analysis to discover the correlations between different muscle groups, generating dynamic musculoskeletal association data, including information such as the association relationships and correlation strengths between different muscle groups. Based on this dynamic musculoskeletal association data, potential risk analysis is performed. By assessing the correlation strength and abnormalities between different muscle groups, the potential musculoskeletal risks of the user are determined, generating dynamic musculoskeletal potential risk data, including information such as the degree of potential risk in musculoskeletal associations and descriptions of abnormalities. Based on this musculoskeletal potential risk data… Risk trend prediction is performed using time series analysis or machine learning algorithms. By analyzing historical data and trends, the future development trend of musculoskeletal risk is predicted, generating musculoskeletal risk trend data, including predicted values ​​and trend changes. Recurrent convolutional algorithms, such as Recurrent Convolutional Neural Networks (RCNN), are used to process the musculoskeletal risk trend data to help extract time-series features and perform global risk analysis. Combined with digital facial models, musculoskeletal risk trend data and digital facial models are used for holographic visual modeling. The holographic visual modeling presents the dynamic correlation and changes between musculoskeletal risk trends and digital facial models, constructing a dynamic holographic visual model. This model displays the digital integration results of the user's health status, including the visualization of musculoskeletal risk trends and the combination with digital facial models to form a dynamic holographic visual effect.

[0168] In this embodiment, a digital health status integration system for smart mirrors is provided, comprising:

[0169] The facial structure analysis module acquires user facial images and full-body motion images using the embedded camera in the smart mirror; it enhances the details of the user's facial images to generate enhanced facial images; it performs facial structure analysis on the enhanced facial images to generate facial structure data; and it performs micro-expression recognition on the enhanced facial images based on the facial structure data to generate user emotion data.

[0170] The pupil morphology feature module performs user visual feature analysis on the enhanced facial image to generate user visual feature data; based on the user visual feature data, it performs pupil morphology change analysis on the enhanced facial image to generate pupil morphology feature data; and it performs optical difference sensitivity analysis on the pupil morphology feature data to generate user eye physiological signal data.

[0171] The skin feature module performs subtle facial skin analysis on enhanced facial images based on user emotion data to generate subtle facial skin feature data; it also constructs a user facial profile based on user eye physiological signal data and subtle facial skin feature data to build a digital facial model.

[0172] The muscle morphology and structure module performs dynamic feature recognition on full-body motion images to generate user dynamic feature data; analyzes the muscle morphology and structure of the full-body motion images based on the user dynamic feature data to generate muscle morphology and structure data; and analyzes the muscle health status of the muscle morphology and structure data to generate muscle health status data.

[0173] The 3D skeleton model module reconstructs the user's 3D skeleton from the full-body motion image to generate a user 3D skeleton model; it analyzes the skeletal structure features of the user 3D skeleton model to generate skeletal structure feature data; and it analyzes abnormal parts of the skeletal structure feature data to generate abnormal skeletal structure data.

[0174] The holographic visual model module uses deep learning algorithms to perform real-time fusion analysis of muscle health status data and skeletal abnormality structure data to generate dynamic musculoskeletal data for users; it performs potential risk trend analysis on the dynamic musculoskeletal data to generate musculoskeletal risk trend data; and it uses a recurrent convolution algorithm to perform holographic visual modeling on the musculoskeletal risk trend data and digital facial model to construct a dynamic holographic visual model to perform digital integration of health status.

[0175] This invention utilizes a facial structure analysis module to enhance facial images and recognize micro-expressions, generating user facial structure and emotion data to help assess facial muscle activity, expressions, and emotional states. A pupil morphology feature module analyzes enhanced facial images and pupil morphology changes to generate user visual feature data and pupil morphology feature data, used to assess visual attention, pupillary response, and eye health. A skin feature module analyzes subtle facial skin details to generate detailed facial skin feature data, revealing the user's skin condition, such as texture, pigmentation, and pattern, to assess skin health and overall health. A muscle group morphology and structure module identifies muscle groups through full-body motion images and dynamic features. The system generates user dynamic feature data and muscle group morphology data to assess the user's posture, movements, and muscle usage, helping to understand muscle health and athletic performance. The 3D skeletal model module generates skeletal structure feature data and abnormal skeletal structure data through user 3D skeletal reconstruction and skeletal structure feature analysis, assessing the user's skeletal health, the impact of posture on the skeleton, and existing skeletal abnormalities. The holographic vision model module uses deep learning algorithms to fuse and analyze muscle health data and abnormal skeletal structure data, generating user dynamic musculoskeletal data and musculoskeletal risk trend data. Comprehensive analysis provides a more comprehensive musculoskeletal health assessment and risk prediction, helping to detect potential health problems early.

[0176] Therefore, the embodiments should be considered as exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of the equivalents of the application are intended to be included within the invention.

[0177] The above description is merely a specific embodiment of the present invention, enabling those skilled in the art to understand or implement the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein are implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the present invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features of the invention herein.

Claims

1. A method for digitally integrating health status data in a smart mirror, characterized in that, Includes the following steps: Step S1: Acquire user facial images and full-body motion images using the embedded camera in the smart mirror; enhance the details of the user's facial images to generate enhanced facial images; perform facial structure analysis on the enhanced facial images to generate facial structure data; perform micro-expression recognition on the enhanced facial images based on the facial structure data to generate user emotion data. Step S2 is as follows: Step S21: Perform user visual feature analysis on the detail-enhanced facial image to generate user visual feature data; Step S22: Perform eye trajectory optical flow tracking on the detail-enhanced facial image based on the user's visual feature data to generate user eye trajectory data; Step S23: Perform gaze focus identification on the user's eye trajectory data to generate the eye gaze focus; Step S24: Analyze pupil morphology changes in the detail-enhanced facial image based on eye gaze focus to generate pupil morphology feature data; Step S25: Perform optical difference sensitivity analysis on the pupil morphology feature data to generate user eye physiological signal data; The specific steps of step S24 are as follows: Step S241: Perform pupil scaling analysis on the detail-enhanced facial image based on eye gaze focus to generate pupil scaling data; Step S242: Perform curve fitting on the pupil dilation data to generate a pupil dilation curve; Step S243: Perform edge contour evolution analysis on the detail-enhanced facial image based on the pupil scaling curve to generate pupil edge contour change data; Step S244: Perform non-circularity structure calculation on the pupil edge contour change data to generate pupil contour non-circularity parameters; Step S245: Analyze the dynamic change characteristics of the pupil by using the non-circularity parameter of the pupil contour to generate the dynamic pupil scaling pattern; Step S246: Analyze the pupil morphology changes in the detail-enhanced facial image based on the dynamic scaling rules of the pupil to generate pupil morphology feature data; Step S3: Perform facial fine skin analysis on the enhanced facial image based on user emotion data to generate facial fine skin feature data; construct a user facial profile based on user eye physiological signal data and facial fine skin feature data to build a digital facial model; Step S4: Perform dynamic feature recognition on the full-body motion image to generate user dynamic feature data; perform muscle group morphology and structure analysis on the full-body motion image based on the user dynamic feature data to generate muscle group morphology and structure data; perform muscle health status analysis on the muscle group morphology and structure data to generate muscle health status data. Step S5: Perform 3D skeleton reconstruction on the full-body motion image to generate a 3D skeleton model of the user; perform skeletal structure feature analysis on the 3D skeleton model of the user to generate skeletal structure feature data; perform abnormal site analysis on the skeletal structure feature data to generate abnormal skeletal structure data. Step S6: Use deep learning algorithms to perform real-time fusion analysis on muscle health status data and skeletal abnormality structure data to generate dynamic musculoskeletal data for users; perform potential risk trend analysis on the dynamic musculoskeletal data to generate musculoskeletal risk trend data; use recurrent convolution algorithms to perform holographic visual modeling on the musculoskeletal risk trend data and digital facial model to construct a dynamic holographic visual model to perform digital integration of health status.

2. The method for digital integration of health status applied to smart mirrors according to claim 1, characterized in that, The specific steps of step S1 are as follows: Step S11: Acquire the user's facial image and full-body motion image using the embedded camera in the smart mirror; Step S12: Enhance the details of the user's facial image to generate a detail-enhanced facial image; Step S13: Perform facial feature node recognition on the detail-enhanced facial image to generate facial feature node position data; Step S14: Perform facial structure analysis on the detail-enhanced facial image based on the facial feature node position data to generate facial structure data; Step S15: Perform micro-expression recognition on the enhanced facial image based on facial structure data to generate user micro-expression data; Step S16: Perform emotional fluctuation analysis on the user's micro-expression data to generate user emotional data.

3. The method for digital integration of health status applied to smart mirrors according to claim 1, characterized in that, The specific steps of step S25 are as follows: Step S251: Expose the user to multi-frequency light to obtain pupil light response data; Step S252: Perform multi-frequency light response rate analysis on the user's pupillary light response data to generate multi-frequency light intensity response characteristic curves; Step S253: Detect pupil dilation saturation time based on the multi-frequency light intensity response characteristic curve to generate dilation saturation time data; Step S254: Perform multi-frequency optical inertial strain analysis on the pupil morphology feature data to generate multi-frequency pupil morphological qualitative change data; Step S255: Analyze the pupillary light loss recovery characteristics of the multi-frequency pupillary morphological qualitative change data using the scaling saturation time data to generate pupillary recovery data; Step S256: Perform photosensitive analysis on the pupil restorative data to generate pupil photosensitivity data; Step S257: Perform eye physiological health analysis on pupil morphological feature data based on pupil photosensitivity data to generate user eye physiological signal data.

4. The method for digital integration of health status applied to smart mirrors according to claim 1, characterized in that, Facial micro-skin analysis includes skin texture analysis and facial pigmentation distribution analysis. The specific steps in step S3 are as follows: Step S31: Based on user emotion data, perform skin texture recognition on the detail-enhanced facial image to generate skin texture data; Step S32: Perform skin texture analysis on the skin texture data to generate facial skin texture data; Step S33: Perform facial pigment distribution analysis on the enhanced facial image to generate pigment uniformity data; Step S34: Perform skin feature analysis on facial skin texture data and pigmentation evenness data to generate fine facial skin feature data; Step S35: Construct a user facial profile based on the user's eye physiological signal data and facial fine skin feature data to build a digital facial model.

5. The method for digital integration of health status applied to smart mirrors according to claim 1, characterized in that, The specific steps of step S4 are as follows: Step S41: Perform dynamic feature recognition on the full-body motion image to generate user dynamic feature data; Step S42: Perform fine-grained muscle tissue image segmentation on the whole-body motion image based on the user's dynamic feature data to generate a muscle tissue image; Step S43: Quantize muscle mass in the muscle tissue image to generate muscle mass data; Step S44: Perform muscle group distribution analysis on the muscle mass data to generate muscle group distribution data; Step S45: Perform muscle group morphology and structure analysis on muscle tissue images using muscle group distribution data to generate muscle group morphology and structure data. Step S46: Analyze the muscle group morphology and structure data to generate muscle health status data.

6. The method for digital integration of health status applied to smart mirrors according to claim 1, characterized in that, The specific steps of step S5 are as follows: Step S51: Use computer vision technology to perform joint point cloud recognition on the whole-body motion image to generate joint point cloud data; Step S52: Perform user 3D skeleton reconstruction on the joint point cloud data to generate a user 3D skeleton model; Step S53: Perform skeletal structure symmetry evaluation on the user's 3D skeletal model to generate skeletal symmetry data; Step S54: Perform skeletal structural feature analysis on the skeletal symmetry data to generate skeletal structural feature data; Step S55: Analyze the abnormal parts of the skeletal structure feature data to generate abnormal skeletal structure data.

7. The method for digital integration of health status applied to smart mirrors according to claim 1, characterized in that, The specific steps of step S6 are as follows: Step S61: Use deep learning algorithms to perform real-time fusion analysis on muscle health data and abnormal skeletal structure data to generate dynamic musculoskeletal data for users. Step S62: Perform implicit association analysis on the user's dynamic musculoskeletal data to generate dynamic musculoskeletal association data; Step S63: Perform potential risk analysis on the user's dynamic musculoskeletal data based on the dynamic musculoskeletal correlation data to obtain potential musculoskeletal risk data; Step S64: Perform risk trend prediction on the musculoskeletal potential risk data to generate musculoskeletal risk trend data; Step S65: Use the recurrent convolution algorithm to perform holographic visual modeling on musculoskeletal risk trend data and digital facial model to construct a dynamic holographic visual model for performing digital integration of health status.

8. A digital health status integration system applied to smart mirrors, characterized in that, For performing the method for digital integration of health status applied to a smart mirror as described in claim 1, comprising: The facial structure analysis module acquires user facial images and full-body motion images using the embedded camera in the smart mirror; it enhances the details of the user's facial images to generate enhanced facial images; it performs facial structure analysis on the enhanced facial images to generate facial structure data; and it performs micro-expression recognition on the enhanced facial images based on the facial structure data to generate user emotion data. The pupil morphology feature module performs user visual feature analysis on the enhanced facial image to generate user visual feature data; based on the user visual feature data, it performs pupil morphology change analysis on the enhanced facial image to generate pupil morphology feature data; and it performs optical difference sensitivity analysis on the pupil morphology feature data to generate user eye physiological signal data. The skin feature module performs subtle facial skin analysis on enhanced facial images based on user emotion data to generate subtle facial skin feature data; it also constructs a user facial profile based on user eye physiological signal data and subtle facial skin feature data to build a digital facial model. The muscle morphology and structure module performs dynamic feature recognition on full-body motion images to generate user dynamic feature data; analyzes the muscle morphology and structure of the full-body motion images based on the user dynamic feature data to generate muscle morphology and structure data; and analyzes the muscle health status of the muscle morphology and structure data to generate muscle health status data. The 3D skeleton model module reconstructs the user's 3D skeleton from the full-body motion image to generate a user 3D skeleton model; it analyzes the skeletal structure features of the user 3D skeleton model to generate skeletal structure feature data; and it analyzes abnormal parts of the skeletal structure feature data to generate abnormal skeletal structure data. The holographic visual model module uses deep learning algorithms to perform real-time fusion analysis of muscle health status data and skeletal abnormality structure data to generate dynamic musculoskeletal data for users; it performs potential risk trend analysis on the dynamic musculoskeletal data to generate musculoskeletal risk trend data; and it uses a recurrent convolution algorithm to perform holographic visual modeling on the musculoskeletal risk trend data and digital facial model to construct a dynamic holographic visual model to perform digital integration of health status.

Images