Pretreatment disease management system based on multi-mode large model

Abstract

The invention discloses a preventive disease management system based on a multi-mode large model, and relates to the technical field of medical management, the preventive disease management system collects user lingual surface images and on-line questionnaire data to obtain traditional Chinese medicine physique, health symptoms and personal basic information, and the image data is subjected to preprocessing technologies such as filtering and enhancing to obtain the preventive disease information; key tongue picture characteristics such as tongue color, tongue coating and tongue shape are extracted; performing comprehensive health assessment on the multi-modal data by using a pre-trained deep learning model, and preliminarily judging viscera functions, meridian qi and blood states and potential health risks of the user; an evaluation result is fused with auxiliary information such as original data, a current season and a regional environment, and a personalized health planning scheme covering diet, work and rest and exercise suggestions is generated through a multi-modal large model; an AI-AGENT-based intelligent agent is adopted to execute a health management task, and real-time data feedback is utilized to realize continuous optimization of dynamic planning; and the problems of insufficient intelligence and individuation of the traditional disease prevention management are solved.

Description

Technical Field

[0001] The present invention relates to the field of medical management technology, and in particular to a preventive disease management system based on a multimodal large model. Background Art

[0002] At present, preventive health management is gradually gaining attention from all walks of life in terms of preventing diseases and regulating sub-health conditions.

[0003] Existing technologies primarily rely on traditional health scales and expert interviews for management. Most health management systems rely solely on user-completed questionnaires and simple indicator calculations, making it difficult to accurately and real-timely assess an individual's health status. Furthermore, traditional technologies lack the dynamic acquisition of internal and external signals, such as tongue images, and are unable to capture subtle changes in health characteristics. Furthermore, most existing methods employ universal solutions and cannot provide targeted health plans based on the user's specific constitution and living environment. Doctors or experts are limited to targeted guidance, making it difficult to adjust health plans in a timely manner, resulting in insufficient continuity and effectiveness in health management. Therefore, there is an urgent need for an innovative preventive health management system that can comprehensively collect multimodal data, implement intelligent health assessment and personalized planning, and possess a dynamic feedback optimization mechanism. Summary of the Invention

[0004] In view of the above existing problems, the present invention is proposed.

[0005] The present invention provides a preventive disease management system based on a multimodal large model to solve the problem that the existing technology mainly relies on traditional health scales and expert manual consultation for management. Most health management systems only rely on users to fill out questionnaires and simple indicator calculations, making it difficult to achieve accurate and real-time judgment of individual health status.

[0006] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0007] The embodiment of the present invention provides a preventive disease management system based on a multimodal large model, which includes:

[0008] The data collection module is used to collect users' tongue images and physical constitution, health symptoms and personal information obtained through online questionnaires;

[0009] The feature extraction module is used to remove noise, enhance the image, and extract tongue features from the collected tongue surface images, and to format and integrate the questionnaire data and meridian data transmitted by the supporting equipment to generate structured data;

[0010] The feature extraction module includes:

[0011] The image preprocessing unit uses median filtering, Gaussian filtering and histogram equalization algorithms to remove noise and enhance contrast of the collected tongue surface images;

[0012] The feature extraction unit extracts tongue color, tongue coating color and texture, and tongue shape features from the preprocessed image based on the theory of tongue diagnosis in Traditional Chinese Medicine, and combines them with the data collected in the questionnaire to construct a multidimensional feature vector;

[0013] The initial health assessment module uses a combination of convolutional neural network (CNN) and recurrent neural network (RNN) to perform a preliminary health status assessment on multimodal feature vectors;

[0014] The health planning module uses a multimodal large model to combine preliminary health assessment results and auxiliary information to generate a health planning plan, including dietary recommendations, work and rest adjustments, and exercise guidance;

[0015] The data feedback module, based on the AI-AGENT architecture, breaks down health plans into specific tasks, manages users through voice, text, or terminal notifications, and collects feedback in real time;

[0016] The dynamic optimization module analyzes feedback data and continuously optimizes the original health planning plan based on traditional Chinese medicine theory and environmental changes.

[0017] As a preferred solution of the multimodal large model-based disease prevention management system of the present invention, the data acquisition module includes:

[0018] The tongue surface image acquisition unit captures the user's tongue surface, including the tip, middle, root, and edges of the tongue, in a standard posture under uniform lighting conditions.

[0019] The user information collection unit collects the user's physical information, health symptoms and personal information through online questionnaires.

[0020] As a preferred solution of the multimodal large model-based disease prevention management system of the present invention, the feature extraction unit performs color feature extraction:

[0021] Define the color distribution index as:

[0022]

[0023] Among them, f col (c) represents the proportion of pixels within the color interval c, c represents the predefined color interval, N is the total number of pixels in the tongue surface area, Ω represents the tongue surface image area, I(x,y) represents the pixel value of the preprocessed image at coordinate (x,y), δ(I(x,y),c) is the judgment function, which takes 1 when I(x,y) falls into the color interval c and 0 otherwise;

[0024] In the feature extraction unit, texture feature extraction is performed:

[0025] Local texture coding is constructed using local binary patterns, and its formula is:

[0026]

[0027] Among them, f tex (x,y) represents the local texture encoding at the coordinate (x,y), x,y are spatial coordinates, i is the sampling point number, P represents the total number of sampling points in the neighborhood, x i ,y i is the coordinate of the ith neighborhood sampling point, i(x i ,y i ) represents the pixel value of the i-th sampling point, I(x,y) is the center pixel value, s(·) is the step function, 2 i is the corresponding weight factor;

[0028] The step function is defined as:

[0029] If z≥0, then s(z)=1,

[0030] If z<0, then s(z)=0;

[0031] Among them, s(z) represents the judgment of input z, and z is the input difference;

[0032] Calculate the global texture mean for the entire tongue area:

[0033]

[0034] Among them, F tex represents the mean value of the overall texture feature, |Ω| is the total number of pixels in the tongue surface area, and f tex (x,y) is the local texture code;

[0035] Constructing tongue coating thickness index:

[0036]

[0037] Among them, T thick represents the tongue coating thickness index, 1{·} is the indicator function, which takes 1 when the condition is met, τ is the texture threshold to distinguish thickness, |Ω| is the total number of pixels in the tongue surface area, and f tex (x,y) is the local texture code;

[0038] In the feature extraction unit, morphological feature extraction is performed:

[0039] Introducing the morphological feature vector F morph :

[0040] Among them, F morph represents the morphological feature vector, P is the perimeter of the tongue edge, A is the area of ​​the tongue surface, is the shape regularity index, π is the circumference of a circle, and D represents the degree of dispersion of the main directions of the tongue surface, reflecting the inclination and irregularity of the tongue;

[0041] Combine each feature with the structured questionnaire data to construct the overall feature vector:

[0042] F=[f col (c1),f col (c2),…,F tex ,F morph ,T thick ,Q],

[0043] Among them, F represents the final multidimensional feature vector, f col (c1),f col (c2), respectively represent the color distribution index in different predefined color intervals, F tex is the overall texture feature mean, F morph is the morphological feature vector, T thick is the tongue coating thickness index, and Q is the indicator vector composed of questionnaire data.

[0044] As a preferred solution of the preventive disease management system based on a multimodal large model described in the present invention, the initial health assessment module adopts a pre-trained deep learning model and takes a multi-dimensional feature vector as input to make a preliminary assessment of the user's health status, including the functional status of the internal organs, the status of qi and blood in the meridians, and disease risk prompts.

[0045] As a preferred solution of the multimodal large model-based disease prevention management system of the present invention, the processing flow of making a preliminary assessment of the user's health status in the initial health assessment module is divided into three stages: feature mapping, fusion, and output:

[0046] During the fusion process, the multimodal feature vector is divided into the image feature part F img After being mapped separately with the questionnaire data Q through nonlinear functions, the weighted summation is used to achieve fusion. The formula is:

[0047] F u =w t ·g t (F img )+w q ·g q (Q), where F img represents the image feature subvector, the color, texture and morphological information extracted from the preprocessing step, Q represents the structured data subvector composed of the questionnaire, and w tis the image feature weight coefficient, w q is the questionnaire data weight coefficient, g t (·) represents the nonlinear mapping function of image features, g q (·) represents the nonlinear mapping function of questionnaire data, F u is the fused feature vector;

[0048] The fused vector F u After the full connection layer mapping, the intermediate hidden representation is formed, which is mathematically expressed as:

[0049]

[0050] Among them, h represents the hidden layer output feature vector, W f is the weight matrix of the fully connected layer, b f is the corresponding bias term, σ(·) is the activation function;

[0051] The output layer mapping is used to calculate the health assessment results, and we get:

[0052] H=softmax(W oh +b o ),

[0053] Among them, H represents the user's health status assessment vector, and its components correspond to different health risks and status indicators, W o is the output layer weight matrix, b o is the output layer bias term, and the softmax(·) function normalizes the output to a probability distribution;

[0054] K-fold cross validation is used in the training phase, and the loss function is defined as:

[0055]

[0056] in, Represents the overall loss value, K is the cross-validation fold, M(F (k) ; Θ) is the model's predicted output for the k-fold training data, Θ represents the model parameter set, including w t ,w q ,W f ,b f ,W o ,b o , L(·,·) is the cross entropy loss function F (k) Represents the multidimensional feature vector of the k-fold input, H (k) is the true health evaluation label of the k-th fold.

[0057] As a preferred solution of the disease prevention management system based on a multimodal large model described in the present invention, the health planning module integrates the preliminary health assessment results with the user's original data, the current season and regional environmental information, and generates a personalized health planning plan after processing by the multimodal large model. The plan covers aspects such as dietary recommendations, work and rest adjustments, and exercise guidance.

[0058] As a preferred solution of the multimodal large model-based disease prevention management system of the present invention, the process of generating a personalized health planning plan in the health planning module includes:

[0059] The health planning output scheme P is defined as the mapping function of the multimodal large model to each information input:

[0060] P=G(H,Q r ,S,E;Φ),

[0061] Where P represents the generated personalized health planning plan, G(·;Φ) is a multimodal large model with a parameter set of Φ, H is the initial health assessment result, and Q r Represents the user's original data, including questionnaire information before processing, S represents the current season information, and E represents the regional environment information;

[0062] The large model uses the attention mechanism to assign weights to each input information and constructs feature representations for different information, which can be expressed as:

[0063]

[0064] Calculate the alignment score for each information:

[0065]

[0066] Among them, e j represents the alignment score of the jth category information, v is the vector used to calculate the attention weight, W j is the weight matrix corresponding to the j-th type of information, b j is the corresponding bias term, tanh(·) is the hyperbolic tangent nonlinear activation function;

[0067] Calculate attention weights:

[0068]

[0069] Among them, α j is the attention weight of the j-th category information;

[0070] The weighted summation forms an integrated feature representation:

[0071]

[0072] Among them, z is the integrated feature vector after fusion;

[0073] The planning scheme is generated based on the integrated features and is represented by the mapping function:

[0074] Among them, W p is the output layer weight matrix, b p is the output layer bias term, is the activation mapping function that transforms continuous output into specific recommendations.

[0075] As a preferred solution of the preventive disease management system based on a multimodal large model described in the present invention, the data feedback module delivers health management tasks to users through interactive methods including voice, text, and mobile terminal push, and at the same time collects real-time data on the user's execution status in diet, exercise, and daily routine through user active feedback and system periodic inquiries.

[0076] As a preferred solution of the preventive disease management system based on a multimodal large model described in the present invention, the dynamic optimization module performs statistics and analysis on the user execution data, and dynamically adjusts the original health planning plan in combination with traditional Chinese medicine health theory and environmental change information.

[0077] As a preferred solution of the multimodal large model-based disease prevention management system of the present invention, in which: in the dynamic optimization module, the method of dynamically adjusting the original health planning plan is:

[0078] The dynamic optimization module performs statistical and pattern recognition analysis on the execution data of the user's diet, rest and exercise management tasks. Let the execution data of the i-th health management task at time point t be recorded as E i (t), where i = 1, 2, ..., M represents different task dimensions and t = 1, 2, ..., T represents the sampling time;

[0079] The trend slope is calculated using linear regression, and the formula is:

[0080] Where ΔE i Indicates the changing trend of task i execution data over time, E i (t) represents the execution data of the i-th task at time t, t represents the time sampling point, represents the mean value of all sampling moments, that is, represents the mean of task i at all sampling moments, that is,

[0081] By statistical slope ΔE i With the preset threshold θi Contrast, θ i Used to determine the significance of the trend, if ΔE i >θ i , then the health planning adjustment of the i-th task is triggered, where θ i represents the adjustment trigger threshold of the i-th task, which is set based on historical data and TCM theory and experience. The threshold setting ensures sensitivity to abnormal fluctuations;

[0082] After detecting that the execution trend of a task reaches the trigger condition, the original plan is adjusted, and the dynamic adjustment process introduces external environment information F env and TCM health theory feedback TCM Quantitative indicators, and calculate the adjustment amount, the specific mapping function is defined as: ΔP i =λΔE i +μF env +νF TCM , where ΔP i represents the adjustment amount of the i-th health planning scheme, λ represents the weight coefficient of the execution data trend on the adjustment amount, ΔE i represents the trend slope of task i execution data, μ represents the influence weight of environmental information on planning adjustment, and F env represents the quantitative index of the current seasonal and regional climate and environmental change information, ν represents the weight of the TCM health theory feedback in the adjustment, and F TCM It represents feedback information converted into quantitative indicators based on traditional Chinese medicine theory;

[0083] After adjustment, the new planning scheme is updated as follows: in, represents the planning value of the i-th task in the original health planning scheme, Indicates the adjusted planning scheme value;

[0084] At the same time, the system sets a fixed feedback cycle T cycle At the end of each cycle, the collected execution data is statistically processed and the model is reconstructed. The periodic optimization process can be described as:

[0085]

[0086] in, It represents the optimization loss index within a cycle, which is used to quantify the deviation between the planned solution and the actual execution of the user. cycle It represents the adjustment and optimization cycle, which is in days or weeks and is set according to actual needs. P(t) represents the actual execution plan value at time t within the cycle. new represents the health plan updated according to the adjustment strategy at the end of the period, l(·,·) represents the loss function, which is used to measure the deviation between the execution data and the plan;

[0087] During this cycle, the system uses feedback data to continuously calculate the loss and uses gradient descent or other optimization algorithm strategies to fine-tune the weight coefficients λ, μ, and ν to ensure stable convergence of the dynamic adjustment algorithm.

[0088] The beneficial effects of the present invention are as follows: the present invention can more comprehensively reflect the user's health status and accurately judge organ dysfunction, qi and blood abnormalities and disease risks through multimodal collection of tongue images and online questionnaire data; use deep learning to process multidimensional features, and combine with large models to generate personalized health plans, and then implement tasks and provide real-time feedback through AI-AGENT-based intelligent bodies to form a complete closed-loop health management system; the system uses timed reminders, task decomposition and data feedback mechanisms to not only improve users' acceptance of health guidance, but also enhance the continuity of plan execution; with the help of intelligent processing and automatic feedback, it can not only reduce the workload of doctors and health management experts, but also achieve large-scale, multi-level personalized health management; convert traditional Chinese medicine health assessment methods into digital indicators, and combine them with modern data analysis technology to provide a scientific and dynamically optimized implementation path for Chinese medicine to prevent diseases. BRIEF DESCRIPTION OF THE DRAWINGS

[0089] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0090] Figure 1 This is a schematic diagram of the framework of the preventive disease management system based on the multimodal large model in Example 1. DETAILED DESCRIPTION

[0091] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0092] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0093] Secondly, the term "one embodiment" or "embodiment" herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in various places throughout this specification does not necessarily refer to the same embodiment, nor does it refer to a separate or selective embodiment that is mutually exclusive of other embodiments.

[0094] Example 1, reference Figure 1 This embodiment provides a preventive disease management system based on a multimodal large model, including:

[0095] The data collection module is used to collect users' tongue images and physical constitution, health symptoms and personal information obtained through online questionnaires;

[0096] The data acquisition module includes:

[0097] The tongue surface image acquisition unit captures the user's tongue surface, including the tip, middle, root, and edges of the tongue, in a standard posture under uniform lighting conditions.

[0098] User information collection unit, which collects users' physical information, health symptoms and personal information through online questionnaires;

[0099] The feature extraction module is used to remove noise, enhance the image, and extract tongue features from the collected tongue surface images, and to integrate the questionnaire data with the meridian data transmitted by the supporting equipment to generate structured data;

[0100] The feature extraction module includes:

[0101] The image preprocessing unit uses median filtering, Gaussian filtering and histogram equalization algorithms to remove noise and enhance contrast of the collected tongue surface images;

[0102] The feature extraction unit extracts tongue color, tongue coating color and texture, and tongue shape features from the preprocessed image based on the theory of tongue diagnosis in Traditional Chinese Medicine, and combines them with the data collected in the questionnaire to construct a multidimensional feature vector;

[0103] In the feature extraction unit, color feature extraction is performed:

[0104] Define the color distribution index as:

[0105]

[0106] Among them, f col (c) represents the proportion of pixels within the color interval c, c represents the predefined color interval, N is the total number of pixels in the tongue surface area, Ω represents the tongue surface image area, I(x,y) represents the pixel value of the preprocessed image at coordinate (x,y), δ(I(x,y),c) is the judgment function, which takes 1 when I(x,y) falls into the color interval c and 0 otherwise;

[0107] In the feature extraction unit, texture feature extraction is performed:

[0108] Local texture coding is constructed using local binary patterns, and its formula is:

[0109]

[0110] Among them, f tex (x,y) represents the local texture encoding at the coordinate (x,y), x,y are spatial coordinates, i is the sampling point number, P represents the total number of sampling points in the neighborhood, x i ,y i is the coordinate of the ith neighborhood sampling point, I(x i ,y i ) represents the pixel value of the i-th sampling point, I(x,y) is the center pixel value, s(·) is the step function, 2 i is the corresponding weight factor;

[0111] The step function is defined as:

[0112] If z≥0, then s(z)=1,

[0113] If z<0, then s(z)=0;

[0114] Among them, s(z) represents the judgment of input z, and z is the input difference;

[0115] Calculate the global texture mean for the entire tongue area:

[0116]

[0117] Among them, F tex represents the mean value of the overall texture feature, |Ω| is the total number of pixels in the tongue surface area, and f tex (x,y) is the local texture code;

[0118] Constructing tongue coating thickness index:

[0119]

[0120] Among them, T thick represents the tongue coating thickness index, 1{·} is the indicator function, which takes 1 when the condition is met, τ is the texture threshold to distinguish thickness, |Ω| is the total number of pixels in the tongue surface area, and f tex (x,y) is the local texture code;

[0121] In the feature extraction unit, morphological feature extraction is performed:

[0122] Introducing the morphological feature vector F morph :

[0123] Among them, F morph represents the morphological feature vector, P is the perimeter of the tongue edge, A is the area of ​​the tongue surface, is the shape regularity index, π is the circumference of a circle, and D represents the degree of dispersion of the main directions of the tongue surface, reflecting the inclination and irregularity of the tongue;

[0124] Combine each feature with the structured questionnaire data to construct the overall feature vector:

[0125] F=[f col (c1),f col (c2),…,F tex ,F morph ,T thick ,Q],

[0126] Among them, F represents the final multidimensional feature vector, f col (c1),f col (c2), respectively represent the color distribution index in different predefined color intervals, F tex is the overall texture feature mean, F morph is the morphological feature vector, T thick is the tongue coating thickness index, Q is the index vector composed of questionnaire data;

[0127] Specifically, tongue image characteristics are quantified from three perspectives: color, texture, and morphology, transforming the subjective judgment of traditional Chinese medicine into an objective quantitative indicator;

[0128] The initial health assessment module uses a combination of convolutional neural network (CNN) and recurrent neural network (RNN) to perform a preliminary health status assessment on multimodal feature vectors;

[0129] The initial health assessment module uses a pre-trained deep learning model and a multi-dimensional feature vector as input to make a preliminary assessment of the user's health status, including the functional status of the internal organs, the status of the meridians and blood, and disease risk indicators;

[0130] In the initial health assessment module, the process of making a preliminary assessment of the user's health status is divided into three stages: feature mapping, fusion, and output:

[0131] During the fusion process, the multimodal feature vector is divided into the image feature part F img After being mapped separately with the questionnaire data Q through nonlinear functions, the weighted summation is used to achieve fusion. The formula is:

[0132] F u =w t ·g t (F img )+w q ·g q(Q), where F img represents the image feature subvector, the color, texture and morphological information extracted from the preprocessing step, Q represents the structured data subvector composed of the questionnaire, and w t is the image feature weight coefficient, w q is the questionnaire data weight coefficient, g t (·) represents the nonlinear mapping function of image features, g q (·) represents the nonlinear mapping function of questionnaire data, F u is the fused feature vector;

[0133] The fused vector F u After the full connection layer mapping, the intermediate hidden representation is formed, which is mathematically expressed as:

[0134]

[0135] Among them, h represents the hidden layer output feature vector, W f is the weight matrix of the fully connected layer, b f is the corresponding bias term, σ(·) is the activation function;

[0136] The output layer mapping is used to calculate the health assessment results, and we get:

[0137] H=softmax(W oh +b o ),

[0138] Among them, H represents the user's health status assessment vector, and its components correspond to different health risks and status indicators, W o is the output layer weight matrix, b o is the output layer bias term, and the softmax(·) function normalizes the output to a probability distribution;

[0139] K-fold cross validation is used in the training phase, and the loss function is defined as:

[0140]

[0141] in, Represents the overall loss value, K is the cross-validation fold, M(F (k) ; Θ) is the model's predicted output for the k-fold training data, Θ represents the model parameter set, including w t ,w q ,W f ,b f ,W o ,b o , L(·,·) is the cross entropy loss function F (k) Represents the multidimensional feature vector of the k-fold input, H (k)is the true health evaluation label of the k-th fold;

[0142] Specifically, the pre-trained deep learning model is divided into multiple layers. Nonlinear mapping and weight adjustment are used between modules to achieve adaptive feature fusion of data from different modalities. After image features and questionnaire data are processed independently, the information is integrated using a weighted summation method to effectively retain their respective key information. The hidden layer obtains a comprehensive representation through fully connected mapping, and the softmax function is used to output the probability distribution of health status, thereby achieving a preliminary assessment.

[0143] The health planning module uses a multimodal large model to combine preliminary health assessment results and auxiliary information to generate a health planning plan, including dietary recommendations, work and rest adjustments, and exercise guidance;

[0144] The health planning module integrates preliminary health assessment results with the user's original data, current season and regional environmental information, and generates a personalized health plan after processing through a multimodal large model. The plan covers dietary recommendations, work and rest adjustments, and exercise guidance.

[0145] In the health planning module, the process of generating a personalized health planning plan includes:

[0146] The health planning output scheme P is defined as the mapping function of the multimodal large model to each information input:

[0147] P=G(H,Q r ,S,E;Φ),

[0148] Where P represents the generated personalized health planning plan, G(·;Φ) is a multimodal large model with a parameter set of Φ, H is the initial health assessment result, and Q r Represents the user's original data, including questionnaire information before processing, S represents the current season information, and E represents the regional environment information;

[0149] The large model uses the attention mechanism to assign weights to each input information and constructs feature representations for different information, which can be expressed as:

[0150]

[0151] Calculate the alignment score for each information:

[0152]

[0153] Among them, e j represents the alignment score of the jth category information, v is the vector used to calculate the attention weight, W j is the weight matrix corresponding to the j-th type of information, b j is the corresponding bias term, tanh(·) is the hyperbolic tangent nonlinear activation function;

[0154] Calculate attention weights:

[0155]

[0156] Among them, α j is the attention weight of the j-th category information;

[0157] The weighted summation forms an integrated feature representation:

[0158]

[0159] Among them, z is the integrated feature vector after fusion;

[0160] The planning scheme is generated based on the integrated features and is represented by the mapping function:

[0161] Among them, W p is the output layer weight matrix, b p is the output layer bias term, To activate the mapping function, which is used to transform the continuous output into specific suggestions;

[0162] Specifically, a multimodal large model is constructed to fully integrate the preliminary health assessment results with the user's original data, current season and regional environmental information. The attention mechanism assigns reasonable weights to different categories of information, enabling the model to capture the different impacts of each information on health management under the guidance of Traditional Chinese Medicine theory. The calculation of alignment scores and the weight assignment process ensure the model's refined processing at the semantic understanding level, thereby generating personalized plans based on the integration of traditional Chinese medicine theory, individual user health status and external environmental factors. After the integrated features are fully mapped, a diet, work and rest and exercise plan is formed;

[0163] The data feedback module, based on the AI-AGENT architecture, breaks down health plans into specific tasks, manages users through voice, text, or terminal notifications, and collects feedback in real time;

[0164] The data feedback module delivers health management tasks to users through interactive methods including voice, text, and mobile terminal push. At the same time, it collects real-time data on users' diet, exercise, and daily routines through active user feedback and system periodic inquiries.

[0165] Dynamic optimization module analyzes feedback data and continuously optimizes the original health planning plan based on traditional Chinese medicine theory and environmental changes;

[0166] The dynamic optimization module collects statistics and analyzes user execution data, and dynamically adjusts the original health planning plan based on traditional Chinese medicine health theory and environmental change information;

[0167] In the dynamic optimization module, the method of dynamically adjusting the original health planning plan is as follows:

[0168] The dynamic optimization module performs statistical and pattern recognition analysis on the execution data of the user's diet, rest and exercise management tasks. Let the execution data of the i-th health management task at time point t be recorded as E i (t), where i = 1, 2, ..., M represents different task dimensions and t = 1, 2, ..., T represents the sampling time;

[0169] The trend slope is calculated using linear regression, and the formula is:

[0170] Where ΔE i Indicates the changing trend of task i execution data over time, E i (t) represents the execution data of the i-th task at time t, t represents the time sampling point, represents the mean value of all sampling moments, that is, represents the mean of task i at all sampling moments, that is,

[0171] By statistical slope ΔE i With the preset threshold θ i Contrast, θ i Used to determine the significance of the trend, if ΔE i >θ i , then the health planning adjustment of the i-th task is triggered, where θ i represents the adjustment trigger threshold of the i-th task, which is set based on historical data and TCM theory and experience. The threshold setting ensures sensitivity to abnormal fluctuations;

[0172] Specifically, through statistical analysis methods, the trend of the data of various tasks in the user's health management process over time is quantified using mathematical formulas, and linear regression is used to calculate the trend slope to determine the improvement or degradation of the user's execution status. Preset thresholds can be used to identify tasks with abnormal fluctuations in execution data, providing a judgment basis for subsequent dynamic adjustments;

[0173] After detecting that the execution trend of a task reaches the trigger condition, the original plan is adjusted, and the dynamic adjustment process introduces external environment information F env and TCM health theory feedback TCM Quantitative indicators, and calculate the adjustment amount, the specific mapping function is defined as: ΔP i =λΔE i +μF env +νF TCM , where ΔP irepresents the adjustment amount of the i-th health planning scheme, λ represents the weight coefficient of the execution data trend on the adjustment amount, ΔE i represents the trend slope of task i execution data, μ represents the influence weight of environmental information on planning adjustment, and F env represents the quantitative index of the current seasonal and regional climate and environmental change information, ν represents the weight of the TCM health theory feedback in the adjustment, and F TCM It represents feedback information converted into quantitative indicators based on traditional Chinese medicine theory;

[0174] After adjustment, the new planning scheme is updated as follows: in, represents the planning value of the i-th task in the original health planning scheme, Indicates the adjusted planning scheme value;

[0175] At the same time, the system sets a fixed feedback cycle T cycle At the end of each cycle, the collected execution data is statistically processed and the model is reconstructed. The periodic optimization process can be described as:

[0176]

[0177] in, It represents the optimization loss index within a cycle, which is used to quantify the deviation between the planned solution and the actual execution of the user. cycle It represents the adjustment and optimization cycle, which is in days or weeks and is set according to actual needs. P(t) represents the actual execution plan value at time t within the cycle. new represents the health plan updated according to the adjustment strategy at the end of the period, l(·,·) represents the loss function, which is used to measure the deviation between the execution data and the plan.

[0178] During this cycle, the system continuously calculates the loss using feedback data and fine-tunes the weight coefficients λ, μ, and ν using gradient descent or other optimization algorithm strategies to ensure stable convergence of the dynamic adjustment algorithm.

[0179] The preventive treatment management system provided in this embodiment also includes an AI-AGENT-based intelligent agent. The intelligent agent uses perception, decision-making, execution, and communication modules to decompose health planning plans into specific tasks and establish mapping relationships with each task to facilitate real-time task push and health management implementation.

[0180] A unified data structure and naming rules are used between modules to perform standardized data processing and transmission;

[0181] The intelligent body is equipped with a natural language processing unit to realize voice or text interaction with the user, so that the health plan content can be conveyed to the user in an easy-to-understand manner, and the user is encouraged to implement health management according to the recommended plan.

[0182] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A preventive disease management system based on a multimodal large model, characterized by: include, The data collection module is used to collect users' tongue images and physical constitution, health symptoms and personal information obtained through online questionnaires; The feature extraction module is used to remove noise, enhance the image, and extract tongue features from the collected tongue surface images, and to format and integrate the questionnaire data and meridian data transmitted by the supporting equipment to generate structured data; The feature extraction module includes: The image preprocessing unit uses median filtering, Gaussian filtering and histogram equalization algorithms to remove noise and enhance contrast of the collected tongue surface images; The feature extraction unit extracts tongue color, tongue coating color and texture, and tongue shape features from the preprocessed image based on the theory of tongue diagnosis in Traditional Chinese Medicine, and combines them with the data collected in the questionnaire to construct a multidimensional feature vector; The initial health assessment module uses a combination of convolutional neural network (CNN) and recurrent neural network (RNN) to perform a preliminary health status assessment on multimodal feature vectors; The health planning module uses a multimodal large model to combine preliminary health assessment results and auxiliary information to generate a health planning plan, including dietary recommendations, work and rest adjustments, and exercise guidance; The data feedback module, based on the AI-AGENT architecture, breaks down health plans into specific tasks, manages users through voice, text, or terminal notifications, and collects feedback in real time; The dynamic optimization module analyzes feedback data and continuously optimizes the original health planning plan based on traditional Chinese medicine theory and environmental changes.

2. The preventive disease management system based on a multimodal large model according to claim 1, characterized in that: The data acquisition module includes: The tongue surface image acquisition unit captures the user's tongue surface, including the tip, middle, root, and edges of the tongue, in a standard posture under uniform lighting conditions. The user information collection unit collects the user's physical information, health symptoms and personal information through online questionnaires.

3. The preventive disease management system based on a multimodal large model according to claim 1, characterized in that: In the feature extraction unit, color feature extraction is performed: Define the color distribution index as: Among them, f col (c) represents the proportion of pixels within the color interval c, c represents the predefined color interval, N is the total number of pixels in the tongue surface area, Ω represents the tongue surface image area, I(x,y) represents the pixel value of the preprocessed image at coordinate (x,y), δ(I(x,y),c) is the judgment function, which takes 1 when I(x,y) falls into the color interval c and 0 otherwise; In the feature extraction unit, texture feature extraction is performed: Local texture coding is constructed using local binary patterns, and its formula is: Among them, f tex (x,y) represents the local texture encoding at the coordinate (x,y), x,y are spatial coordinates, i is the sampling point number, P represents the total number of sampling points in the neighborhood, x i ,y i is the coordinate of the ith neighborhood sampling point, I(x i ,y i ) represents the pixel value of the i-th sampling point, I(x,y) is the center pixel value, s(·) is the step function, 2 i is the corresponding weight factor; The step function is defined as: If z≥0, then s(z)=1, If z<0, then s(z)=0; Among them, s(z) represents the judgment of input z, and z is the input difference; Calculate the global texture mean for the entire tongue area: Among them, F tex represents the mean value of the overall texture feature, |Ω| is the total number of pixels in the tongue surface area, and f tex (x,y) is the local texture code; Constructing tongue coating thickness index: Among them, T thick represents the tongue coating thickness index, 1{·} is the indicator function, which takes 1 when the condition is met, τ is the texture threshold to distinguish thickness, |Ω| is the total number of pixels in the tongue surface area, and f tex (x,y) is the local texture code; In the feature extraction unit, morphological feature extraction is performed: Introducing the morphological feature vector F morph : Among them, F morph represents the morphological feature vector, P is the perimeter of the tongue edge, A is the area of ​​the tongue surface, is the shape regularity index, π is the circumference of a circle, and D represents the degree of dispersion of the main directions of the tongue surface, reflecting the inclination and irregularity of the tongue; Combine each feature with the structured questionnaire data to construct the overall feature vector: F=[f col (c1),f col (c2),…,F tex ,F morph ,T thick ,Q], Among them, F represents the final multidimensional feature vector, f col (c1),f col (c2), respectively represent the color distribution index in different predefined color intervals, F tex is the overall texture feature mean, F morph is the morphological feature vector, T thick is the tongue coating thickness index, and Q is the indicator vector composed of questionnaire data.

4. The preventive disease management system based on a multimodal large model according to claim 1, characterized in that: The initial health assessment module adopts a pre-trained deep learning model and takes a multi-dimensional feature vector as input to make a preliminary assessment of the user's health status, including the functional status of the internal organs, the status of the meridians and blood, and disease risk prompts.

5. The preventive disease management system based on a multimodal large model as claimed in claim 4, characterized in that: In the initial health assessment module, the process of making a preliminary assessment of the user's health status is divided into three stages: feature mapping, fusion, and output: During the fusion process, the multimodal feature vector is divided into the image feature part F img After being mapped separately with the questionnaire data Q through nonlinear functions, the weighted summation is used to achieve fusion. The formula is: F u =w i ·g t (F img )+w q ·g q (Q), where F img represents the image feature subvector, the color, texture and morphological information extracted from the preprocessing step, Q represents the structured data subvector composed of the questionnaire, and w t is the image feature weight coefficient, w q is the questionnaire data weight coefficient, g t (·) represents the nonlinear mapping function of image features, g q (·) represents the nonlinear mapping function of questionnaire data, F u is the fused feature vector; The fused vector F u After the full connection layer mapping, the intermediate hidden representation is formed, which is mathematically expressed as: Among them, h represents the hidden layer output feature vector, W f is the weight matrix of the fully connected layer, b f is the corresponding bias term, σ(·) is the activation function; The output layer mapping is used to calculate the health assessment results, and we get: H=softmax(W oh +b o ), Among them, H represents the user's health status assessment vector, and its components correspond to different health risks and status indicators, W o is the output layer weight matrix, b o is the output layer bias term, and the softmax(·) function normalizes the output to a probability distribution; K-fold cross validation is used in the training phase, and the loss function is defined as: in, Represents the overall loss value, K is the cross-validation fold, M(F (k) ; Θ) is the model's predicted output for the k-fold training data, Θ represents the model parameter set, including w t ,w q ,W f ,b f ,W o ,b o , L(·,·) is the cross entropy loss function F (k) Represents the multidimensional feature vector of the k-fold input, H (k) is the true health evaluation label of the k-th fold.

6. The preventive disease management system based on a multimodal large model according to claim 1, characterized in that: The health planning module integrates the preliminary health assessment results with the user's original data, current season and regional environmental information, and generates a personalized health planning plan after processing through a multimodal large model.

7. The preventive disease management system based on a multimodal large model according to claim 6, characterized in that: In the health planning module, the process of generating a personalized health planning plan includes: The health planning output scheme P is defined as the mapping function of the multimodal large model to each information input: P=G(H,Q r ,S,E;Φ), Where P represents the generated personalized health planning plan, G(·;Φ) is a multimodal large model with a parameter set of Φ, H is the initial health assessment result, and Q r Represents the user's original data, including questionnaire information before processing, S represents the current season information, and E represents the regional environment information; The large model uses the attention mechanism to assign weights to each input information and constructs feature representations for different information, which can be expressed as: Calculate the alignment score for each information: Among them, e j represents the alignment score of the jth category information, v is the vector used to calculate the attention weight, W j is the weight matrix corresponding to the j-th type of information, b j is the corresponding bias term, tanh(·) is the hyperbolic tangent nonlinear activation function; Calculate attention weights: Among them, α j is the attention weight of the j-th category information; The weighted summation forms an integrated feature representation: Among them, z is the integrated feature vector after fusion; The planning scheme is generated based on the integrated features and is represented by the mapping function: Among them, W p is the output layer weight matrix, b p is the output layer bias term, is the activation mapping function that transforms continuous output into specific recommendations.

8. The preventive disease management system based on a multimodal large model according to claim 1, characterized in that: The data feedback module delivers health management tasks to users through interactive methods including voice, text, and mobile terminal push, and at the same time collects real-time data on users' performance in diet, exercise, and daily routines through active user feedback and system periodic inquiries.

9. The preventive disease management system based on a multimodal large model according to claim 1, characterized in that: The dynamic optimization module collects and analyzes the user execution data, and dynamically adjusts the original health planning program in combination with traditional Chinese medicine health theory and environmental change information.

10. The preventive disease management system based on a multimodal large model according to claim 9, characterized in that: In the dynamic optimization module, the method of dynamically adjusting the original health planning plan is as follows: The dynamic optimization module performs statistical and pattern recognition analysis on the execution data of the user's diet, rest and exercise management tasks. Let the execution data of the i-th health management task at time point t be recorded as E i (t), where i = 1, 2, ..., M represents different task dimensions and t = 1, 2, ..., T represents the sampling time; The trend slope is calculated using linear regression, and the formula is: Where ΔE i Indicates the changing trend of task i execution data over time, E i (t) represents the execution data of the i-th task at time t, t represents the time sampling point, represents the mean value of all sampling moments, that is, represents the mean of task i at all sampling moments, that is, By statistical slope ΔE i With the preset threshold θ i Contrast, θ i Used to determine the significance of the trend, if ΔE i >θ i , then the health planning adjustment of the i-th task is triggered, where θ i represents the adjustment trigger threshold of the i-th task, which is set based on historical data and TCM theory and experience; After detecting that the execution trend of a task reaches the trigger condition, the original plan is adjusted, and the dynamic adjustment process introduces external environment information F env and TCM health theory feedback TCM Quantitative indicators, and calculate the adjustment amount, the specific mapping function is defined as: ΔP i =λΔE i +μF env +νF TCM , where ΔP i represents the adjustment amount of the i-th health planning scheme, λ represents the weight coefficient of the execution data trend on the adjustment amount, ΔE i represents the trend slope of task i execution data, μ represents the influence weight of environmental information on planning adjustment, and F env represents the quantitative index of the current seasonal and regional climate and environmental change information, ν represents the weight of the TCM health theory feedback in the adjustment, and F TCM It represents feedback information converted into quantitative indicators based on traditional Chinese medicine theory; After adjustment, the new planning scheme is updated as follows: in, represents the planning value of the i-th task in the original health planning scheme, Indicates the adjusted planning scheme value.

Images