The invention relates to the field of
artificial intelligence and health management, in particular to a multi-
modal health state assessment method fusing tongue image,
heart rate and text, which takes the
tongue coating condition as the main judgment basis, measures the
heart rate of a user, and combines the
body condition manually input by the user, the text information such as
past medical history and the like to assess the health state of the user. A DeepSeek
large model is used for giving a health state
evaluation result, a user is assisted to effectively judge the physical condition of the user, in a tongue picture analysis module, YOLOv8 is used for positioning a
tongue body area, and a ResNet18-Transform mixed architecture model is used for
tongue coating classification; in the
heart rate measurement module, a
cascade classifier is used for carrying out
face detection, and after it is determined that a user directly faces a camera, the heart rate is measured through an Euler video amplification
algorithm; and in a suggestion generation module, a DeepSeek
large model is guided through cue word
engineering to analyze the multi-
modal information, and a targeted health state
evaluation result is generated.