Health information generation method, storage medium, and electronic device

Abstract

The present disclosure provides a health information generation method, a storage medium and an electronic device, the method comprising: acquiring multi-modal health data of a user, the multi-modal health data comprising at least: basic information data and physiological information data, the basic information data being used to represent personal basic characteristics of the user, and the physiological information data being used to represent physiological state of the user; determining health suggestion information of the user according to the multi-modal health data; and outputting the health suggestion information. In the health information generation method provided by the present disclosure, the device can generate health suggestion information for the current user based on the real-time acquired multi-modal health data. The fusion analysis of the health data of the user in different modes is realized, and the limitation of only processing a single index in the related technical solutions is broken through, and more real-time, scientific and comprehensive health suggestion information can be provided for the user.

Description

Technical Field

[0001] This disclosure relates to the field of health management technology, and in particular to a method for generating health information, a storage medium, and an electronic device. Background Technology

[0002] With the ever-accelerating pace of modern life, people are paying increasing attention to their health. Traditional health management often focuses on periodic physical examinations at medical institutions or understanding one's health status through the monitoring of a single indicator (such as heart rate monitoring).

[0003] However, such methods often suffer from problems such as delayed feedback and limited data dimensions, making it difficult to meet people's needs for real-time, comprehensive, and intelligent health management. Summary of the Invention

[0004] In view of this, embodiments of the present disclosure provide a method for generating health information, a storage medium, and an electronic device.

[0005] According to a first aspect of this disclosure, a method for generating health information is proposed, the method comprising:

[0006] Acquire the user's multimodal health data, which includes at least: basic information data and physiological information data, wherein the basic information data is used to characterize the user's basic personal characteristics and the physiological information data is used to characterize the user's physiological state;

[0007] Based on the multimodal health data, determine the user's health recommendation information;

[0008] Output the aforementioned health advice information.

[0009] In conjunction with any of the embodiments provided in this disclosure,

[0010] The multimodal health data also includes at least one of the following:

[0011] The user's dietary information data is used to characterize the user's nutritional intake;

[0012] The user's motion information data, which is used to characterize the user's motion status;

[0013] The health advice information includes at least one of the following:

[0014] The user's health status assessment results;

[0015] The dietary guidance and advice information for the user;

[0016] The user's exercise guidance and suggestions information.

[0017] In conjunction with any embodiment provided in this disclosure, when the multimodal health data includes the dietary information data, the step of obtaining the user's multimodal health data includes:

[0018] Receive food images uploaded by users;

[0019] The food images are then subjected to image recognition based on an image recognition model to obtain the image recognition results.

[0020] The dietary information data is determined based on the image recognition results.

[0021] In conjunction with any embodiment provided in this disclosure, after obtaining the image recognition result and before determining the dietary information data, the method further includes:

[0022] Receive correction information obtained by the user to correct the image recognition result;

[0023] The image recognition result is updated based on the correction information.

[0024] In conjunction with any embodiment provided in this disclosure, determining the user's health recommendation information based on the multimodal health data includes:

[0025] The multimodal health data is input into a preset model to obtain health advice information for the user output by the preset model.

[0026] In conjunction with any of the embodiments provided in this disclosure, the preset model is trained in the following manner:

[0027] Acquire training data, which includes at least one set of multimodal health data labeled with health advice information;

[0028] The first multimodal health data in the training data is input into the preset model to be trained, and the predicted health suggestion information output by the preset model to be trained, corresponding to the first multimodal health data, is obtained.

[0029] The network parameters of the preset model are adjusted based on the difference between the predicted health advice information and the health advice information in the training data corresponding to the first multimodal health data.

[0030] In conjunction with any of the embodiments provided in this disclosure, the output of the health advice information includes at least one of the following:

[0031] Output health advice information for the user in natural language text description;

[0032] The health advice information described in the natural language text will be output via voice.

[0033] Based on the health advice information described in the natural language text, a text-to-image model is used to generate visualized images and / or videos, and the visualized images and / or videos are output.

[0034] In any embodiment provided by this disclosure, the multimodal data further includes: the user's health management goals;

[0035] The step of determining the user's health advice information based on the multimodal health data includes:

[0036] The basic information data, physiological information data, and health management goals are input into the large model to obtain health recommendation information output by the large model to achieve the health management goals.

[0037] In conjunction with any of the embodiments provided in this disclosure, the acquisition of the user's multimodal health data includes:

[0038] Based on the time of receiving a user's health management request, historical multimodal health data within a preset time period prior to the receiving time is obtained.

[0039] According to a second aspect of this disclosure, an electronic device is provided, comprising:

[0040] The data acquisition module is used to acquire the user's multimodal health data, which includes at least: basic information data and physiological information data. The basic information data is used to characterize the user's basic personal characteristics, and the physiological information data is used to characterize the user's physiological state.

[0041] An information generation module is used to determine health recommendation information for the user based on the multimodal health data;

[0042] The information output module is used to output the health advice information.

[0043] In conjunction with any of the embodiments provided in this disclosure,

[0044] The multimodal health data also includes at least one of the following:

[0045] The user's dietary information data is used to characterize the user's nutritional intake;

[0046] The user's motion information data, which is used to characterize the user's motion status;

[0047] The health advice information includes at least one of the following:

[0048] The user's health status assessment results;

[0049] The dietary guidance and advice information for the user;

[0050] The user's exercise guidance and suggestions information.

[0051] In conjunction with any embodiment provided in this disclosure, when the multimodal health data includes the dietary information data, the data acquisition module, when acquiring the user's multimodal health data, includes:

[0052] Receive food images uploaded by users;

[0053] The food images are then subjected to image recognition based on an image recognition model to obtain the image recognition results.

[0054] The dietary information data is determined based on the image recognition results.

[0055] In conjunction with any embodiment provided in this disclosure, the electronic device further includes:

[0056] The correction information receiving module is used to receive correction information obtained by the user correcting the image recognition result;

[0057] The image recognition result update module is used to update the image recognition result based on the correction information.

[0058] In conjunction with any embodiment provided in this disclosure, the information generation module, when used to determine the user's health recommendation information based on the multimodal health data, includes:

[0059] The multimodal health data is input into a preset model to obtain health advice information for the user, described in natural language text output by the preset model.

[0060] In conjunction with any embodiment provided in this disclosure, the preset model is trained based on a training device for the preset model, the training device for the preset model comprising:

[0061] The training data acquisition module is used to acquire training data, which includes at least one set of multimodal health data labeled with health advice information.

[0062] The prediction result acquisition module is used to input the first multimodal health data in the training data into the preset model to be trained, and obtain the predicted health suggestion information output by the preset model to be trained corresponding to the first multimodal health data.

[0063] The network parameter adjustment module is used to adjust the network parameters of the preset model based on the difference between the predicted health advice information and the health advice information in the training data corresponding to the first multimodal health data.

[0064] In conjunction with any embodiment provided in this disclosure, the information output module, when used to output the health advice information, includes at least one of the following:

[0065] Output health advice information for the user in natural language text description;

[0066] The health advice information described in the natural language text will be output via voice.

[0067] Based on the health advice information described in the natural language text, a text-to-image model is used to generate visualized images and / or videos, and the visualized images and / or videos are output.

[0068] In any embodiment provided by this disclosure, the multimodal data further includes: the user's health management goals;

[0069] The information generation module, when used to determine the user's health recommendation information based on the multimodal health data, includes:

[0070] The basic information data, physiological information data, and health management goals are input into the large model to obtain health recommendation information output by the large model to achieve the health management goals.

[0071] In conjunction with any embodiment provided in this disclosure, the data acquisition module, when used to acquire a user's multimodal health data, includes:

[0072] Based on the time of receiving a user's health management request, historical multimodal health data within a preset time period prior to the receiving time is obtained.

[0073] According to a third aspect of this disclosure, a computer-readable storage medium is provided, the machine-readable storage medium storing machine-readable instructions, which, when invoked and executed by a processor, cause the processor to implement a health information generation method according to any embodiment of this disclosure.

[0074] According to a fourth aspect of this disclosure, an electronic device is provided, comprising:

[0075] processor;

[0076] Memory used to store processor-executable instructions;

[0077] The processor is configured to execute a health information generation method according to any embodiment of this disclosure.

[0078] The technical solutions provided by the embodiments of this disclosure may include the following beneficial effects:

[0079] The health information generation method, storage medium, and electronic device provided in this disclosure can generate health advice information for the current user based on real-time acquired multimodal health data, realize the fusion analysis of user health data of different modalities, break through the limitation of only processing a single indicator in related technical solutions, and provide users with more real-time, scientific, and comprehensive health advice information.

[0080] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0081] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0082] Figure 1 This disclosure is a system architecture diagram of a health monitoring system according to an exemplary embodiment;

[0083] Figure 2 This is a flowchart illustrating a method for generating health information according to an exemplary embodiment of the present disclosure;

[0084] Figure 3 This is a flowchart illustrating another method for generating health information according to an exemplary embodiment of the present disclosure;

[0085] Figure 4 This is a flowchart illustrating another method for generating health information according to an exemplary embodiment of the present disclosure;

[0086] Figure 5 This is a schematic diagram of the structure of an electronic device according to an exemplary embodiment of the present disclosure;

[0087] Figure 6 This is a schematic diagram of the structure of another electronic device according to an exemplary embodiment of the present disclosure. Detailed Implementation

[0088] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. In the following description relating to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements.

[0089] With the ever-accelerating pace of modern life, people are paying increasing attention to their health. Traditional health management often focuses on periodic physical examinations at medical institutions or on monitoring single indicators (such as heart rate) to understand one's health status. However, these methods often suffer from problems such as delayed feedback and limited data dimensions, making it difficult to meet people's needs for real-time, comprehensive, and intelligent health management.

[0090] In view of this, embodiments of this disclosure provide a method for generating health information. In this method, a device can generate health advice information for the current user based on real-time acquired multimodal health data using a large model. By using a large model to achieve fusion analysis of user health data from different modalities, this method overcomes the limitations of related technical solutions that only process single indicators, and can provide users with more real-time, scientific, and comprehensive health advice information.

[0091] To better describe the health information generation method provided in this disclosure, the system architecture of the AI ​​(Artificial Intelligence)-based health monitoring system corresponding to this method is first introduced as follows. Users can obtain health advice information based on this health monitoring system; that is, when a user uses the health monitoring system, the device can execute the health information generation method provided in this disclosure.

[0092] like Figure 1 As shown, the health monitoring system may include: infrastructure layer, data layer, algorithm layer, service layer, application layer and access layer.

[0093] 1. Infrastructure layer:

[0094] The infrastructure layer provides underlying hardware and basic communication capabilities, including but not limited to data acquisition, communication, storage, scheduling, and computation. Specifically, it can collect physiological information data (including but not limited to heart rate, sleep, blood oxygen, and calories) based on wearable devices such as watches and wristbands. Communication is conducted via network communication buses and data transmission links (such as Wi-Fi, Bluetooth, and cellular networks). Data is stored using RAM (Random Access Memory) and HDD (Hard Disk Drive). Scheduling is performed using the CPU (Central Processing Unit), and computation is performed using the GPU (Graphics Processing Unit).

[0095] 2. Data Layer:

[0096] The data layer can aggregate multimodal data, including but not limited to:

[0097] Physiological data (heart rate, sleep, blood oxygen, calories, steps, mood, etc.).

[0098] Basic information data (age, gender, height, weight, etc.).

[0099] Exercise information data (type of exercise, duration of exercise, calories burned, etc.).

[0100] Dietary information data (breakfast, lunch, dinner, etc.).

[0101] Intelligent insights into data (trend analysis and anomaly detection of multimodal data).

[0102] Among them, intelligent insight data is acquired based on other information data. For example, trend analysis can be performed on heart rate information collected over a period of time to obtain intelligent insight data 1 (heart rate increases or decreases over time). Alternatively, anomaly detection can be performed on heart rate information collected over a period of time. Specifically, heart rate values ​​that are outside the preset heart rate range can be detected in the heart rate information collected during this period to obtain intelligent insight data 2.

[0103] The data layer provides raw data and intelligent insight data for use by upper-layer modules, and is the starting point for this disclosure to realize multimodal health data fusion and intelligent analysis.

[0104] 3. Algorithm layer:

[0105] The algorithm layer is the core of the intelligence disclosed herein, including large model clusters and related fine-tuning strategies.

[0106] The large model cluster includes:

[0107] (1) Image-to-text capability (i.e., large image-to-text models, such as Qwen2-VL, Qwen-VL-Plus, etc.): used to identify food images uploaded by users, extract food types, and provide accurate data support for dietary guidance and suggestions.

[0108] (2) Text-to-text capability (i.e., large text-to-text models, such as Qwen-2.5, GPT-4o-mini, etc.): used for understanding and generating natural language text descriptions.

[0109] (3) Text-to-image capability (i.e., large text-to-image model, such as Stable Diffusion, FLUX.1schnell, Dreambooth, etc.): used to generate visual images and / or videos (such as exercise demonstrations, diet pairing illustrations, etc.) to represent the user's current mood state and health advice information.

[0110] Fine-tuning strategies include:

[0111] SFT (Supervised Fine-Tuning), DPO (Direct Preference Optimization), and LoRA (Low-Rank Adaptation) are some of the techniques used in this study.

[0112] 4. Service layer:

[0113] The service layer, based on a FastAPI-based microservice architecture, achieves loose coupling, flexible scalability, and high availability of business logic. This layer includes:

[0114] User login verification service: Ensures the privacy and security of user data.

[0115] Multimodal data transmission service: responsible for transmitting data from terminals and wearable devices to the backend data storage and algorithm layer for processing.

[0116] Large Model Invocation Service: Provides a unified interface management for the algorithm layer, so that the application layer can dynamically invoke large models.

[0117] Cache optimization service: Utilizes Redis (a data structure storage system) to cache frequently used data and analysis results, accelerating user queries and real-time feedback.

[0118] 5. Application Layer:

[0119] The application layer is the functional module design and interface development layer for users and business scenarios. Through functional module division, interaction style planning, and page element enhancement, it achieves data visualization and intuitive feedback. The application layer presents the analysis results (health advice analysis) to users in an easy-to-understand way.

[0120] The application layer can also display health advice information in multiple dimensions (such as heart rate change charts, comparison of step count and calorie burning rate, and distribution of dietary nutrition), and provide an interactive interface (such as switching data for different time periods, viewing comparisons of past reports, and receiving personalized push messages).

[0121] 6. Access Layer:

[0122] The access layer is responsible for integrating the health monitoring system into user-accessible terminal scenarios, including system-level access (such as access to an operating system) and application-level access (such as access to an application). Through the access layer, users can use the health monitoring system on mobile devices, PCs, and web devices.

[0123] The following is an example description of implementing the disclosed solution based on the aforementioned system architecture.

[0124] 1. Data Acquisition and Transmission (Infrastructure Layer & Access Layer):

[0125] (1) Real-time monitoring of users’ physiological information data (heart rate, sleep, blood oxygen, calories, steps, mood, etc.) through wearable devices such as smartwatches and wristbands, and transmission of the monitored physiological information data to the server (e.g., cloud server) using underlying communication capabilities (such as Bluetooth, Wi-Fi or cellular network).

[0126] (2) Users can upload their basic information and food photos (real photos of breakfast, lunch, and dinner) via smartphones, PCs, or web applications. The application or terminal transmits the user-uploaded data to the server via a network communication link.

[0127] 2. Data Collection and Preprocessing (Data Layer):

[0128] (1) All types of data received (including but not limited to physiological signal data, basic information data, exercise information data, and dietary information data) are archived and classified in the data storage service.

[0129] (2) Deeply analyze the trends and anomalies in various types of data to obtain intelligent insight data.

[0130] (3) Perform preliminary preprocessing on the food images (such as format conversion, clarity enhancement, etc.).

[0131] (4) With the support of the database and cache database, complete the data indexing and query optimization so that the subsequent algorithm layer can quickly access high-quality, integrated data.

[0132] 3. Multimodal analysis and feature extraction (algorithm layer):

[0133] (1) The algorithm layer obtains access to the underlying storage data by calling the service (located in the service layer) through the large model, and obtains the user's multimodal data from the data layer.

[0134] (2) Based on the large-scale graph-text model, image recognition is performed on food images to obtain image recognition results containing various ingredients, and the image information is transformed into feature data that can be described in a structured way.

[0135] (3) Input the multimodal data into the Wenshengwen Big Data Model for comprehensive semantic analysis and inference to generate health advice information for the current user (including but not limited to health status assessment results, dietary guidance advice information, and exercise guidance advice information). If necessary, the Wenshengtu Big Data Model can be called based on the generated health advice information to obtain visualized images and / or videos.

[0136] (4) Use fine-tuning techniques such as SFT, DPO and LoRA to personalize the large model to ensure that the recommendations are more in line with the user’s specific needs and health management goals.

[0137] 4. Business Logic and Data Services (Service Layer):

[0138] (1) The multimodal data transmission service continuously synchronizes data with wearable devices and terminal applications, enabling the latest user health data to be quickly obtained by the algorithm layer.

[0139] (2) User login verification service ensures user privacy and data security. Only verified users can view and obtain health advice information.

[0140] (3) The large model call service is responsible for the unified management of algorithm layer model calls and load balancing, ensuring that the system can still respond quickly under high concurrency access.

[0141] (4) Cache optimization service: Cache frequently queried key data (such as the latest health advice information of users) to reduce redundant calculations and improve response speed.

[0142] 5. Data visualization and interactive presentation (application layer):

[0143] (1) The application layer displays the health advice information generated by the Wenshengwen big data model in the form of charts, text, videos, etc. Users can view the heart rate change curve, step count and calorie consumption comparison chart, and diet nutrition ratio chart in the interface, and can click to view detailed explanations and improvement suggestions.

[0144] (2) The application layer provides an interactive interface that allows users to provide feedback and make fine-tuning of the results (such as marking the reasons for abnormal diet on a certain day, or recording temporary changes in health status). User feedback is sent back to the algorithm layer through the service layer, and the large model will adaptively adjust subsequent suggestions based on the new input.

[0145] 6. Scene Access and Terminal Support (Access Layer):

[0146] (1) Users can choose to access the health monitoring system on smartphones, PCs, web interfaces or wearable devices.

[0147] (2) The access layer provides a unified and standardized interface for different terminals.

[0148] 7. Dynamic optimization and long-term closed loop (continuous iteration):

[0149] (1) The health monitoring system can continuously track changes in user health data, dynamically analyze and update long-term trends, and make recommendations always match the user's actual health status and goals through regular fine-tuning of the large model.

[0150] (2) The health monitoring system can continuously optimize nutrition advice and exercise programs, providing users with more accurate and efficient health management strategies, and ultimately forming a long-term closed-loop management system of health monitoring-assessment-intervention-feedback.

[0151] Having understood the system architecture of the health monitoring system provided in this disclosure as described above, the health information generation method of the embodiments of this disclosure will now be described in detail with reference to the accompanying drawings.

[0152] Figure 2This is a flowchart illustrating a method for generating health information according to an exemplary embodiment of this disclosure. Figure 2 As shown, the exemplary embodiment method may include the following steps:

[0153] In step 201, the user's multimodal health data is obtained.

[0154] In practical applications, when a user logs into the aforementioned health monitoring system using their account and password, the system can determine whether the user is already registered. If so, the system loads the user's multimodal health data; otherwise, it redirects the user to the registration page, where the user completes the registration process. The system then obtains the user's input and / or the multimodal health data monitored by the user's wearable devices, optical heart rate monitors, smart scales, etc.

[0155] In one optional example, after a user successfully logs into the health monitoring system, the system can output a query message asking the user what type of health advice information they need. The types of health advice information include, but are not limited to: health status assessment results, dietary guidance advice, and exercise guidance advice.

[0156] After receiving the type of health advice information selected by the user based on the aforementioned inquiry message, the user's health data required for that type of health advice information can be obtained.

[0157] For example, when a user selects a health status assessment result, the acquired multimodal health data should include: basic information data and physiological information data. The basic information data characterizes the user's basic personal characteristics, including but not limited to the user's age, gender, height, and weight. The physiological information data characterizes the user's physiological state, including but not limited to the user's heart rate, sleep, blood oxygen, calories, steps, and mood.

[0158] When a user selects dietary guidance and recommendations, the multimodal health data acquired should include: basic information data, physiological information data, and dietary information data. Dietary information data is used to characterize the user's nutritional intake; for example, it could include data on the user's three meals consumed over the past week or three days.

[0159] When a user selects exercise guidance suggestions, the multimodal health data acquired should include: basic information data, physiological information data, and exercise information data. Among these, exercise information data characterizes the user's exercise activity. For example, it could include the user's exercise data over the past month.

[0160] It should be noted that users may also select at least two types of health advice information simultaneously, and this disclosure does not limit this. For example, a user may select both the aforementioned dietary guidance advice information and exercise guidance advice information. In this case, the acquired multimodal health data should include: basic information data, physiological information data, dietary information data, and exercise information data.

[0161] In step 202, health advice information for the user is determined based on the multimodal health data.

[0162] In practical applications, after obtaining a user's multimodal health data, health recommendations for the current user can be determined based on rule engines or preset models, and this disclosure does not limit this. For ease of description, the following embodiments will be described using the example of determining health recommendations for the current user based on a preset model, such as a large model.

[0163] In conjunction with the above, when a user selects a health status assessment result, the basic information data and physiological information data of the user can be input into a pre-trained large model to obtain the health status assessment result of the current user output by the large model, which can be a health status score. The higher the score, the healthier the user is.

[0164] When a user selects dietary guidance suggestions, the acquired basic user information, physiological information, and dietary information can be input into a pre-trained large model to obtain dietary guidance suggestions tailored to the current user. Specifically, this could include suggestions on the types and quantities of food to consume at each meal. Alternatively, it could provide suggestions for improvement based on the user's current dietary habits, such as reducing carbohydrate intake. This disclosure does not limit the specific content of the dietary guidance suggestions.

[0165] When a user selects exercise guidance suggestions, the acquired basic user information, physiological information, and exercise information can be input into a pre-trained large model to obtain exercise guidance suggestions tailored to the current user. Specifically, this may include at least one of the following: suggested exercise frequency, suggested exercise type, duration of each exercise session, and the standard movements corresponding to the suggested exercise type. Alternatively, it may provide suggestions for improvement based on the user's current exercise habits, such as shortening exercise duration. This disclosure does not limit the specific content of the exercise guidance suggestions.

[0166] In step 203, the health advice information is output.

[0167] The health information generation method provided in this disclosure can generate health advice information for the current user based on real-time acquired multimodal health data using a large model. By using a large model to achieve the fusion analysis of user health data from different modalities, it overcomes the limitations of processing only a single indicator in related technical solutions, and can provide users with more real-time, scientific, and comprehensive health advice information. At the same time, the application of the large model can also significantly improve the accuracy of the health advice information.

[0168] In one embodiment, the aforementioned step 201 may specifically include: based on the receiving time of the user's health management request, obtaining historical multimodal health data within a preset time period prior to the receiving time.

[0169] In summary, based on the foregoing, corresponding multimodal health data can be obtained according to the type of health advice information the user selects. In this example, the time when the user selects the type of health advice information can be used as the time when the user's health management request is received. Then, historical multimodal health data within a preset time period prior to that receiving time, such as one week or three days, can be obtained to generate health advice information based on this historical multimodal health data.

[0170] Optionally, historical multimodal health data can be retrieved at predetermined intervals, and new health recommendations can be generated based on the retrieved historical multimodal health data, thereby ensuring that users can receive continuous health guidance and improvement directions, and realize a closed loop of long-term health management.

[0171] The health information generation method provided in this disclosure can generate corresponding health advice information based on historical multimodal health data within a preset time period prior to the time of receiving a user's health management request. This allows for a more accurate understanding of the user's health status and trends, thereby providing more personalized and targeted health advice.

[0172] In one embodiment, step 202 may specifically include: inputting the multimodal health data into the Wenshengwenda model to obtain health advice information for the user in natural language text description output by the Wenshengwenda model.

[0173] In the health information generation method provided in this embodiment, the acquired multimodal health data of the user can be input into a pre-trained text-based big data model to obtain health advice information for the current user in natural language text output by the text-based big data model, thereby improving the user's understanding and acceptance and enhancing user stickiness.

[0174] In one embodiment, step 203 may include at least one of the following:

[0175] 1. Output health advice information for the user in natural language text description.

[0176] 2. Output the health advice information described in the natural language text via voice.

[0177] 3. Based on the health advice information described in the natural language text, generate visualized images and / or videos using the text-to-image model, and output the visualized images and / or videos.

[0178] For example, when the health advice information is dietary guidance for the user, a large-scale image model can be used to output images containing the recommended food types and corresponding portion sizes for each meal. Similarly, when the health advice information is exercise guidance for the user, a large-scale image model can be used to output images or videos containing the standard movements corresponding to the recommended exercise types. Alternatively, the standard movements corresponding to the recommended exercise types can be output using AR (Augmented Reality) technology, allowing users to view the standard movements by wearing AR glasses. This disclosure does not limit the output format of the health advice information.

[0179] The health information generation method provided in this disclosure can output health advice information through natural language text description, voice, or visual images and / or videos, which can improve users' understanding of the health advice information, thereby enhancing users' ability to implement the health advice information and improving the user experience.

[0180] In one embodiment, such as Figure 3 As shown, the aforementioned preset model can be trained in the following way:

[0181] In step 301, training data is acquired.

[0182] The training data includes at least one set of multimodal health data labeled with health advice information.

[0183] In step 302, the first multimodal health data in the training data is input into the preset model to be trained to obtain the predicted health suggestion information output by the preset model to be trained, which corresponds to the first multimodal health data.

[0184] In step 303, the network parameters of the preset model are adjusted based on the difference between the predicted health advice information and the health advice information in the training data corresponding to the first multimodal health data.

[0185] It should be understood that when the preset model to be trained is a model used to output health status assessment results, the training data should include at least one set of multimodal health data labeled with health status assessment results. This multimodal health data includes basic information data and physiological information data. When the preset model to be trained is a Wenshengwenda model used to output dietary guidance and recommendations, the training data should include at least one set of multimodal health data labeled with dietary guidance and recommendations. This multimodal health data includes basic information data, physiological information data, and dietary information data. And when the preset model to be trained is a Wenshengwenda model used to output exercise guidance and recommendations, the training data should include at least one set of multimodal health data labeled with exercise guidance and recommendations. This multimodal health data includes basic information data, physiological information data, and exercise information data.

[0186] This method allows for the training of a pre-defined model that generates health recommendations based on users' multimodal health data. This model can learn the characteristics of different users under various health conditions, thereby providing each user with more personalized and accurate health recommendations.

[0187] In one embodiment, health recommendations can also be output based on the user's multimodal health data to help achieve the user's health management goals.

[0188] At this point, the acquired multimodal health data should include not only the user's basic information and physiological information, but also the user's health management goals, such as fat loss and muscle gain.

[0189] Furthermore, the user's basic information data, physiological information data, and health management goals can be input into a pre-trained large model to obtain health advice information output by the large model to achieve the user's set health management goals.

[0190] For example, the output health advice information may include: the types of food and corresponding portion sizes recommended for each meal. Alternatively, it may include recommended types of exercise, exercise intensity, and exercise duration.

[0191] The health information generation method provided in this embodiment can generate health advice information to achieve the user's health management goals based on the obtained user's health management goals through a large model, which can improve the user's experience.

[0192] In one embodiment, such as Figure 4 As shown, the user's dietary information data can be obtained through the following steps:

[0193] In step 401, the user uploads images of food.

[0194] As mentioned earlier, users can upload images of their meals, including but not limited to photos of their three meals a day, via smartphones, PCs, or web applications.

[0195] In step 402, the food image is subjected to image recognition based on an image recognition model to obtain the image recognition result.

[0196] After receiving the food image uploaded by the user, the image can be recognized based on a pre-trained image recognition model (i.e., a graph-based image model) to obtain the image recognition result. For example, the image recognition result may include the names of the ingredients contained in the food image.

[0197] In step 403, the dietary information data is determined based on the image recognition results.

[0198] After obtaining the image recognition results by performing image recognition on food images based on image recognition models, the nutritional information of the ingredients contained in the current image recognition results can be determined based on the nutritional structure database, and the nutritional information of the ingredients contained in the image recognition results can be identified as the food information data corresponding to the current food image.

[0199] Optionally, since image recognition models may contain errors when recognizing food images, in this example, the user can correct the image recognition result after it is obtained. After receiving the correction information from the user, the image recognition result can be updated based on this information, thereby improving the accuracy of the image recognition result and further enhancing the accuracy of subsequent dietary guidance suggestions based on the image recognition result.

[0200] In the health information generation method provided in this embodiment, an image recognition model, namely the aforementioned image-to-text model, can be used to perform image recognition on a food image to obtain the ingredients contained in the food image, and further determine the nutritional information corresponding to the ingredients. That is, the image information is captured and converted into nutritional information that can be described in a structured way, so as to provide dietary guidance and suggestions for the current user based on the nutritional information.

[0201] The multimodal health data of users involved in this disclosure (including but not limited to basic information data and physiological information data) are all data authorized by the users or fully authorized by all parties. The collection, use and processing of the relevant data shall comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation portals shall be provided for users to choose to authorize or refuse.

[0202] For the foregoing method embodiments, in order to simplify the description, they are all described as a series of actions. However, those skilled in the art should know that this disclosure is not limited to the described order of actions, because according to this disclosure, some steps may be performed in other orders or simultaneously.

[0203] Figure 5 This is a schematic diagram of the structure of an electronic device according to an exemplary embodiment of the present disclosure, such as... Figure 5 As shown, the electronic device may include:

[0204] The data acquisition module 51 is used to acquire the user's multimodal health data, which includes at least: basic information data and physiological information data. The basic information data is used to characterize the user's basic personal characteristics, and the physiological information data is used to characterize the user's physiological state.

[0205] The information generation module 52 is used to determine the user's health recommendation information based on the multimodal health data.

[0206] The information output module 53 is used to output the health advice information.

[0207] Optionally, the multimodal health data further includes at least one of the following:

[0208] The user's dietary information data is used to characterize the user's nutritional intake.

[0209] The user's motion information data is used to characterize the user's motion status.

[0210] The health advice information includes at least one of the following:

[0211] The user's health status assessment results.

[0212] The dietary guidance and advice information for the user.

[0213] The user's exercise guidance and suggestions information.

[0214] Optionally, when the multimodal health data includes the dietary information data, the data acquisition module 51, when acquiring the user's multimodal health data, includes:

[0215] Receive food images uploaded by users.

[0216] The food images are then subjected to image recognition based on an image recognition model to obtain the image recognition results.

[0217] The dietary information data is determined based on the image recognition results.

[0218] Optional, in Figure 5 Based on the modules shown, the electronic device may further include:

[0219] The correction information receiving module is used to receive correction information obtained by the user in correcting the image recognition results.

[0220] The image recognition result update module is used to update the image recognition result based on the correction information.

[0221] Optionally, the information generation module 52, when determining the user's health recommendation information based on the multimodal health data, includes:

[0222] The multimodal health data is input into a preset model to obtain health advice information for the user output by the preset model.

[0223] Optionally, the preset model is trained based on a preset model training device, which includes:

[0224] The training data acquisition module is used to acquire training data, which includes at least one set of multimodal health data labeled with health advice information.

[0225] The prediction result acquisition module is used to input the first multimodal health data in the training data into the preset model to be trained, and obtain the predicted health suggestion information output by the preset model to be trained, which corresponds to the first multimodal health data.

[0226] The network parameter adjustment module is used to adjust the network parameters of the preset model based on the difference between the predicted health advice information and the health advice information in the training data corresponding to the first multimodal health data.

[0227] Optionally, the information output module 53, when used to output the health advice information, includes at least one of the following:

[0228] Output health advice information for the user, described in natural language text.

[0229] The health advice information described in the natural language text is output via voice.

[0230] Based on the health advice information described in the natural language text, a text-to-image model is used to generate visualized images and / or videos, and the visualized images and / or videos are output.

[0231] Optionally, the multimodal data may also include: the user's health management goals.

[0232] The information generation module 52, when used to determine the user's health recommendation information based on the multimodal health data, includes:

[0233] The basic information data, physiological information data, and health management goals are input into the large model to obtain health recommendation information output by the large model to achieve the health management goals.

[0234] Optionally, the data acquisition module 51, when acquiring the user's multimodal health data, includes:

[0235] Based on the time of receiving a user's health management request, historical multimodal health data within a preset time period prior to the receiving time is obtained.

[0236] For the apparatus embodiment, since it basically corresponds to the method embodiment, the relevant parts can be referred to in the description of the method embodiment.

[0237] Figure 6 This is a schematic diagram illustrating the structure of another electronic device 600 according to an exemplary embodiment of this disclosure. The electronic device may be a terminal device (e.g., a smartphone) or a server device.

[0238] Reference Figure 6 The electronic device 600 may include one or more of the following components: a processing component 602, a memory 604, a power supply component 606, a multimedia component 608, an audio component 610, an input / output (I / O) interface 612, a sensor component 614, and a communication component 616.

[0239] Processing component 602 typically controls the overall operation of electronic device 600, such as operations associated with display, telephone calls, data communication, camera operation, and recording operations. Processing component 602 may include one or more processors 620 to execute instructions to perform all or part of the steps of the methods described above. Furthermore, processing component 602 may include one or more modules to facilitate interaction between processing component 602 and other components. For example, processing component 602 may include a multimedia module to facilitate interaction between multimedia component 608 and processing component 602.

[0240] Memory 604 is configured to store various types of data to support the operation of device 600. Examples of this data include instructions for any application or method operating on electronic device 600, contact data, phonebook data, messages, pictures, videos, etc. Memory 604 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0241] Power supply component 606 provides power to various components of electronic device 600. Power supply component 606 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to electronic device 600.

[0242] Multimedia component 608 includes a screen that provides an output interface between the aforementioned electronic device 600 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touchscreen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors may sense not only the boundaries of touch or swipe actions but also the duration and pressure associated with the touch or swipe operation. In some embodiments, multimedia component 608 includes a front-facing camera and / or a rear-facing camera. When the electronic device 600 is in an operating mode, such as a shooting mode or a video mode, the front-facing camera and / or the rear-facing camera may receive external multimedia data. Each front-facing camera and rear-facing camera may be a fixed optical lens system or have focal length and optical zoom capabilities.

[0243] Audio component 610 is configured to output and / or input audio signals. For example, audio component 610 includes a microphone (MIC) configured to receive external audio signals when electronic device 600 is in an operating mode, such as call mode, recording mode, and voice recognition mode. The received audio signals may be further stored in memory 604 or transmitted via communication component 616. In some embodiments, audio component 610 also includes a speaker for outputting audio signals.

[0244] I / O interface 612 provides an interface between processing component 602 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to, home buttons, volume buttons, power buttons, and lock buttons.

[0245] Sensor assembly 614 includes one or more sensors for providing state assessments of various aspects of electronic device 600. For example, sensor assembly 614 can detect the on / off state of electronic device 600, the relative positioning of components such as the display and keypad of electronic device 600, changes in position of electronic device 600 or a component of electronic device 600, the presence or absence of user contact with electronic device 600, orientation or acceleration / deceleration of electronic device 600, and temperature changes of electronic device 600. Sensor assembly 614 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. Sensor assembly 614 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, sensor assembly 614 may also include an accelerometer, gyroscope, magnetometer, pressure sensor, or temperature sensor.

[0246] Communication component 616 is configured to facilitate wired or wireless communication between electronic device 600 and other devices. Electronic device 600 can access wireless networks based on communication standards, such as WiFi, 4G or 5G, 4G LTE, 5G NR, or combinations thereof. In one exemplary embodiment, communication component 616 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, the aforementioned communication component 616 also includes a near-field communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on radio frequency identification (RFID) technology, Infrared Data Association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0247] In an exemplary embodiment, the electronic device 600 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the methods described above.

[0248] In an exemplary embodiment, a non-transitory computer-readable storage medium is also provided, such as a memory 604 including instructions, which, when executed by a processor 620 of an electronic device 600, enables the electronic device 600 to perform the health information generation method of any embodiment of this disclosure.

[0249] The non-transitory computer-readable storage medium may be ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc.

[0250] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. A method for generating health information, characterized in that, The method includes: Acquire the user's multimodal health data, which includes at least: basic information data and physiological information data, wherein the basic information data is used to characterize the user's basic personal characteristics and the physiological information data is used to characterize the user's physiological state; Based on the multimodal health data, determine the user's health recommendation information; Output the aforementioned health advice information.

2. The method according to claim 1, characterized in that, The multimodal health data also includes at least one of the following: The user's dietary information data is used to characterize the user's nutritional intake; The user's motion information data, which is used to characterize the user's motion status; The health advice information includes at least one of the following: The user's health status assessment results; The dietary guidance and advice information for the user; The user's exercise guidance and suggestions information.

3. The method according to claim 2, characterized in that, When the multimodal health data includes the dietary information data, acquiring the user's multimodal health data includes: Receive food images uploaded by users; The food images are then subjected to image recognition based on an image recognition model to obtain the image recognition results. The dietary information data is determined based on the image recognition results.

4. The method according to claim 3, characterized in that, After obtaining the image recognition result and before determining the dietary information data, the method further includes: Receive correction information obtained by the user to correct the image recognition result; The image recognition result is updated based on the correction information.

5. The method according to claim 1, characterized in that, The step of determining the user's health advice information based on the multimodal health data includes: The multimodal health data is input into a preset model to obtain health advice information for the user output by the preset model.

6. The method according to claim 5, characterized in that, The preset model is trained in the following manner: Acquire training data, which includes at least one set of multimodal health data labeled with health advice information; The first multimodal health data in the training data is input into the preset model to be trained, and the predicted health suggestion information output by the preset model to be trained, corresponding to the first multimodal health data, is obtained. The network parameters of the preset model are adjusted based on the difference between the predicted health advice information and the health advice information in the training data corresponding to the first multimodal health data.

7. The method according to claim 1, characterized in that, The output of the health advice information includes at least one of the following: Output health advice information for the user in natural language text description; The health advice information described in the natural language text will be output via voice. Based on the health advice information described in the natural language text, a text-to-image model is used to generate visualized images and / or videos, and the visualized images and / or videos are output.

8. The method according to claim 1, characterized in that, The multimodal data also includes: the user's health management goals; The step of determining the user's health advice information based on the multimodal health data includes: The basic information data, physiological information data, and health management goals are input into the large model to obtain health recommendation information output by the large model to achieve the health management goals.

9. The method according to claim 1, characterized in that, The acquisition of the user's multimodal health data includes: Based on the time of receiving a user's health management request, historical multimodal health data within a preset time period prior to the receiving time is obtained.

10. An electronic device, characterized in that, include: The data acquisition module is used to acquire the user's multimodal health data, which includes at least: basic information data and physiological information data. The basic information data is used to characterize the user's basic personal characteristics, and the physiological information data is used to characterize the user's physiological state. An information generation module is used to determine health recommendation information for the user based on the multimodal health data; The information output module is used to output the health advice information.

11. A computer-readable storage medium having a computer program stored thereon that, when executed by a processor, implements the steps of the method of any one of claims 1-9.

12. An electronic device, comprising: processor; Memory used to store processor-executable instructions; The processor is configured to perform the steps of the method according to any one of claims 1-9.

Images