Personalized sleep intervention method and system based on large language model and multi-modal dynamic knowledge graph

Abstract

The invention discloses a personalized sleep intervention method and system based on a large language model and a multi-modal dynamic knowledge graph. The method comprises the following steps: (S1) collecting multi-modal data including physiological data, environmental data and user logs; (S2) on the basis of the data, utilizing a dynamic relation mining and conflict detection mechanism to update a multi-modal dynamic knowledge graph in real time; (S3) extracting knowledge sub-graphs related to the current user state from the knowledge graph; (S4) inputting the knowledge sub-graph and the real-time data into an LLM reasoning layer through a hierarchical prompt project, and generating personalized intervention suggestions; and (S5) converting the suggestion into an equipment control instruction through an instruction conversion engine and an MQTT protocol, and driving the smart home equipment to execute intervention. After execution, the system collects an intervention effect and reversely optimizes LLM reasoning. According to the method, the problems that LLM knowledge is lagged, interpretability is lacked and intervention cannot be closed-loop are solved, and the accuracy of sleep intervention and user compliance are remarkably improved.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence medical and health technology, and in particular to a personalized sleep intervention method and system based on a large language model and a multimodal dynamic knowledge graph. Background Technology

[0002] Sleep, as a core physiological activity, is directly related to an individual's cognitive function, mood regulation, metabolism, and even cardiovascular health. With the accelerating pace of modern life, sleep disorders have become increasingly prominent, posing a global public health challenge. Therefore, developing precise and personalized sleep intervention technologies has significant social and clinical value.

[0003] In recent years, the rapid development of artificial intelligence (AI) technology, especially large language models (LLM), has brought new opportunities to the healthcare field. With its powerful natural language understanding, contextual reasoning, and knowledge integration capabilities, LLM has begun to be applied to complex health data analysis. In the field of sleep medicine, researchers are attempting to use LLM to integrate and analyze multimodal information, including electroencephalogram (EEG), sleep logs, wearable device data, and environmental sensor data, in order to provide more accurate sleep quality assessments and intervention recommendations.

[0004] However, existing LLM-based sleep intervention technologies still face four major technical shortcomings in practical applications:

[0005] LLM's dynamic fusion with multimodal data is insufficient: Most studies only use LLM as an "analysis tool" for static data. For example, some schemes use GPT-4 to perform zero-shot learning on EEG images and sleep self-report texts, achieving a certain prediction accuracy, but these data are usually retrospective or non-real-time. LLM cannot capture the real-time dynamic correlation between physiological signals (such as changes in EEG waveforms) and environmental factors (such as sudden changes in nighttime noise) during sleep, resulting in a lack of context-adaptability in its analysis results.

[0006] LLM reasoning lacks dynamic knowledge support: LLM's reasoning ability largely relies on its pre-training data or external static knowledge bases. Pre-training data suffers from knowledge lag and cannot reflect an individual's latest situation; while static knowledge graphs (such as those built based on expert annotations) cannot be updated in real time. When new sleep-influencing factors (such as developing a new habit of using electronic devices before bed) establish a new causal relationship between a user's sleep quality (such as delayed sleep onset), LLM cannot incorporate this dynamic knowledge into its reasoning in a timely manner, leading to delayed or ineffective intervention recommendations.

[0007] The imbalance between interpretability and personalization in LLM output: The "black box nature" of LLM often results in suggestions lacking a clear "causal chain" explanation. For example, an LLM might only suggest "adjusting the temperature" without explaining the specific relationship between that temperature and the user's current EEG waveform (such as delta wave power). Furthermore, the suggestions lack personalization, failing to adequately consider the individual's real-time sleep stage (e.g., REM vs. N3) and the effects of past interventions (e.g., a user's sensitivity to "white noise" but adaptation to "earplugs"), leading to low user compliance.

[0008] LLM fails to form a closed loop of "analysis-intervention": In existing technical solutions, LLM mostly remains at the "recommendation generation" stage. For example, some health management models based on Gemini, while integrating wearable data, require users to manually operate smart home devices or wear earplugs for their recommendations (such as "reducing noise"). This disconnect between "analysis" and "intervention" prevents recommendations from being implemented in real time, weakening the immediacy and effectiveness of interventions, and failing to form a complete closed loop of "perception-analysis-intervention-feedback".

[0009] Therefore, how to solve the problems of outdated LLM knowledge, insufficient dynamic integration, poor interpretability, and inability to implement closed-loop intervention are the technical bottlenecks that urgently need to be overcome in the field of personalized sleep intervention. Summary of the Invention

[0010] (I) Purpose of the Invention

[0011] The purpose of this invention is to overcome the shortcomings of existing technologies, such as insufficient dynamic fusion of LLM and multimodal data, lack of dynamic knowledge support for reasoning, poor interpretability of output, and failure to form an "analysis-intervention" closed loop, and to provide a personalized sleep intervention method and system based on a large language model and a multimodal dynamic knowledge graph.

[0012] This invention aims to achieve deep and dynamic integration of LLM (Low-Modal Memory Management) with multimodal data such as EEG and environmental data by constructing a technical system of "real-time multimodal data acquisition - dynamic knowledge graph update - LLM-enhanced dynamic reasoning - smart home closed-loop intervention". Through a real-time updated dynamic knowledge graph, it provides immediate and personalized knowledge support for LLM. Layered prompting engineering and interpretability enhancement technologies improve the accuracy of LLM reasoning and user compliance. Furthermore, through real-time linkage with smart homes, it transforms LLM analysis suggestions into closed-loop intervention actions, ultimately achieving intelligent sleep health management encompassing "perception-analysis-intervention-feedback".

[0013] (II) Technical Solution

[0014] To achieve the above objectives, this invention provides a personalized sleep intervention method based on a large language model and a multimodal dynamic knowledge graph, characterized by comprising:

[0015] (S1) Collect multimodal data, including physiological data, environmental data and user log data;

[0016] (S2) Based on the collected multimodal data, update a multimodal dynamic knowledge graph in real time;

[0017] (S3) When the intervention condition is triggered, extract the knowledge subgraph related to the current user state from the dynamic knowledge graph;

[0018] (S4) The knowledge subgraph and the real-time collected multimodal data are used as inputs to call the large language model inference layer to generate personalized intervention suggestions;

[0019] (S5) The personalized intervention suggestion is converted into a device control command and sent to the smart home device in the intervention execution layer to perform the intervention.

[0020] In a preferred embodiment, the step of updating the multimodal dynamic knowledge graph in real time includes:

[0021] The multimodal data is received through a real-time data access module;

[0022] The dynamic relationship mining module mines association rules between nodes based on multimodal data within a preset time window and calculates the dynamic weights of the association rules.

[0023] When newly discovered association rules conflict with existing relationships in the knowledge graph, the relationship with the higher preset probability is retained through conflict detection and version control modules.

[0024] In a preferred embodiment, the dynamic weight calculation combines the support, confidence, and time decay factor of the association rule.

[0025] In a preferred embodiment, the step of invoking the large language model inference layer includes:

[0026] The knowledge subgraph is converted into a knowledge layer hint described in natural language using the subgraph extraction API;

[0027] The knowledge layer prompts, the data layer prompts generated based on the multimodal data, and the preset task layer prompts are combined into a layered prompt engineering template and input into the medical fine-tuning large language model.

[0028] In a preferred embodiment, the personalized intervention recommendation includes interpretability information;

[0029] The interpretability information includes: a "cause-recommendation-expected effect" causal chain generated based on the knowledge subgraph, and a SHAP value that quantifies the contribution of each modality of data to the intervention recommendation.

[0030] In a preferred embodiment, the method further includes:

[0031] (S6) After the smart home device performs the intervention, multimodal data after the intervention is collected in real time to analyze the intervention effect;

[0032] (S7) If the intervention effect does not achieve the preset target, the intervention effect is used as a new input, and the large language model inference layer is called again to generate a new personalized intervention suggestion.

[0033] In a preferred embodiment, the step of converting the personalized intervention suggestion into device control commands includes:

[0034] The natural language in the intervention recommendations is parsed into standardized device commands using an instruction conversion engine.

[0035] The standardized device commands are sent to the smart home devices via the MQTT Internet of Things protocol.

[0036] In a preferred embodiment, the physiological data is electroencephalogram (EEG) data, electrooculogram (EOG) data, or respiratory data;

[0037] The method further includes a data preprocessing step before acquiring multimodal data:

[0038] The EEG data is cleaned and filtered, and the power ratio of the EEG data in the preset sleep frequency band is calculated by Fast Fourier Transform (FFT) to determine the real-time sleep stage.

[0039] In a preferred embodiment, according to claim 1, the multimodal dynamic knowledge graph comprises:

[0040] The nodes include sleep state nodes, physiological characteristic nodes, environmental factor nodes, behavioral factor nodes, and intervention measure nodes;

[0041] An edge represents a dynamic causal relationship between nodes and carries a dynamic weight and a timestamp.

[0042] This invention also provides another technical solution: a personalized sleep intervention system based on a large language model and a multimodal dynamic knowledge graph, characterized in that it includes:

[0043] A multimodal data layer (125) includes a data preprocessing module (127) for collecting multimodal data, which includes physiological data, environmental data and user log data;

[0044] A multimodal dynamic knowledge graph layer (122) is connected to the multimodal data layer (125) and includes a dynamic relationship mining module (123) and a conflict detection and version control module (123) for updating a multimodal dynamic knowledge graph in real time based on the collected multimodal data;

[0045] A large language model reasoning layer (118) is connected to the dynamic knowledge graph layer (122), including a subgraph extraction API (120) and a medical fine-tuning large language model (121), which is used to extract knowledge subgraphs (120) from the dynamic knowledge graph when intervention conditions are triggered, and use the knowledge subgraphs (120) and the real-time collected multimodal data as input to generate personalized intervention suggestions;

[0046] An intervention execution layer (116), connected to the large language model inference layer (118), includes an instruction conversion engine (117) and an MQTT Internet of Things protocol (117) for converting the personalized intervention suggestions into device control instructions and sending them to smart home devices (117) to execute the intervention.

[0047] (III) Beneficial Effects

[0048] Compared with the prior art, the present invention, through the above technical solution, has at least the following beneficial effects:

[0049] Significantly improves the accuracy of LLM inference: This invention provides LLM with real-time updated and personalized causal relationship knowledge (e.g., "Bedroom temperature > 25℃ → This user's N3 phase percentage decreases") by constructing a multimodal dynamic knowledge graph layer. LLM inference no longer relies solely on lagging pre-training data, but combines real-time EEG features with dynamic knowledge graph subgraphs, increasing its accuracy in analyzing factors affecting sleep quality (e.g., from less than 60% in existing technologies to over 85%). This solves the problems of existing technologies where LLM cannot capture dynamic changes and provides lagging suggestions.

[0050] Significantly Enhances the Interpretability and User Compliance of LLM Output: This invention, through a specific hierarchical suggestion engineering, requires LLM to include a causal chain (referencing a knowledge graph) and SHAP value feature contribution analysis when generating suggestions. This makes the "black box" decision-making process of LLM transparent and interpretable. Users can clearly understand the basis of the suggestions (e.g., "insufficient deep sleep due to excessively high temperature"), thereby greatly improving trust and compliance with the suggestions (e.g., increasing from less than 50% in existing technologies to over 90%).

[0051] This invention achieves a closed loop of "LLM analysis-intervention," enhancing the practicality of interventions: Through the intervention execution layer, utilizing an instruction conversion engine and the MQTT protocol, it transforms the natural language suggestions generated by LLM into automatic control commands for smart home devices (such as air conditioners and white noise generators) in real time. This solves the core deficiency of existing technologies where LLM suggestions cannot be implemented, realizing a complete closed loop of "perception-analysis-automatic intervention." Tests show that this closed-loop system can significantly increase the proportion of users in deep sleep (an average increase of 18%-25%), far exceeding existing technologies (5%-10%).

[0052] To ensure the dynamic updating of LLM knowledge and maintain stable long-term effects, this invention designs a dynamic knowledge graph with minute-level update capabilities and a version control mechanism. When a user's behavioral patterns or physiological responses change (such as developing a new habit of drinking coffee before bed), the system can promptly capture new causal relationships (such as "drinking coffee before bed → delayed sleep onset") through dynamic relationship mining and update the LLM knowledge base. This solves the problem of lagging LLM knowledge and resulting in diminishing long-term intervention effects in existing technologies, ensuring that the intervention effect remains stable even after several months of continuous use. Attached Figure Description

[0053] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0054] Figure 1 This is a flowchart of a personalized sleep intervention method provided in one embodiment of the present invention.

[0055] Figure 2 This is an architectural block diagram of a personalized sleep intervention system provided in one embodiment of the present invention.

[0056] Explanation of markings in the diagram:

[0057] 116: Intervention Execution Level

[0058] 117: Instruction translation engine / MQTT IoT protocol / Smart home devices

[0059] 118: Large Language Model Inference Layer

[0060] 120: Subgraph Extraction API / Knowledge Subgraph

[0061] 121: Medical Fine-tuning of Large Language Models / Interpretable Information

[0062] 122: Multimodal Dynamic Knowledge Graph Layer / Multimodal Dynamic Knowledge Graph

[0063] 123: Dynamic Relationship Mining Module / Conflict Detection and Version Control Module

[0064] 124: Nodes / Edges

[0065] 125: Multimodal Data Layer

[0066] 127: Data Preprocessing Module Detailed Implementation

[0067] To make the objectives, technical solutions, and advantages of this invention clearer, the grounding protection system of this invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0068] In the description of this invention, unless otherwise stated, "a plurality of" means two or more; the terms "center," "longitudinal," "lateral," "upper," "lower," "left," "right," "inner," "outer," "front end," "rear end," "head," "tail," "vertical," "horizontal," "top," "bottom," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings, and are only for the convenience of describing the invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of the invention. Furthermore, the terms "first," "second," "third," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0069] In the description of this utility model, it should be noted that, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "joining" should be interpreted broadly. For example, they can refer to fixed connections, detachable connections, or integral connections; they can refer to mechanical connections or electrical connections; they can refer to direct connections or indirect connections through an intermediate medium; and they can refer to the internal communication between two components. Those skilled in the art will be able to understand the specific meaning of the above terms in this utility model based on the specific circumstances.

[0070] The technical solutions of this solution will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this solution, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this solution without creative effort are within the scope of protection of this solution.

[0071] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0072] It should be understood that the terms "comprising," "including," or any other variations thereof as used in this invention are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of additional identical elements in the process, method, article, or apparatus that includes said element.

[0073] The core idea of ​​this invention is to construct a four-layer collaborative closed-loop system. The "multimodal dynamic knowledge graph layer" solves the knowledge lag and dynamic perception defects of the "large language model reasoning layer", and the "intervention execution layer" implements the intelligent analysis of LLM into real-time intervention.

[0074] Reference Figure 2 This invention provides a personalized sleep intervention system based on a large language model and a multimodal dynamic knowledge graph. This system can achieve... Figure 1 The method is shown. The system specifically includes: a multimodal data layer (125), a multimodal dynamic knowledge graph layer (122), a large language model reasoning layer (118), and an intervention execution layer (116).

[0075] The following will be strictly combined Figure 1 The method flow shown and Figure 2 The system architecture shown illustrates the technical solution of this invention in detail.

[0076] First layer: Multimodal data layer (125)

[0077] The starting point of this invention is to perceive the user's physiological state and environment in real time and in multiple dimensions. The multimodal data layer (125) is responsible for collecting multimodal data in step (S1) of the method flow.

[0078] 1.1 Data Acquisition (S1)

[0079] This step aims to comprehensively cover the core factors that affect sleep.

[0080] Physiological data (weight 8):

[0081] Electroencephalography (EEG) data: As the "gold standard" for assessing sleep stages, this embodiment uses a portable EEG device (such as the Emotiv EPOC or a similar device) for acquisition. Emphasis is placed on acquiring data from core sleep channels (such as C3-A2 and C4-A1) to ensure signal quality while also considering user comfort. The sampling frequency is set to 256Hz to capture key sleep-related frequency bands (such as 0.5-30Hz).

[0082] Electrooculography (EOG) data: collected using a multi-parameter sleep monitor at a sampling frequency of 128 Hz. EOG data is primarily used to assist in verifying the rapid eye movement (REM) sleep phase.

[0083] Respiratory data: Respiratory rate and depth are collected by a monitor at a sampling frequency of 1 Hz. This data is used to help eliminate interference from sleep structure caused by events such as sleep apnea.

[0084] Environmental data:

[0085] Internet of Things (IoT) sensors are deployed in users' bedrooms to collect key environmental parameters in real time.

[0086] Temperature: The acquisition range is 10-40℃, and the accuracy requirement is ±0.5℃.

[0087] Noise: Acquisition range 30-100dB, accuracy requirement ±1dB.

[0088] Illumination: Acquisition range 0-1000 lux, accuracy requirement ±10 lux.

[0089] The environmental data mentioned above are collected and reported at a low frequency (e.g., once per minute) to reflect steady-state changes in the environment.

[0090] User log data:

[0091] Collect users' subjective feelings and objective behaviors through smartphone apps or voice input.

[0092] Subjective indicators: such as time to fall asleep, number of times you wake up during the night, and sleep quality score upon waking up in the morning (e.g., 1-5 points).

[0093] Objective behaviors: such as whether electronic devices are used before bed, caffeine intake, daytime naps, etc.

[0094] 1.2 Data Preprocessing

[0095] The raw data contains noise and has inconsistent formats, making it unsuitable for direct use by knowledge graphs or LLMs. The data preprocessing module (127) in the multimodal data layer (125) is responsible for cleaning, standardizing, and extracting features from the collected raw data.

[0096] EEG data preprocessing:

[0097] Signal cleaning and filtering: First, a 50Hz notch filter is used to remove power frequency interference; second, a 0.5-30Hz bandpass filter is used to retain the core frequency bands (δ, θ, α, β waves) related to sleep, while filtering out high-frequency noise and low-frequency drift.

[0098] Frequency domain feature calculation (FFT): The cleaned EEG time-domain signal is segmented into segments within a specific time window (e.g., 30 seconds). A Fast Fourier Transform (FFT) is applied to each segment to convert it into a frequency domain signal X(f). The power proportion P of each sleep frequency band (e.g., delta waves: 0.5-4Hz, theta waves: 4-8Hz) is calculated. band The calculation formula is as follows:

[0099]

[0100] Among them, f start and f end These are the start and end frequencies of the target frequency band.

[0101] Preliminary sleep stage determination: Based on the American Academy of Sleep Medicine (AASM) standards, combined with EEG power spectrum characteristics and EOG data, a preliminary sleep stage determination is performed for each 30-second window. For example, a delta wave power ratio exceeding 20% ​​is determined to be stage N3 (deep sleep); if the delta wave ratio is low and frequent eye movements are detected by EOG, it is determined to be stage REM sleep. Real-time sleep stage labels are output (awake / W, N1, N2, N3, REM).

[0102] Environmental and log data preprocessing:

[0103] Environmental data smoothing: To remove transient outliers from sensors (such as a sudden spike in noise caused by closing a door), a 5-minute sliding window is used to calculate the mean of the environmental data. If an extreme outlier is detected (such as noise > 120dB), the data is replaced by interpolation of the preceding and following 5 data points.

[0104] Log text vectorization: The sleep log text input by the user (such as "I went to sleep at 11 pm last night and woke up 2 times") is encoded using a pre-trained BERT model to generate a 768-dimensional vector representation. At the same time, named entity recognition (NER) technology is used to extract key entities (such as "sleep time: 23:00" and "number of awakenings: 2"), which are used to update the attributes of knowledge graph nodes.

[0105] The preprocessed structured features (such as delta wave proportion, real-time temperature, and sleep stage labels) are sent to the multimodal dynamic knowledge graph layer (122) and the large language model inference layer (118).

[0106] Second layer: Multimodal dynamic knowledge graph layer (122)

[0107] This is one of the core innovations of this invention, which aims to provide an LLM with a real-time updated, personalized "dynamic memory". The multimodal dynamic knowledge graph layer (122) is responsible for executing step (S2) to update the multimodal dynamic knowledge graph in real time.

[0108] 2.1 Knowledge Graph Structure Design

[0109] The structure of the multimodal dynamic knowledge graph (122) is carefully designed to store multidimensional entities related to sleep and their dynamic relationships.

[0110] Node (124): Represents an entity. This embodiment defines five types of core nodes:

[0111] Sleep state nodes: such as "N3 stage", "REM stage", "nighttime awakening". Attributes include: duration, time of occurrence, and percentage of delta waves (for N3 stage).

[0112] Physiological characteristic nodes: such as "delta wave power" and "respiratory rate". Attributes include: numerical value, acquisition time, and corresponding sleep stage.

[0113] Environmental factor nodes: such as "temperature" and "noise". Attributes include: numerical value, collection time, and whether it exceeds the optimal range (e.g., the optimal temperature range is 18-22℃).

[0114] Behavioral factors include: such as "using electronic devices before bed" and "caffeine intake." Attributes include: duration and time elapsed since sleep onset.

[0115] Intervention measures include: such as "using earplugs" and "adjusting the air conditioner to 20°C". Attributes include: historical intervention effects (such as "deep sleep rate increased by 10%)", user acceptance, and implementation status.

[0116] Edge (124): Represents the relationship between nodes. In this invention, the "edge" is dynamic, and each edge must carry two key attributes:

[0117] Dynamic weights: represent the strength of the relationship (between 0 and 1).

[0118] Timestamp: Represents the timeliness of the relationship.

[0119] Example: ("Temperature") - [Relationship: Influence, Weight: 0.7, Timestamp: 2024-10-01 02:00] → ("Percentage of N3 Period"). Edge (124) represents the dynamic causal relationship between nodes.

[0120] 2.2 Knowledge Graph Dynamic Update Mechanism

[0121] To address the static and outdated nature of existing knowledge graph technologies, this invention designs a dynamic update process that operates on a minute-by-minute basis:

[0122] Real-time data access:

[0123] Kafka message queue is used as the real-time data access module. The preprocessed data (such as EEG features pushed every 30 seconds and environmental data pushed every minute) of the multimodal data layer (125) is sent to the Kafka topic.

[0124] The Flink stream processing framework is used to consume data from Kafka and convert the data stream into "node attribute update requests" (e.g., "temperature node value updated to 26℃, timestamp updated") in real time for the knowledge graph.

[0125] Dynamic Relationship Mining:

[0126] The dynamic relationship mining module (123) uses the improved Apriori algorithm (an association rule mining algorithm) to mine association rules between multimodal data streams in a specific preset time window (e.g., 1 hour).

[0127] For example, mining the frequency of "temperature > 25°C" events and "N3 phase percentage decrease > 10%" events occurring simultaneously in the past hour.

[0128] Dynamic weight calculation (weight 3):

[0129] This invention specifically defines a method for calculating dynamic weights, which must combine the support, confidence, and time decay factor of the association rule.

[0130]

[0131] Explanation: Support(A∩B) is the frequency at which A and B occur simultaneously; Confidence(A→B) is the probability that B occurs when A occurs.

[0132] The time decay factor is one of the key aspects of this invention. It ensures that recent data contributes more to the relation weights (e.g., data from 1 hour ago has a TimeDecay of 1, while data from 24 hours ago has a TimeDecay that decays to 0.5). This allows the knowledge graph to dynamically reflect recent physiological changes in users.

[0133] Example: If within one hour, "temperature > 25℃" (A) and "the proportion of N3 period decreases" (B) appear 8 times simultaneously, and A appears 10 times alone, then Support(A∩t) = 0.8, Confidence(A→B) = 0.8. At this time, TimeDecay = 1, so the weight of this edge is Weight = 0.8 × 0.8 × 1 = 0.64.

[0134] Conflict detection and version control:

[0135] When a newly discovered association rule (such as "temperature > 25℃ → N3 period rises") conflicts with an existing relationship in the knowledge graph (such as "temperature > 25℃ → N3 period falls"), the conflict detection and version control module (123) is activated.

[0136] This embodiment introduces a Bayesian network to calculate the posterior probability of two conflicting relationships, retaining the relationship with the higher preset probability (i.e., higher posterior probability). For example, based on user historical data, if the posterior probability of "temperature 25℃ → decrease in N3 period" is calculated to be 0.9, then this relationship will be retained first.

[0137] Meanwhile, this module employs a Git-like version control mechanism, automatically generating a knowledge graph snapshot every hour to record all changes to nodes and edges. This ensures the system's traceability; when intervention fails, the previous night's knowledge graph state can be reviewed to pinpoint the cause.

[0138] Third layer: Large language model inference layer (118)

[0139] The large language model reasoning layer (118) is the "brain" of the system, responsible for making decisions with the support of dynamic knowledge.

[0140] 3.1 Triggering Intervention and Knowledge Subgraph Extraction (S3)

[0141] Triggering intervention condition (S3): The system does not intervene continuously, but rather initiates intervention when specific conditions are met. Triggering conditions can be:

[0142] Physiological signal triggers: such as the data preprocessing module (127) detecting that the user's N3 stage (deep sleep) percentage is lower than the user's average baseline for 10 consecutive minutes; or detecting a nighttime awakening event.

[0143] User log trigger: such as when a user reports "poor sleep quality" (score < 2 points) in the APP upon waking up in the morning.

[0144] Extracting the knowledge subgraph (S3):

[0145] Once the intervention condition is triggered, the large language model inference layer (118) immediately calls the subgraph extraction API (120).

[0146] The API (120) extracts the knowledge subgraph (120) most relevant to the current user state (such as the current sleep stage, current environmental data, and historical intervention measures) from the multimodal dynamic knowledge graph (122).

[0147] Example: The extracted subgraph (120) may contain nodes: “Current sleep stage = N2 stage”, “Temperature = 26℃”, “Historical intervention = use of earplugs”, and dynamic relationship edges between them (such as “Temperature > 25℃ → N3 stage proportion decreases (weight 0.64)”).

[0148] 3.2 Invoking LLM to generate suggestions

[0149] The core of calling the (S4) large language model inference layer lies in how to "feed" multimodal data and dynamic knowledge to the LLM. This invention adopts a layered prompting engineering template, taking the knowledge subgraph (120) and real-time multimodal data (125) as input.

[0150] LLM Selection and Fine-tuning:

[0151] In this embodiment, an LLM (such as Llama 3 70B Medical or GPT-4oMedical) that has been fine-tuned in the medical field is selected as the medical fine-tuning large language model (121). These models already possess basic sleep medicine knowledge (such as the AASM staging system).

[0152] Furthermore, the basic model was fine-tuned using labeled data containing 100,000 "EEG features + environmental data + intervention effects" to enhance its understanding of the "physiology-environment-intervention" mapping relationship.

[0153] Layered prompt project template:

[0154] Knowledge layer hint: The knowledge subgraph (120) is converted into a natural language description through the subgraph extraction API (120).

[0155] Example: "

Knowledge Layer

[0156] Data layer hint: A data layer hint generated based on the multimodal data.

[0157] Example: "[Data Layer] Current user's real-time data: 1. Physiological data: EEG delta wave percentage 8% (N2 phase), EOG no frequent eye movements; 2. Environmental data: bedroom temperature 26℃, noise 55dB; 3. Sleep log: fell asleep at 23:30 last night, woke up 2 times during the night."

[0158] Task-level prompts: Preset task-level prompts.

[0159] Example: "[Task Level] Based on the above data and knowledge, please complete: 1. Analyze the core reasons for poor sleep quality; 2. Generate 1-2 personalized intervention suggestions (based on the knowledge graph); 3. Estimate the improvement of sleep indicators after intervention."

[0160] 3.3 Generate interpretable intervention recommendations

[0161] To address the black-box problem of LLM, this invention requires that the output of the medical fine-tuning large language model (121) must include interpretability information (121).

[0162] Causal Chain: The output of LLM is formatted as a causal chain of "cause-suggestion-expected effect" and must cite evidence from the knowledge graph.

[0163] Example output: "Core reason: 1. Bedroom temperature is 26℃ (higher than the optimal 18-22℃). According to the knowledge graph, 'temperature > 25℃ → decrease in N3 stage proportion (weight 0.64)', resulting in the current delta wave proportion being only 8% (insufficient deep sleep). Intervention suggestion: 1. Adjust the air conditioner temperature to 20℃ (based on: the knowledge graph 'historically, adjusting the temperature to 20℃ resulted in a 20-minute earlier sleep onset time'). Expected effect: It is estimated that the delta wave proportion can be increased to 15%, entering the N3 stage."

[0164] SHAP value: During internal inference, LLM runs a SHAP (SHapley Additive exPlanations) analysis to quantify the contribution of each modality of data to intervention recommendations (such as "insufficient deep sleep").

[0165] Example output: "Contribution of each factor to 'insufficient deep sleep': temperature (SHAP value = 0.45) > phone use before bed (SHAP value = 0.32) > noise (SHAP value = 0.15)".

[0166] This interpretability (121) information allows users to clearly understand the source and importance of the advice, greatly improving compliance.

[0167] Fourth layer: Intervention execution layer (116)

[0168] The intervention implementation layer (116) is responsible for transforming the "wisdom" of LLM into "action" to address the shortcomings of recommendations that cannot be implemented.

[0169] 4.1 Instruction Conversion and Execution

[0170] Step (S5) converts the personalized intervention suggestion into device control instructions.

[0171] Instruction translation engine (117):

[0172] The engine (117) is responsible for parsing the natural language in the intervention recommendations into standardized device instructions.

[0173] Example:

[0174] LLM recommends: "Adjust the air conditioner temperature to 20°C".

[0175] The instruction conversion engine (117) parses the instruction into a JSON instruction: {"device":"AirConditioner","action":"set_temp","value":20,"time":"immediate"}

[0176] LLM recommends: "Please wear earplugs to reduce noise" (when noise cannot be controlled automatically).

[0177] The instruction translation engine (117) interprets the instruction as: {"device":"AppNotifier","action":"push_alert","content":"Noise of 55dB detected affecting deep sleep, please wear earplugs."}

[0178] MQTT Internet of Things Protocol (117):

[0179] The system acts as an MQTT client, and the standardized device commands are sent to the gateway of the smart home device (117) (such as a smart air conditioner, smart speaker, white noise machine, or smart curtain) via the MQTT Internet of Things protocol (117).

[0180] Implementation and Feedback:

[0181] Smart home devices (117) (such as air conditioners) receive instructions and perform interventions.

[0182] After the device completes its operation, it sends an "execution status" (e.g., "Air conditioner has been adjusted to 20℃") back to the system via the MQTT protocol.

[0183] The feedback status is stored in the dynamic knowledge graph (122) to update the attributes of the "intervention node" and form a closed loop of intervention.

[0184] Step 5: Feedback and optimization closed loop

[0185] The final step of this invention is to evaluate the intervention effect and use the effect to optimize the LLM decision-making, forming a continuously evolving closed loop.

[0186] 5.1 Real-time effect acquisition

[0187] After the smart home device (117) performs the intervention, multimodal data after the intervention is collected in real time to analyze the intervention effect.

[0188] Example: After the air conditioner is set to 20°C, the system (multimodal data layer 125) continuously monitors EEG data and recalculates the delta wave proportion over the next 30 minutes.

[0189] 5.2 Effect Judgment and Re-reasoning

[0190] If the intervention effect does not achieve the preset target:

[0191] Example: The expected proportion of delta waves was 15%, but after 30 minutes it only increased to 10%.

[0192] The system takes the information "intervention effect not met (temperature 20℃, δ wave proportion only 10%)" as new input and calls back the large language model inference layer (118) (S7).

[0193] Generate new personalized intervention recommendations:

[0194] LLM(121) combines this new information with the updated knowledge subgraph(120) (which may contain the knowledge that “the percentage of deep sleep increases by 8% after the white noise machine is historically turned on”) to generate new suggestions.

[0195] Example of new recommendation: "The current temperature is 20°C, but the delta wave ratio is still insufficient. It is recommended to turn on the white noise generator at the same time to reduce noise interference (55dB)."

[0196] The new recommendation is again implemented at the intervention implementation level (116).

[0197] If the intervention is effective:

[0198] The system records the successful experience of "temperature 20℃ → δ wave proportion increased to 15%" in the dynamic knowledge graph (122) to strengthen the relationship weight between the intervention and its effect.

[0199] Long-term iteration:

[0200] Every morning, the system generates a "sleep intervention report" for users, summarizing the effectiveness of the previous night's LLM recommendations (e.g., "Of the 5 recommendations this week, 4 achieved improvements in sleep indicators, with an effectiveness rate of 80%").

[0201] Based on this report, the prompt template of the large language model reasoning layer (118) is optimized over a long period of time (such as adding the constraint "prioritize interventions with historical effectiveness > 80%" to the task-level prompts), so that the system becomes more and more adaptable to the uniqueness of individuals.

[0202] In summary, this invention, through the collaborative operation of a four-layer architecture consisting of a multimodal data layer (125), a dynamic knowledge graph layer (122), a large language model reasoning layer (118), and an intervention execution layer (116), strictly implements [the required procedures / measures]. Figure 1 The steps S1 to S7 are shown. This invention successfully combines the powerful reasoning ability of LLM with real-time physiological perception, dynamic personalized knowledge, interpretable decision-making processes, and closed-loop physical intervention, solving the core defects of existing technologies such as lagging LLM recommendations, lack of interpretability, and inability to be implemented. It provides a complete, efficient, and intelligent solution for personalized sleep health management.

[0203] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A personalized sleep intervention method based on a large language model and a multimodal dynamic knowledge graph, characterized in that, include: (S1) Collect multimodal data, including physiological data, environmental data and user log data; (S2) Based on the collected multimodal data, update a multimodal dynamic knowledge graph in real time; (S3) When the intervention condition is triggered, extract the knowledge subgraph related to the current user state from the dynamic knowledge graph; (S4) The knowledge subgraph and the real-time collected multimodal data are used as inputs to call the large language model inference layer to generate personalized intervention suggestions; (S5) The personalized intervention suggestion is converted into a device control command and sent to the smart home device in the intervention execution layer to perform the intervention.

2. The method according to claim 1, characterized in that, The steps for real-time updating of the multimodal dynamic knowledge graph include: receiving the multimodal data through a real-time data access module; mining association rules between nodes based on the multimodal data within a preset time window through a dynamic relationship mining module, and calculating the dynamic weight of the association rules; when a newly mined association rule conflicts with an existing relationship in the knowledge graph, retaining the relationship with a higher preset probability through a conflict detection and version control module.

3. The method according to claim 2, characterized in that, The dynamic weight calculation combines the support, confidence, and time decay factor of the association rule.

4. The method according to claim 1, characterized in that, The step of calling the inference layer of the large language model includes: converting the knowledge subgraph into a knowledge layer prompt in natural language description through the subgraph extraction API; combining the knowledge layer prompt, the data layer prompt generated based on the multimodal data, and the preset task layer prompt into a hierarchical prompt engineering template, and inputting it into the medical fine-tuning large language model.

5. The method according to claim 1, characterized in that, The personalized intervention recommendations include interpretable information; The interpretability information includes: a "cause-recommendation-expected effect" causal chain generated based on the knowledge subgraph, and a SHAP value that quantifies the contribution of each modality of data to the intervention recommendation.

6. The method according to claim 1, characterized in that, The method further includes: after the smart home device performs the intervention, collecting multimodal data after the intervention in real time to analyze the intervention effect; if the intervention effect does not achieve the preset target, the intervention effect is used as a new input to re-call the large language model inference layer to generate new personalized intervention suggestions.

7. The method according to claim 1, characterized in that, The step of converting the personalized intervention suggestion into device control commands includes: parsing the natural language in the intervention suggestion into standardized device commands through a command conversion engine; and sending the standardized device commands to the smart home device via the MQTT Internet of Things protocol.

8. The method according to claim 1, characterized in that, The physiological data are electroencephalogram (EEG) data, electrooculogram (EOG) data, or respiratory data; before collecting multimodal data, the method also includes a data preprocessing step: performing signal cleaning and filtering on the EEG data, and calculating the power ratio of the EEG data in a preset sleep frequency band through fast Fourier transform (FFT) to determine the real-time sleep stage.

9. The method according to claim 1, characterized in that, The multimodal dynamic knowledge graph includes: nodes, which include sleep state nodes, physiological characteristic nodes, environmental factor nodes, behavioral factor nodes, and intervention measure nodes; and edges, which represent the dynamic causal relationships between nodes and carry dynamic weights and timestamps.

10. A personalized sleep intervention system based on a large language model and a multimodal dynamic knowledge graph, characterized in that, include: A multimodal data layer (125) includes a data preprocessing module (127) for collecting multimodal data, including physiological data, environmental data, and user log data; a multimodal dynamic knowledge graph layer (122) connected to the multimodal data layer (125) includes a dynamic relationship mining module (123) and a conflict detection and version control module (123) for updating a multimodal dynamic knowledge graph in real time based on the collected multimodal data; a large language model inference layer (118) connected to the dynamic knowledge graph layer (122) includes a subgraph extraction layer. API (120) and medical fine-tuning large language model (121) are used to extract knowledge subgraph (120) from the dynamic knowledge graph when intervention conditions are triggered, and the knowledge subgraph (120) and the real-time collected multimodal data are used as input to generate personalized intervention suggestions; an intervention execution layer (116) is connected to the large language model inference layer (118), including instruction conversion engine (117) and MQTT Internet of Things protocol (117), used to convert the personalized intervention suggestions into device control instructions and send them to smart home devices (117) to execute the intervention.

Images