A sleep disorder diagnosis and treatment scheme determination method and system based on a language large model

Abstract

This application discloses a method and system for determining sleep disorder diagnosis and treatment plans based on a large language model, relating to the fields of artificial intelligence and smart healthcare. The method acquires medical record text, time-series physiological data, medical imaging data, and patient lifestyle and behavioral pattern data, aligns and encodes multimodal sleep information, and constructs a multimodal representation in a unified semantic space. This multimodal representation is input into a large language model enhanced with medical knowledge, and a comprehensive semantic representation characterizing sleep disorder symptoms and their correlations is generated through a cross-modal attention mechanism. Combining the patient's past medical history and current sleep state, the type and severity of the sleep disorder are inferred. Based on the diagnostic results, a personalized treatment plan, including treatment strategies, intervention priorities, and follow-up recommendations, is generated through an intelligent matching model. This invention improves the accuracy, interpretability, and individual adaptability of sleep disorder diagnosis and treatment.

Description

Technical Field

[0001] This specification relates to the fields of artificial intelligence and smart healthcare. More specifically, this application relates to a method and system for determining a sleep disorder diagnosis and treatment plan based on a large language model. Background Technology

[0002] Sleep disorders are a common and complex health problem, encompassing various types such as insomnia, obstructive sleep apnea, and narcolepsy. Their symptoms are diverse, their causes complex, and there are significant individual differences among patients. The diagnosis and treatment of sleep disorders typically require a comprehensive analysis of the patient's medical history, chief complaints, physiological monitoring data, imaging results, and long-term lifestyle and behavioral habits.

[0003] Current methods for diagnosing and treating sleep disorders primarily rely on human experience or fragmented information systems. This makes it difficult to perform unified modeling and collaborative analysis of multi-source information, including medical records, time-series physiological signals, imaging data, and behavioral data. Diagnostic results are highly dependent on physician experience, lacking consistency and stability. While some existing technologies incorporate machine learning or deep learning models for auxiliary analysis, they often focus on single modalities or simple feature combinations, failing to effectively characterize the intrinsic relationships between different symptoms, physiological indicators, and behavioral factors.

[0004] Furthermore, existing technologies often lack clear medical knowledge constraints when using large models or intelligent algorithms for diagnostic reasoning and treatment recommendations. The reasoning process is not interpretable enough, and it is difficult to simultaneously consider efficacy, safety risks, and patient compliance during the treatment plan generation stage, which limits the application effect of intelligent sleep disorder diagnosis and treatment technology in clinical scenarios.

[0005] Therefore, there is an urgent need for a technical solution that can integrate multimodal sleep data, incorporate medical knowledge constraints, and generate personalized treatment plans to improve the scientific rigor and practicality of sleep disorder diagnosis and treatment. Summary of the Invention

[0006] The summary section introduces a series of simplified concepts, which will be further explained in detail in the detailed description section. This summary section is not intended to limit the key and essential technical features of the claimed technical solution, nor is it intended to determine the scope of protection of the claimed technical solution.

[0007] Firstly, this application proposes a method for determining a sleep disorder diagnosis and treatment plan based on a large language model, including: Acquire sleep-related multimodal data of the subjects to be diagnosed and treated, including medical record text data, time-series physiological data, medical imaging data, and patient lifestyle and behavioral pattern data; Modality alignment and feature encoding are performed on the above sleep-related multimodal data to map them into a multimodal representation in a unified semantic space; The above multimodal representations are input into a large language model enhanced with medical domain knowledge. Using a Transformer-based cross-modal attention mechanism, a comprehensive semantic representation representing the symptom features of sleep disorders and their interrelationships is generated. Based on the above comprehensive semantic representation, combined with the patient's past medical history and current sleep state, the type and severity of sleep disorder are inferred, and a diagnostic result is obtained; Based on the above diagnostic results, a treatment plan matching the individual characteristics of the patient is generated through an intelligent matching model. The treatment plan includes recommendations for treatment strategies, intervention priorities, and follow-up adjustment recommendations.

[0008] In one feasible implementation, the above-mentioned modality alignment and feature encoding processing of the sleep-related multimodal data to map it into a multimodal representation in a unified semantic space includes: Natural language processing based on medical named entity recognition and semantic relation extraction is performed on the above medical record text data to extract descriptions of sleep-related symptoms, diagnostic conclusions, medication records and disease progression information, and these are encoded into the above text semantic features through word vector models or context semantic coding models. The above-mentioned time-series physiological data were subjected to time synchronization processing based on a sliding time window, and noise suppression was performed using a bandpass filtering algorithm. Temporal features reflecting sleep stages, number of awakenings, and sleep continuity were extracted from the data. The above medical image data were preprocessed based on image normalization and region of interest segmentation, and image representation features corresponding to upper airway structures, brain regions and other sleep disorder-related structures were extracted using convolutional neural networks. The lifestyle and behavioral pattern data of the above patients were used to model behavioral patterns based on hidden Markov models to generate behavioral characteristics that reflect circadian rhythms, sleep habit stability and behavioral risk factors. Based on the cross-modal embedding alignment strategy, the above-mentioned text semantic features, temporal features, image representation features and behavioral features are mapped to a unified semantic embedding space, and the above-mentioned multimodal representation is formed by feature weighted fusion.

[0009] In one feasible implementation, the aforementioned multimodal representation is input into a large language model enhanced with medical domain knowledge. A Transformer-based cross-modal attention mechanism is then used to generate a comprehensive semantic representation characterizing the symptom features of sleep disorders and their interrelationships, including: Construct a medical domain knowledge structure that includes the relationships between sleep disorder types, symptoms, physiological indicators, imaging features, and treatment methods; By introducing the aforementioned medical knowledge structure as an external knowledge constraint into the reasoning process of the aforementioned language model, semantic completion and consistency correction are performed on the aforementioned multimodal representation. Based on the aforementioned cross-modal attention mechanism, the weights of the aforementioned text semantic features, temporal features, image representation features, and behavioral features are dynamically allocated in the reasoning process. By integrating the weighted modal features, a comprehensive semantic representation is generated that characterizes the aforementioned sleep disorder symptoms and their intrinsic relationships.

[0010] In one feasible implementation, the above-mentioned medical domain knowledge structure is introduced as an external knowledge constraint into the reasoning process of the above-mentioned large language model, and semantic completion and consistency correction are performed on the above-mentioned multimodal representation, including: The aforementioned medical knowledge structure is constructed as a knowledge graph structure including nodes and edges. The nodes of the knowledge graph structure are used to represent the types of sleep disorders, symptoms, physiological indicators, imaging features, and treatment methods. The edges of the graph structure are used to represent the causal relationships, correlation relationships, or diagnostic and treatment constraints between the nodes. During the reasoning process of the above-mentioned large language model, based on the symptom features and physiological indicator features identified in the above-mentioned multimodal representation, candidate knowledge nodes associated with them are retrieved in the above-mentioned knowledge graph structure to complete the missing medical semantic information in the above-mentioned multimodal representation. Based on the association strength and constraint relationship between the nodes, the consistency of the association rationality between different modal features in the multimodal representation is checked. When a feature combination that does not conform to the constraints of medical knowledge is detected, the weight of the corresponding feature is suppressed or corrected. The multimodal representation, after semantic completion and consistency correction, is fed back into the cross-modal attention mechanism, enabling the large language model to prioritize features that are more relevant to the sleep disorder and conform to medical knowledge constraints during subsequent reasoning, in order to generate the comprehensive semantic representation.

[0011] In one feasible implementation, based on the aforementioned comprehensive semantic representation, and combined with the patient's past medical history and current sleep state, the type and severity of the sleep disorder are inferred to obtain a diagnostic result, including: The above-mentioned comprehensive semantic representation is combined with the above-mentioned patient's past medical history for joint analysis to determine the candidate type set of the above-mentioned sleep disorders; Based on the temporal physiological characteristics changes corresponding to the current sleep state, the above candidate type set is screened and confidence is evaluated. Based on the symptom association strength and physiological index deviation reflected in the above comprehensive semantic representation, the severity of the above sleep disorders is classified. The output includes the above-mentioned diagnostic results, which include the above-mentioned sleep disorder types and the above-mentioned severity classifications.

[0012] In one feasible implementation, the above-mentioned comprehensive semantic representation is jointly analyzed with the patient's past medical history to determine a set of candidate types of the sleep disorder, including: Based on the symptom features, physiological indicators and their correlations represented in the above comprehensive semantic representation, a semantic feature vector representing the current abnormal sleep state of the patient is constructed. The historical diagnostic information, past symptom evolution information, and past treatment response information recorded in the past medical history of the above patients are analyzed in a structured manner to form a representation of medical history characteristics; Based on the preset semantic template of sleep disorder type, the semantic feature vector and the medical history feature representation are compared with the semantic template of sleep disorder type to calculate the similarity and obtain the matching score corresponding to each sleep disorder type. Based on the matching score and the comorbidity and exclusive diagnostic constraints recorded in the patients’ past medical history, sleep disorder types that do not meet the medical rationality constraints are eliminated. The sleep disorder types retained after the above similarity calculation and the above diagnostic constraint screening are determined as the candidate type set of the above sleep disorders.

[0013] In one feasible implementation, the aforementioned sleep disorder type semantic template is a pre-constructed structured semantic template for different sleep disorder types; The above-mentioned semantic feature vector and medical history feature representation are compared with the above-mentioned sleep disorder type semantic template for similarity calculation, including: For each type of sleep disorder, a multi-layered semantic template structure is constructed, including semantic sub-templates for core symptoms, semantic sub-templates for physiological indicators, semantic sub-templates for behavioral patterns, and semantic sub-templates for typical disease progression. The symptom features, physiological indicator features and behavioral features in the above comprehensive semantic representation are mapped to the corresponding semantic sub-template spaces to form multi-dimensional matching vectors corresponding to the above sleep disorder types. The medical history features obtained from the analysis of the patient's past medical history are mapped to the typical disease course evolution semantic sub-template to characterize the degree of matching between the patient's historical state and the typical evolution path of the sleep disorder type. Based on the similarity results of the above multidimensional matching vector and the above medical history features at the semantic sub-template level, a weighted fusion strategy is adopted to calculate the comprehensive matching score corresponding to each type of sleep disorder. Among them, the weight of each semantic sub-template is preset or adaptively adjusted according to the diagnostic criticality of the sleep disorder type, so that different types of sleep disorders have different discrimination emphases in the similarity calculation process.

[0014] In one feasible implementation, the above-mentioned method of generating a treatment plan that matches the individual characteristics of the patient based on the diagnostic results using an intelligent matching model includes: Based on the type and severity classification of sleep disorders in the above diagnostic results, a set of corresponding candidate treatment strategies is determined, which includes drug treatment strategies, non-drug intervention strategies, and behavioral intervention strategies. Obtain individual constraint information related to the above-mentioned patients, including patient age characteristics, past medication history, adverse reaction records, comorbidity information, and compliance assessment results; Based on the above comprehensive semantic representation and the above individual constraint information, a treatment plan matching feature vector is constructed, and the above treatment plan matching feature vector is input into the above intelligent matching model to evaluate the suitability of each treatment strategy in the above candidate treatment strategy set. In the above fit assessment process, a multi-objective optimization strategy is adopted to generate a comprehensive score corresponding to each diagnosis and treatment strategy; Based on the comprehensive score, the candidate treatment strategies are ranked, and the treatment strategies that meet the preset safety constraints are selected from the ranking results to form the treatment plan for the above-mentioned patients.

[0015] In one feasible implementation, during the aforementioned fit assessment process, a multi-objective optimization strategy is employed to generate a comprehensive score corresponding to each treatment strategy, including: For each candidate treatment strategy, construct a multidimensional optimization objective function that includes at least the treatment effectiveness objective, the safety risk control objective, and the patient compliance objective; Based on individual patient characteristics and diagnostic results, the above-mentioned safety risk control objectives are set as hard constraints, while the above-mentioned treatment effectiveness objectives and patient compliance objectives are set as optimizable objectives. Under the premise of satisfying the above hard constraints, a multi-objective decision-making algorithm is used to jointly solve the above optimizable objectives to obtain the comprehensive evaluation results of each candidate treatment strategy in the multi-objective space. Based on the above comprehensive evaluation results, a comprehensive score is generated to characterize the overall suitability of each candidate treatment strategy, and the above comprehensive score is used as the basis for the decision-making of treatment plan selection and ranking.

[0016] Secondly, this application proposes a system for determining sleep disorder diagnosis and treatment plans based on a large language model, including: The acquisition unit is used to acquire sleep-related multimodal data of the subject to be diagnosed and treated, wherein the aforementioned sleep-related multimodal data includes medical record text data, time-series physiological data, medical imaging data, and patient lifestyle and behavioral pattern data; The processing unit is used to perform modality alignment and feature encoding on the above-mentioned sleep-related multimodal data to map it into a multimodal representation in a unified semantic space; The generation unit is used to input the above multimodal representations into a large language model enhanced with medical domain knowledge, and use a Transformer-based cross-modal attention mechanism to generate a comprehensive semantic representation that characterizes the symptom features of sleep disorders and their interrelationships. The inference unit is used to infer the type and severity of sleep disorder based on the above comprehensive semantic representation, combined with the patient's past medical history and current sleep state, and to obtain a diagnostic result; The matching unit is used to generate a treatment plan that matches the individual characteristics of the patient based on the above diagnostic results through an intelligent matching model. The treatment plan includes treatment strategy suggestions, intervention priorities, and follow-up adjustment suggestions.

[0017] In summary, this invention acquires medical record text data, time-series physiological data, medical imaging data, and patient lifestyle and behavioral pattern data. By performing modal alignment and feature encoding on these multimodal data, it achieves a fusion expression of sleep-related information from different sources and scales within a unified semantic space. This effectively overcomes the problems of fragmented multimodal data and difficulty in collaborative analysis in existing technologies, thus enabling a more comprehensive and accurate depiction of the patient's sleep state and potential abnormal characteristics. Based on multimodal fusion, this invention introduces a large-scale language model enhanced with medical domain knowledge and utilizes a Transformer-based cross-modal attention mechanism to dynamically weight and model the relationships between different modal features. This allows the model to simultaneously consider the intrinsic connections between symptom presentation, physiological indicators, imaging structure, and behavioral factors during inference, thereby improving the ability to identify complex sleep disorder symptom patterns and avoiding misjudgments caused by relying solely on a single indicator or local feature. This invention combines a patient's past medical history and current sleep state during the diagnostic stage to jointly infer the type and severity of sleep disorders. It not only provides a clear diagnostic type but also grades the condition based on symptom correlation strength and deviations in physiological indicators, outputting structured and interpretable diagnostic results, thus improving the stability and clinical credibility of diagnostic conclusions. In the treatment plan generation stage, this invention introduces an intelligent matching model that comprehensively considers the diagnostic results and individual patient constraints, evaluating and optimizing the suitability of candidate treatment strategies to generate a personalized treatment plan that simultaneously includes treatment strategy suggestions, intervention priorities, and follow-up adjustment recommendations. This approach avoids the "one-size-fits-all" approach of existing technologies, ensuring that the treatment plan is more in line with the patient's actual implementation capabilities and long-term management needs while maintaining safety. In summary, this invention, through multimodal unified modeling, large-scale model reasoning constrained by medical knowledge, and a multi-objective optimized treatment plan generation mechanism, realizes a complete technical chain from intelligent diagnosis of sleep disorders to personalized treatment decision support. It effectively improves the scientific rigor, consistency, and clinical feasibility of sleep disorder diagnosis and treatment, possessing high practical application value and significant potential for wider application. Attached Figure Description

[0018] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit this specification. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings: Figure 1 A flowchart illustrating a method for determining a sleep disorder diagnosis and treatment plan based on a large language model, provided in an embodiment of this application; Figure 2 This application provides a flowchart illustrating a multimodal representation method mapped to a unified semantic space, as shown in the embodiments of this application. Figure 3 A flowchart illustrating a method for generating a comprehensive semantic representation of sleep disorder symptoms and their interrelationships, provided in an embodiment of this application; Figure 4 This is a flowchart illustrating a method for semantic completion and consistency correction of multimodal representation provided in an embodiment of this application. Figure 5 This is a flowchart illustrating a method for inferring the type and severity of sleep disorders to obtain diagnostic results, provided in an embodiment of this application. Figure 6 This is a flowchart illustrating a method for determining a set of candidate types of the aforementioned sleep disorders, provided in an embodiment of this application. Figure 7 This is a flowchart illustrating a similarity calculation method provided in an embodiment of this application. Figure 8 A schematic flowchart illustrating a method for generating a treatment plan that matches the individual characteristics of a patient, provided in an embodiment of this application; Figure 9 This application provides a flowchart illustrating a method for generating comprehensive scores corresponding to various treatment strategies. Figure 10 This is a schematic diagram of a sleep disorder diagnosis and treatment plan determination system based on a large language model, provided in an embodiment of this application. Detailed Implementation

[0019] The terms "first," "second," "third," "fourth," etc. (if present) in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments described herein can be implemented in a sequence other than that illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus. The technical solutions of the embodiments of this application will now be clearly and completely described in conjunction with the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them.

[0020] Figure 1 This application provides a flowchart illustrating a method for determining a sleep disorder diagnosis and treatment plan based on a large language model. The method may specifically include: S110. Obtain sleep-related multimodal data of the subject to be diagnosed and treated, wherein the aforementioned sleep-related multimodal data includes medical record text data, time-series physiological data, medical imaging data, and patient lifestyle and behavioral pattern data; S120. Perform modality alignment and feature encoding on the above sleep-related multimodal data to map it into a multimodal representation in a unified semantic space; S130. Input the above multimodal representation into a large language model enhanced with medical domain knowledge, and use a Transformer-based cross-modal attention mechanism to generate a comprehensive semantic representation that characterizes the symptom features of sleep disorders and their interrelationships. S140. Based on the above comprehensive semantic representation, combined with the patient's past medical history and current sleep state, the type and severity of the sleep disorder are inferred to obtain the diagnostic result; S150. Based on the above diagnostic results, a treatment plan matching the patient's individual characteristics is generated through an intelligent matching model. The treatment plan includes recommendations for treatment strategies, intervention priorities, and follow-up adjustments.

[0021] For example, in step S110, sleep-related multimodal data of the patient to be diagnosed and treated is acquired. The aforementioned sleep-related multimodal data includes not only medical record text data reflecting the patient's complaints and previous treatment processes, but also time-series physiological data collected by wearable devices or polysomnography devices, as well as medical imaging data used to assist in judging the physiological structural characteristics related to sleep disorders. At the same time, data on the patient's lifestyle and behavioral patterns are also collected, such as sleep patterns, pre-sleep behavior characteristics, and long-term behavioral habits, thereby comprehensively depicting the patient's sleep state and potential abnormal characteristics from multiple dimensions.

[0022] In step S120, the aforementioned sleep-related multimodal data undergoes modality alignment and feature encoding. By uniformly modeling different data modalities at the temporal scale, semantic dimension, and feature representation levels, textual information, physiological signal features, image representation features, and behavioral pattern features are mapped into a unified semantic space, forming a multimodal representation that comprehensively reflects the patient's sleep abnormalities, providing a consistent and fusionable input foundation for subsequent intelligent reasoning.

[0023] In step S130, the multimodal representations described above are input into a large-scale language model enhanced with medical domain knowledge for processing. This large-scale language model incorporates medical knowledge constraints related to sleep disorders during pre-training or inference, and utilizes a Transformer-based cross-modal attention mechanism to dynamically weight and model the relationships between features from different modalities, thereby generating a comprehensive semantic representation that characterizes the symptom features of sleep disorders and their inherent correlations. Through this processing, the model can not only identify single symptom features but also characterize the potential connections between different symptoms, physiological indicators, and behavioral factors.

[0024] In step S140, based on the aforementioned comprehensive semantic representation, the type and severity of the sleep disorder are further inferred by combining the patient's past medical history and current sleep state. By jointly analyzing the current comprehensive semantic representation with the patient's historical diagnostic information, disease progression record, and current physiological state, the possible types of sleep disorders the patient may have are determined. Furthermore, the severity of the sleep disorder is graded based on the strength of symptom association and the degree of deviation of physiological indicators, thereby obtaining a structured and interpretable diagnostic result.

[0025] In step S150, based on the above diagnostic results, a treatment plan matching the patient's individual characteristics is generated through an intelligent matching model. When generating the treatment plan, the intelligent matching model comprehensively considers the type and severity of the sleep disorder, as well as the patient's individual constraints, to evaluate and optimize the suitability of candidate treatment strategies. This results in a personalized treatment plan that includes treatment strategy recommendations, intervention priority settings, and follow-up and dynamic adjustment suggestions, providing decision support for clinicians and improving the scientific rigor and targeted nature of sleep disorder diagnosis and treatment.

[0026] In summary, this invention acquires medical record text data, time-series physiological data, medical imaging data, and patient lifestyle and behavioral pattern data. By performing modal alignment and feature encoding on these multimodal data, it achieves a fusion expression of sleep-related information from different sources and scales within a unified semantic space. This effectively overcomes the problems of fragmented multimodal data and difficulty in collaborative analysis in existing technologies, thus enabling a more comprehensive and accurate depiction of the patient's sleep state and potential abnormal characteristics. Based on multimodal fusion, this invention introduces a large-scale language model enhanced with medical domain knowledge and utilizes a Transformer-based cross-modal attention mechanism to dynamically weight and model the relationships between different modal features. This allows the model to simultaneously consider the intrinsic connections between symptom presentation, physiological indicators, imaging structure, and behavioral factors during inference, thereby improving the ability to identify complex sleep disorder symptom patterns and avoiding misjudgments caused by relying solely on a single indicator or local feature. This invention combines a patient's past medical history and current sleep state during the diagnostic stage to jointly infer the type and severity of sleep disorders. It not only provides a clear diagnostic type but also grades the condition based on symptom correlation strength and deviations in physiological indicators, outputting structured and interpretable diagnostic results, thus improving the stability and clinical credibility of diagnostic conclusions. In the treatment plan generation stage, this invention introduces an intelligent matching model that comprehensively considers the diagnostic results and individual patient constraints, evaluating and optimizing the suitability of candidate treatment strategies to generate a personalized treatment plan that simultaneously includes treatment strategy suggestions, intervention priorities, and follow-up adjustment recommendations. This approach avoids the "one-size-fits-all" approach of existing technologies, ensuring that the treatment plan is more in line with the patient's actual implementation capabilities and long-term management needs while maintaining safety. In summary, this invention, through multimodal unified modeling, large-scale model reasoning constrained by medical knowledge, and a multi-objective optimized treatment plan generation mechanism, realizes a complete technical chain from intelligent diagnosis of sleep disorders to personalized treatment decision support. It effectively improves the scientific rigor, consistency, and clinical feasibility of sleep disorder diagnosis and treatment, possessing high practical application value and significant potential for wider application.

[0027] In one feasible implementation, such as Figure 2 As shown, step S120 above performs modality alignment and feature encoding on the aforementioned sleep-related multimodal data to map it into a multimodal representation under a unified semantic space, including: S1201. Perform natural language processing on the above medical record text data based on medical named entity recognition and semantic relation extraction to extract descriptions of sleep-related symptoms, diagnostic conclusions, medication records and disease progression information, and encode them into the above text semantic features through word vector model or context semantic coding model. S1202. Perform time synchronization processing based on sliding time window on the above time-series physiological data, and use bandpass filtering algorithm to suppress noise, and extract time-series features reflecting sleep stages, number of awakenings and sleep continuity. S1203. Perform image preprocessing on the above medical image data based on image normalization and region of interest segmentation, and use convolutional neural networks to extract image representation features corresponding to upper airway structures, brain-related regions and other sleep disorder-related structures. S1204. Based on the above patient lifestyle and behavior pattern data, behavioral pattern modeling based on hidden Markov model is used to generate behavioral characteristics that reflect circadian rhythm, sleep habit stability and behavioral risk factors. S1205. Based on the cross-modal embedding alignment strategy, the above-mentioned text semantic features, the above-mentioned temporal features, the above-mentioned image representation features and the above-mentioned behavioral features are mapped to a unified semantic embedding space, and the above-mentioned multimodal representation is formed by feature weighted fusion.

[0028] For example, the purpose of step S120 is to transform sleep-related multimodal data from different sources, at different scales, and with significantly different forms of expression into a unified semantic space representation that can be uniformly understood and jointly reasoned by subsequent large language models, thereby avoiding the fragmentation problem of "text for text, signal for signal, and image for image" and enabling multimodal information to be compared, fused, and constrained within the same semantic coordinate system.

[0029] Specifically, in step S1201, natural language processing based on medical named entity recognition and semantic relation extraction is performed on the medical record text data. Using a medical entity vocabulary for sleep disorder scenarios as constraints, symptom entities such as difficulty falling asleep, early awakening, nighttime awakening, daytime sleepiness, snoring, sleepwalking, and REM-related abnormal behaviors are identified, along with disease entities such as OSA, insomnia disorder, narcolepsy, RLS, and RBD. Relationships such as symptom-duration, symptom-exacerbating factors, medication-efficacy / adverse reactions, and diagnosis-evidence sources are extracted to form structured semantic fragments. Further, these semantic fragments are input into a word vector model or a contextual semantic encoding model to obtain text semantic feature vectors. This enables text information to have a continuous representation that can be aligned with other modalities.

[0030] In step S1202, time synchronization based on a sliding time window is first performed on the time-series physiological data (e.g., heart rate, blood oxygen, respiratory flow, chest and abdominal movement, body movement, snoring energy spectrum, EEG / EMG / EOG, etc.) to ensure that different sensor channels are aligned under the same time reference. Let the... The original signal of each channel is At a sampling frequency of Take the window length below (seconds), step size A sliding window of (seconds) then the first The time index set for each window is: Then, bandpass filtering was applied to each channel to suppress noise, resulting in a purified signal: in, This represents the bandpass filter operator. and The first The low and high cutoff frequencies of the channel filter are used. Based on the purified signal, sleep stage and sleep continuity-related features are extracted within each time window, such as the number of awakening events, micro-awakening density, blood oxygen saturation index, and energy changes during apnea / low-flux weather segments, and summarized into a time-series feature vector. .

[0031] In step S1203, medical imaging data (such as upper airway CT / MRI, head and neck structural images, brain-related images if necessary, or other images associated with sleep disorders) are normalized and segmented into regions of interest to focus the model's attention on anatomical regions strongly associated with sleep disorders. Let the image be... The region of interest mask obtained by the segmentation model is Then the ROI image is obtained: in, This represents element-wise multiplication. Furthermore, it will... Input convolutional neural network to extract image representation feature vectors This vector can encode information highly correlated with sleep disorders such as OSA, such as upper airway stenosis morphology, palatopharyngeal segment structure, and tongue root posterior displacement structure, thus providing structural evidence for subsequent multimodal reasoning.

[0032] In step S1204, a Hidden Markov Model is used to model the patient's lifestyle and behavioral pattern data to characterize the potential transition patterns between behavior and sleep states. The daily behavioral sequence or bedtime behavioral sequence is represented as an observation sequence. ,in It can be composed of bedtime, wake-up time, nap duration, caffeine / alcohol intake, exercise intensity, screen exposure time, etc.; a latent state sequence is introduced. (For example, latent states such as stable circadian rhythm, high risk of delayed sleep onset, high risk of nocturnal awakening, and cumulative sleep deprivation), then the joint probability of the Hidden Markov Model (HMM) is: in, Let be the state transition probability. To observe the emission probability, decoding the observed sequence yields behavioral feature vectors reflecting the stability of circadian rhythms, the duration of behavioral risk states, and the intensity of risk state transitions. .

[0033] In step S1205, to map the above four types of features to a unified semantic embedding space and form the final multimodal representation, this embodiment adopts a weighted fusion strategy driven by cross-modal embedding alignment and evidence consistency. Specifically, the features of each modality are first linearly projected to unify the dimensions: in, and The first The projection matrix and bias vector of the modality This is the modal representation mapped to a unified semantic embedding space. To enhance creativity and improve cross-modal evidence consistency, this embodiment introduces a consistency-credibility joint gating weight to calculate the fusion weights for each modality: in, For the first Modal fusion weights, As a weighting factor, This represents the quality or reliability score of the modality data (e.g., normalized results obtained from sensor missing rate, signal-to-noise ratio, image artifact level, text integrity, etc.). This represents the semantic consistency score between this modality and other modalities. The semantic consistency score can be defined using the cosine consistency with the mean of other modalities: in, Represents cosine similarity. Indicates except the first The mean vector of the modes other than the mode. The size of the modality set is given. Therefore, when a modality is reliable but conflicts with evidence from other modalities, or is consistent but has poor data quality, its weight will be adaptively suppressed, thus achieving evidence conflict resolution and noise robustness during the multimodal fusion stage.

[0034] Finally, a multimodal representation in a unified semantic space is formed through feature weighted fusion: in, That is, the multimodal representation output in step S120 can be directly used as the input of the cross-modal attention mechanism in the subsequent step S130, enabling the language big model to jointly reason about sleep disorder-related symptoms, physiological indicators, imaging evidence and behavioral risk factors in a unified semantic space.

[0035] In the above formula, the meanings of each parameter are as follows: This is a text semantic feature vector. This is a temporal physiological feature vector. For image representation feature vectors, This is a behavioral feature vector; and The projection parameters that map different modalities to a unified semantic embedding space; This is the mapped modal representation; For weighting; This is a trade-off coefficient between reliability and consistency. Scoring modal reliability; Scoring modal semantic consistency; For the final multimodal representation; The sampling frequency; The length of the sliding window; This refers to the sliding window step size; It is a bandpass filter operator; For the first The cutoff frequency parameter of the channel bandpass filter.

[0036] Through the above steps, this embodiment not only achieves the alignment and fusion of multimodal data in a unified semantic space, but also enables the fusion weights to be adaptively adjusted according to data quality and cross-modal evidence consistency through a reliability-consistency joint gating mechanism. This allows the suppression of noisy modalities and conflicting evidence to be completed before entering the language big model inference, providing a more stable, interpretable, and medically logical feature basis for sleep disorder diagnosis inference and treatment plan generation.

[0037] In one feasible implementation, such as Figure 3 As shown, step S130 inputs the multimodal representation into a large language model enhanced with medical domain knowledge, and uses a Transformer-based cross-modal attention mechanism to generate a comprehensive semantic representation characterizing the symptom features of sleep disorders and their interrelationships, including: S1301. Construct a medical domain knowledge structure that includes the relationships between sleep disorder types, symptoms, physiological indicators, imaging features, and treatment methods. S1302. Introduce the above-mentioned medical knowledge structure as an external knowledge constraint into the reasoning process of the above-mentioned language model, and perform semantic completion and consistency correction on the above-mentioned multimodal representation. S1303. Based on the above cross-modal attention mechanism, dynamically allocate the weights of the above text semantic features, the above temporal features, the above image representation features and the above behavioral features in the reasoning process; S1304. By integrating the weighted modal features, a comprehensive semantic representation is generated that characterizes the aforementioned sleep disorder symptoms and their intrinsic relationships.

[0038] In one feasible implementation, such as Figure 4 As shown, step S1302 introduces the aforementioned medical domain knowledge structure as an external knowledge constraint into the reasoning process of the aforementioned language model, performing semantic completion and consistency correction on the aforementioned multimodal representation, including: S13021. Construct the above-mentioned medical knowledge structure into a knowledge graph structure including nodes and edges, wherein the nodes of the above-mentioned knowledge graph structure are used to represent the disease type, symptoms, physiological indicators, imaging features and treatment methods of sleep disorders, and the edges of the above-mentioned graph structure are used to represent the causal relationship, correlation relationship or diagnosis and treatment constraint relationship between the above-mentioned nodes. S13022. During the reasoning process of the above-mentioned large language model, based on the symptom features and physiological indicator features identified in the above-mentioned multimodal representation, candidate knowledge nodes associated with them are retrieved in the above-mentioned knowledge graph structure to complete the missing medical semantic information in the above-mentioned multimodal representation. S13023. Based on the association strength and constraint relationship between the above nodes, the consistency of the association rationality between different modal features in the above multimodal representation is checked. When a feature combination that does not conform to the constraints of medical knowledge is detected, the weight of the corresponding feature is suppressed or corrected. S13024. The multimodal representation after semantic completion and consistency correction is fed back to the cross-modal attention mechanism, so that the language big model will prioritize feature information that is more relevant to the sleep disorder and conforms to medical knowledge constraints in the subsequent reasoning process, so as to generate the comprehensive semantic representation.

[0039] For example, after completing the steps After obtaining the multimodal representation in a unified semantic space, this multimodal representation is input into a large-scale language model enhanced with medical domain knowledge. Under the dual mechanisms of medical knowledge constraints and cross-modal attention fusion, the large-scale language model performs joint reasoning on the intrinsic connections between sleep disorder-related symptoms, indicators, images, and behaviors, thereby outputting a comprehensive semantic representation that characterizes the features of sleep disorder symptoms and their interrelationships. Unlike end-to-end inference that relies solely on data-driven approaches, this embodiment introduces medical domain knowledge structures as external constraints into the reasoning process. This allows the model to not only see multimodal evidence but also to perform consistency checks and semantic completion of the evidence according to clinical diagnostic logic, thereby improving the credibility and interpretability of the inference results.

[0040] Specifically, in step S1301, a medical domain knowledge structure is constructed. This structure covers the relationships between sleep disorder types (e.g., insomnia, obstructive sleep apnea, narcolepsy, RLS, RBD, etc.), symptoms (difficulty falling asleep, early awakening, nighttime awakening, daytime sleepiness, snoring, abnormal behavior, etc.), physiological indicators (AHI, ODI, wakefulness index, minimum blood oxygen saturation, sleep efficiency, REM ratio, etc.), imaging features (upper airway stenosis morphology, soft palate / tongue base related structural features, etc.), and treatment methods (CPAP, orthodontic treatment, CBT-I, drug intervention, behavioral management, etc.). To facilitate constraints and retrieval during the reasoning phase, the aforementioned medical domain knowledge structure can be expressed in the form of a knowledge graph, where the knowledge graph is... in, This is a set of nodes used to represent disease type nodes, symptom nodes, physiological indicator nodes, imaging feature nodes, and treatment method nodes. This is a set of edges used to represent causal relationships, correlations, and diagnostic constraints between nodes. Furthermore, to enhance creativity and make the knowledge constraints adjustable in strength, this embodiment assigns a relational weight to each edge. Used for quantizing nodes and The strength of medical associations or constraints between them is used to obtain a weighted knowledge graph. In step S1302, the aforementioned medical knowledge structure is introduced as an external knowledge constraint into the language large-scale model reasoning process to perform semantic completion and consistency correction on the multimodal representation. This process corresponds to a detailed implementation of steps S13021-S13024: first, a knowledge graph structure including nodes and edges is constructed in S13021; ​​then, in S13022, based on the symptom features and physiological indicator features identified in the multimodal representation, candidate knowledge nodes associated with them are retrieved in the knowledge graph to complete the missing or incomplete medical semantic information in the multimodal representation. To make this retrieval process feasible and creative, this embodiment adopts a joint strategy of graph structure neighborhood retrieval and semantic similarity retrieval. Let the set of anchor points parsed from the multimodal representation be denoted as . (Anchor points can be symptom entities or indicator entities), then the candidate node set can be represented as in, Represents anchor node In knowledge graphs The set of nodes within the jump neighborhood. Indicates selecting based on similarity. 1 node The semantic embedding vector of the anchor point. Candidate nodes semantic embedding vector, Cosine similarity can be used. Through the above joint retrieval, it is possible to ensure that candidate knowledge conforms to the knowledge graph structure constraints, while also covering matching biases caused by synonymous or implicit expressions in the text.

[0041] Furthermore, in S13023, a consistency check is performed on the reasonableness of the association between multimodal features. When a feature combination that does not conform to the constraints of medical knowledge is detected, the corresponding feature weight is suppressed or corrected. To demonstrate the inventive point of this embodiment, this embodiment introduces a knowledge-consistency penalty to explicitly measure the degree of agreement between the current multimodal evidence and medical knowledge. Let the activation intensity vector of the multimodal representation for several medical concepts (disease / symptom / indicator / imaging feature) be . ,in Represents a node Given the degree of evidence support in the current reasoning, the knowledge consistency penalty can be defined as: in, Represents the hinge function. The constraint mapping function is determined by the edge type: when the edge represents a causal / promoting relationship, it can take... To encourage consistency and reinforcement; when the edge represents an exclusive / taboo relationship, it can be taken as... Punishment is applied to simultaneously high activation; when an edge represents a treatment constraint relationship, it can be adjusted according to the rules. Set as a threshold-based mapping. Therefore, when conflicts arise, such as mutually exclusive concepts being strongly supported by multimodal evidence simultaneously, or a key causal chain being broken, This will increase significantly, thereby triggering weight suppression or correction.

[0042] in, Represents the hinge function. The constraint mapping function is determined by the edge type: when the edge represents a causal / promoting relationship, it can take... To encourage consistency and reinforcement; when the edge represents an exclusive / taboo relationship, it can be taken as... Punishment is applied to simultaneously high activation; when an edge represents a treatment constraint relationship, it can be adjusted according to the rules. Set as a threshold-based mapping. Therefore, when conflicts arise, such as mutually exclusive concepts being strongly supported by multimodal evidence simultaneously, or a key causal chain being broken, This will increase significantly, thereby triggering weight suppression or correction.

[0043] To ensure that the consistency verification results are truly applied to cross-modal attention allocation, this embodiment feeds back the information after semantic completion and consistency correction to the cross-modal attention mechanism in S13024, enabling the language model to prioritize modal evidence that better aligns with medical knowledge constraints during subsequent reasoning. Specifically, let the embeddings after mapping the four modalities be as follows: In the cross-modal attention of Transformer, let the query vector be... The key and value vectors are respectively Then the basic cross-modal attention can be expressed as in This is a feature dimension. To demonstrate creativity, this embodiment introduces knowledge consistency feedback into the attention logit, forming a knowledge-constrained attention force: in, This is the knowledge constraint strength coefficient. For the first The conflict penalty term corresponding to a mode can be estimated from the contribution of that mode to the knowledge consistency penalty term, for example, taking... The weight of this modality in the fusion or the activation of this modality feature on the node. The contribution coefficient is calculated by explicitly subtracting the conflict penalty term from the attention logit. This reduces the probability of a modal piece of evidence being adopted by the attention mechanism when it conflicts significantly with the knowledge constraints, thereby achieving interpretable reasoning control through conflict evidence suppression and credible evidence reinforcement.

[0044] The weight of this modality in the fusion or the activation of this modality feature on the node. The contribution coefficient is calculated by explicitly subtracting the conflict penalty term from the attention logit. This reduces the probability of a modal piece of evidence being adopted by the attention mechanism when it conflicts significantly with the knowledge constraints, thereby achieving interpretable reasoning control through conflict evidence suppression and credible evidence reinforcement.

[0045] In step S1303, the weights of textual semantic features, temporal features, image representation features, and behavioral features are dynamically allocated during the reasoning process based on the aforementioned cross-modal attention mechanism. Unlike static weighting, the weight allocation in this embodiment is dynamically generated by the attention mechanism based on the current query semantics (e.g., currently focusing on evidence related to sleep apnea or insomnia). This allows the same patient to present different evidence emphases at different reasoning stages. For example, when OSA is suspected, temporal respiratory / blood oxygenation evidence and upper airway imaging evidence are strengthened; when insomnia is suspected, textual complaints and behavioral rhythm evidence are strengthened. Furthermore, after introducing the aforementioned knowledge-constrained attention, the weight allocation will simultaneously satisfy the dual conditions of data-driven relevance and medical knowledge consistency, thereby avoiding bias caused by relying solely on data relevance.

[0046] Finally, in step S1304, the weighted modal features are fused to generate a comprehensive semantic representation. The output obtained from knowledge-constrained attention of each modality can be represented as follows: Then the comprehensive semantic representation can be expressed as in, For comprehensive semantic representation, For the final fusion weight, To score the modal importance generated by a large language model based on context, and Consistent with the foregoing, the comprehensive semantic representation generated in this embodiment not only characterizes the symptom features of sleep disorders, but also explicitly reflects the association structure between symptoms and indicators, and between images and behaviors, providing a more reliable semantic basis for the diagnostic inference in subsequent step S140 and the generation of treatment plans in step S150.

[0047] The parameters in the above formula have the following meanings: For a knowledge graph of medical knowledge with rights, For a set of nodes, Let be the set of edges. Weights representing the relationships between nodes; For the set of anchor points, For the set of candidate knowledge nodes, for The set of neighboring nodes. These are the semantic embeddings of anchor points and candidate nodes, respectively. For similarity functions; Activate the strength vector for the knowledge node; Penalties for knowledge consistency; The constraint mapping function is determined by the edge type; These are the query, key, and value matrices, respectively. For feature dimensions; This refers to the knowledge constraint strength coefficient. This is a modal conflict penalty term; For the first The output of a modality after knowledge-constrained attention; Score the modal importance; is the final fusion weight; s is the comprehensive semantic representation of the output.

[0048] Through the above implementation, this embodiment not only uses the medical knowledge structure of sleep disorders as an external knowledge source for retrieval and completion, but also directly embeds it as a quantifiable consistency constraint into the weight calculation process of cross-modal attention. This enables the reasoning of the language big model to perform interpretable conflict suppression and credible evidence enhancement among multimodal evidence, thereby improving the stability, interpretability, and clinical rationality of sleep disorder diagnosis and treatment plan generation.

[0049] In one feasible implementation, such as Figure 5 As shown, step S140 above, based on the comprehensive semantic representation and combined with the patient's past medical history and current sleep state, infers the type and severity of the sleep disorder to obtain a diagnostic result, including: S1401. Combine the above-mentioned comprehensive semantic representation with the above-mentioned patient's past medical history to determine the candidate type set of the above-mentioned sleep disorders; S1402. Based on the temporal physiological characteristics changes corresponding to the current sleep state, the above candidate type set is screened and confidence is evaluated. S1403. Based on the symptom association strength and physiological index deviation reflected in the above comprehensive semantic representation, the severity of the above sleep disorders is classified. S1404. Output the above diagnostic results, including the above-mentioned sleep disorder type and the above-mentioned severity classification.

[0050] In one feasible implementation, such as Figure 6As shown, step S1401 above performs a joint analysis of the above-mentioned comprehensive semantic representation and the above-mentioned patient's past medical history to determine the candidate type set of the above-mentioned sleep disorder, including: S14011. Based on the symptom features, physiological indicator features and their correlations represented in the above comprehensive semantic representation, construct a semantic feature vector representing the current abnormal sleep state of the patient. S14012. Perform structured analysis on the historical diagnostic information, past symptom evolution information and past treatment response information recorded in the patient's past medical history to form a medical history feature representation; S14013. Based on the preset semantic template for sleep disorder types, the semantic feature vector and the medical history feature representation are compared with the semantic template for sleep disorder types to calculate the similarity and obtain the matching score for each sleep disorder type. S14014. Combining the above matching score with the disease comorbidity and exclusive diagnostic constraints recorded in the patient's past medical history, sleep disorder types that do not meet the medical rationality constraints are eliminated. S14015. The sleep disorder types retained after the above similarity calculation and the above diagnostic constraint screening are determined as the candidate type set of the above sleep disorder.

[0051] In one feasible implementation, the aforementioned sleep disorder type semantic template is a pre-constructed structured semantic template for different sleep disorder types; like Figure 7 As shown, the above-mentioned semantic feature vector and medical history feature representation are compared with the above-mentioned sleep disorder type semantic template for similarity calculation, including: S140131. For each type of sleep disorder, construct a multi-layer semantic template structure including core symptom semantic sub-templates, physiological indicator semantic sub-templates, behavioral pattern semantic sub-templates, and typical disease course evolution semantic sub-templates. S140132. Map the symptom features, physiological indicator features and behavioral features in the above comprehensive semantic representation to the corresponding semantic sub-template space to form a multi-dimensional matching vector corresponding to the above sleep disorder type. S140133. Map the medical history features obtained from the analysis of the patient's past medical history to the typical disease course evolution semantic sub-template to characterize the degree of matching between the patient's historical state and the typical evolution path of the sleep disorder type. S140134. Based on the similarity results of the above multidimensional matching vector and the above medical history feature representation at the semantic sub-template level, a weighted fusion strategy is adopted to calculate the comprehensive matching score corresponding to each type of sleep disorder. Among them, the weight of each semantic sub-template is preset or adaptively adjusted according to the diagnostic criticality of the sleep disorder type, so that different types of sleep disorders have different discrimination emphases in the similarity calculation process.

[0052] For example, based on the comprehensive semantic representation output in step S130, temporal evidence of the patient's past medical history and current sleep state is introduced to generate and screen candidate types of sleep disorders. Confidence assessment and severity grading are then performed, ultimately outputting a structured diagnostic result. Unlike classification based solely on a single detection signal or single-modal text, this embodiment jointly models the comprehensive semantic representation of current multimodal evidence, the long-term evolution trajectory of medical history, and the temporal physiological changes of the current sleep state. This allows the diagnostic process to not only provide a type conclusion but also confidence and grading results, thus better aligning with the decision-making chain of clinical diagnosis.

[0053] First, in step S1401, the comprehensive semantic representation is jointly analyzed with the patient's past medical history to determine the candidate type set of sleep disorders. To ensure the interpretability and controllability of candidate generation, this embodiment further refines the process into S14011-S14015: In S14011, based on the symptom features, physiological indicator features, and their correlations represented by the comprehensive semantic representation, a semantic feature vector representing the current abnormal sleep state of the patient is constructed, denoted as: Where s is the comprehensive semantic representation, The feature readout operator is used to extract joint features of symptoms, indicators, and behaviors relevant to diagnosis from the comprehensive semantic representation. This is the current semantic feature vector. In this process, the past medical history is analyzed in a structured manner to form a representation of medical history characteristics, denoted as: in, This is a collection of past medical history information (historical diagnosis, symptom evolution, treatment response, etc.). This represents the structured parsing operator for medical history. This indicates the characteristics of the medical history.

[0054] Subsequently, in S14013, based on the preset semantic templates for sleep disorder types, the similarity between the current semantic feature vector and the medical history feature representation and the semantic templates for each type is calculated to obtain a matching score. To demonstrate inventiveness, the semantic templates for sleep disorder types in this embodiment adopt a multi-layered semantic sub-template structure, targeting the first... Construction of sleep-related disorders: Sub-templates for core symptoms, physiological indicators, behavioral patterns, and typical disease progression. These are respectively denoted as: In S140132-S140133, symptom features, physiological indicators, and behavioral features in the comprehensive semantic representation are mapped to corresponding sub-template spaces to obtain multi-dimensional matching vectors. Furthermore, the medical history feature representation is mapped to a typical disease progression sub-template to characterize historical evolution consistency. For ease of quantification, this embodiment calculates the similarity between the current feature and the template at each level, constructing hierarchical matching components: in, These are the symptom dimension, physiological indicator dimension, and behavioral dimension sub-vectors obtained by splitting the current semantic feature vector, respectively. For the first Template embedding of sleep disorders at the corresponding sub-template level; The similarity function is used (e.g., cosine similarity or the negative value of the learning distance). In S140134, a weighted fusion strategy is used to obtain a comprehensive matching score. To differentiate the discrimination of different sleep disorders (e.g., OSA relies more on physiological indicators and imaging evidence, while insomnia relies more on subjective complaints and behavioral rhythm evidence), this embodiment introduces a type-adaptive weighting mechanism, setting the sub-template weights as type-related vectors. It also allows it to adaptively adjust based on the current reliability of the evidence: in, For the first A comprehensive matching score for sleep disorders; The weights are for the four sub-template levels; For the first Pre-defined criticality weighting benchmarks for sleep disorders; This is the current evidence reliability vector (which can be formed by modal quality, missing rate, noise level, etc.). For weighted adaptive mapping matrix; The weights are used to normalize the weights so that their sum equals 1. This mechanism allows for an adaptive trade-off between prior typological criticality and the reliability of current evidence, thereby significantly improving the robustness and clinical rationality of template matching.

[0055] In S14014, by combining the matching score with the comorbidity and exclusionary diagnostic constraints in the patient's past medical history, types that do not meet the medical rationality constraints are eliminated. To demonstrate inventiveness, this embodiment explicitly expresses the comorbidity and exclusionary constraints as a constraint matrix. ,in Representation type With type Coexistence is allowed. This indicates that the two are mutually exclusive; at the same time, it introduces the set of previously confirmed types from the medical history. Then for the first... Class candidate type definition feasibility indicator function: when When this occurs, it indicates that the type conflicts with previously identified types and should be eliminated. Furthermore, constraint conflict penalties can be incorporated into the final candidate score: in, This is the mutual exclusion penalty coefficient. Finally, in S14015, the types that satisfy the constraints and have the highest scores are determined as the candidate type set, for example, taking: in, For candidate thresholds, This indicates selecting the top scorer. There are several types. In step S1402, the candidate type set is screened and confidence is assessed based on the temporal physiological characteristic changes corresponding to the current sleep state. This embodiment can use temporal change evidence as a secondary verification factor for candidate types, especially for distinguishing types with similar symptoms but different physiological mechanisms. Let the key change index vector extracted from the temporal physiological data be denoted as... (For example, changes in AHI, ODI, arousal index, sleep efficiency, etc.) for candidate types Define physiological consistency score: in, For the first Reference pattern vector for sleep-like disorders. Confidence level is formed based on matching score and physiological consistency score: in, Candidate type Confidence level, For the Sigmoid function, This is a weighting factor. By introducing a consistency constraint based on physiological changes, the probability of misjudgment caused by relying solely on textual claims can be effectively reduced.

[0056] In step S1403, the severity of sleep disorders is graded based on the symptom association strength and the degree of deviation of physiological indicators reflected in the comprehensive semantic representation. To ensure the grading criteria are both interpretable and scalable, this embodiment defines the severity score as a combination of symptom network strength and indicator deviation risk. Let A be the symptom association strength graph obtained from the comprehensive semantic representation, where... Indicates symptoms Symptoms The correlation strength; the deviation vector of physiological indicators is ,in Indicates the first The severity score can be defined as the degree of normalization deviation of a physiological indicator from the reference range: Where s represents the symptom set, This is a trade-off coefficient between symptom association and indicator deviation. For the first Risk weights for each physiological indicator (e.g., for OSA, increasing the weights related to AHI and ODI; for insomnia, increasing the weights related to sleep efficiency and wakefulness index). Finally, the severity score is compared with a preset grading threshold to obtain the severity level. : in, This is the threshold for grading.

[0057] Finally, in step S1404, a diagnostic result including the type and severity classification of the sleep disorder is output. In this embodiment, the output diagnostic result not only includes the final type conclusion but also a set of candidate types, the confidence level of each candidate type, and the severity score and grade. This provides structured input for generating the treatment plan in subsequent step S150, enabling a one-to-one correspondence between treatment strategy selection and intervention priority settings and the type, confidence level, and severity.

[0058] For comprehensive semantic representation; This is the current semantic feature vector; This is a collection of past medical history information; Characteristic description of medical history; For the first Multi-layered semantic templates for sleep disorders, including Four types of sub-templates; For each sub-template level, there are similarity components; A type-adaptive weight vector; Preset keyness weight benchmark; This is the evidence reliability vector; For weighted adaptive mapping matrix; A comprehensive matching score is given; For comorbidity / mutual exclusion constraint matrices; This is a collection of previously confirmed types; This is a feasibility indicator function; This is the conflict penalty coefficient; Candidate scores after considering constraints; The candidate threshold; A vector of key physiological changes over time; Reference change mode vector; Score for physiological consistency; Confidence level; For the weighting factor; This is a symptom association strength matrix; This represents the deviation vector of physiological indicators; Risk weights are used for indicators; Sev is the severity score. For the weighting factor; This is the threshold for grading; This indicates the severity level.

[0059] The inventiveness of this embodiment lies in the following aspects: In the candidate type generation stage, a multi-layered semantic template is introduced, combined with a type-criticality prior-evidence reliability adaptive weighting mechanism, enabling differentiated discrimination emphases for different sleep disorder types. In the candidate screening stage, consistency verification of current physiological temporal changes and confidence modeling are introduced, ensuring interpretable and credible output of diagnostic results. In the severity grading stage, the symptom association network strength and indicator deviation risk are fused and modeled, thereby achieving joint, controllable, and interpretable inference of sleep disorder type and severity, providing robust input for subsequent treatment plan generation.

[0060] In one feasible implementation, such as Figure 8 As shown, step S150 above, based on the diagnostic results, generates a treatment plan that matches the patient's individual characteristics through an intelligent matching model, including: S1501. Based on the type and severity classification of sleep disorders in the above diagnostic results, determine the corresponding set of candidate treatment strategies, wherein the set of candidate treatment strategies includes drug treatment strategies, non-drug intervention strategies and behavioral intervention strategies. S1502. Obtain individual constraint information related to the above-mentioned patients, wherein the above-mentioned individual constraint information includes the patient's age characteristics, past medication history, adverse reaction records, information on comorbid diseases, and compliance assessment results; S1503. Based on the above comprehensive semantic representation and the above individual constraint information, construct a treatment plan matching feature vector, and input the above treatment plan matching feature vector into the above intelligent matching model to evaluate the suitability of each treatment strategy in the above candidate treatment strategy set. S1504. In the above-mentioned fit assessment process, a multi-objective optimization strategy is adopted to generate a comprehensive score corresponding to each treatment strategy. S1505. Based on the above comprehensive score, the above candidate treatment strategies are ranked, and the treatment strategies that meet the preset safety constraints in the ranking results are selected to form the above treatment plan for the above-mentioned patients.

[0061] In one feasible implementation, such as Figure 9 As shown, in the above-mentioned fit assessment process, step S1504 employs a multi-objective optimization strategy to generate a comprehensive score corresponding to each treatment strategy, including: S15041. For each candidate treatment strategy, construct a multi-dimensional optimization objective function that includes at least the treatment effectiveness objective, the safety risk control objective, and the patient compliance objective. S15042. Based on individual patient characteristics and diagnostic results, the above-mentioned safety risk control objectives are set as hard constraints, while the above-mentioned treatment effectiveness objectives and patient compliance objectives are set as optimizable objectives. S15043. Under the premise of satisfying the above hard constraints, a multi-objective decision-making algorithm is used to jointly solve the above optimizable objectives to obtain the comprehensive evaluation results of each candidate treatment strategy in the multi-objective space. S15044. Based on the above comprehensive evaluation results, a comprehensive score is generated to characterize the overall suitability of each candidate treatment strategy, and the above comprehensive score is used as the basis for the decision-making of treatment plan selection and ranking.

[0062] For example, after obtaining the type and severity classification of the sleep disorder in step S140, the diagnostic results are jointly modeled with the patient's individual constraints. An intelligent matching model is then used to assess the suitability of candidate treatment strategies. Finally, within a multi-objective optimization framework, efficacy, risk, and adherence are comprehensively weighed to output a personalized treatment plan that meets safety constraints and is most suitable for the patient. In this way, this embodiment can not only provide recommended treatment methods but also intervention priorities and follow-up adjustment suggestions, thereby forming a treatment decision output with executable feasibility and the potential for closed-loop optimization.

[0063] Specifically, in step S1501, a set of candidate treatment strategies is determined based on the type and severity classification of the sleep disorder in the diagnostic results. This candidate set includes at least pharmacological treatment strategies, non-pharmacological intervention strategies, and behavioral intervention strategies, and can be mapped by type to classification to provide differentiated coverage for patients with different types and severity levels. For example, for obstructive sleep apnea (OSA), CPAP, orthodontic treatment, and postural therapy can be included in the non-pharmacological candidate set; for insomnia, CBT-I, sleep restriction therapy, and short-term medication when necessary can be included in the candidate set; for RLS / RBD, a combination of pharmacological and behavioral intervention strategies can be formed to create a combined candidate set.

[0064] In step S1502, individual constraint information related to the patient is obtained to avoid irrational recommendations of treating the same disease with the same medication. This individual constraint information includes age characteristics, past medication history, adverse reaction records, information on comorbidities, and adherence assessment results. Among these, information on comorbidities can be used to filter contraindicated regimens, adverse reaction records can be used to penalize high-risk drugs or interventions, and adherence assessment results are used to predict the patient's feasibility with long-term interventions (such as CPAP, CBT-I) or lifestyle changes.

[0065] In step S1503, a matching feature vector for treatment plans is constructed based on the comprehensive semantic representation and individual constraint information, and input into the intelligent matching model to evaluate the suitability of candidate treatment strategies. To ensure the interpretability and scalability of the evaluation, this embodiment defines the matching feature vector as being formed by concatenating the disease condition semantic sub-vector, the risk constraint sub-vector, and the compliance sub-vector: in, This is a comprehensive semantic representation or readout vector (representing the symptom-indicator-behavior association structure). The individual constrained feature vector (structured results such as age, comorbidities, previous medications, adverse reactions, etc.) This represents a compliance-related feature vector (sleep habit stability, persistence of behavioral risk status, completion rate of historical follow-ups, etc.). For any candidate treatment strategy... The intelligent matching model outputs its initial fit score. : in, For parameters Matching model, For strategy The strategy representation vector (which can be encoded by strategy category, indication scope, contraindication rules, execution cost, and follow-up requirements) allows the model to learn the matching rules between patient features and strategy attributes in a unified representation space, rather than simply outputting fixed category recommendations.

[0066] In step S1504, to avoid situations where recommendations based solely on a single fit score lead to high efficacy but high risk, or low risk but unfeasible implementation, this embodiment employs a multi-objective optimization strategy to generate a comprehensive score corresponding to each treatment strategy, and refines this process into S15041-S15044. Specifically, in S15041, a multi-dimensional optimization objective function is constructed for each candidate treatment strategy, encompassing treatment effectiveness, safety and risk control, and patient compliance. To highlight its inventiveness, this embodiment quantifies the three types of objectives into computable functions: efficacy objective... Risk objectives Compliance goals For example, efficacy goals can be modeled as a combination of the expected intensity of symptom improvement and the intensity of physiological indicator regression; risk goals can be modeled as a combination of contraindication conflicts, adverse reaction probabilities, and the risk of triggering comorbidities; and adherence goals can be modeled as a combination of executive burden, difficulty of behavioral change, and device dependence. For ease of unified optimization, the following form can be provided: in, Construct functions based on efficacy, risk, and compliance characteristics, respectively; For the corresponding weight parameters; A normalization function (e.g., Sigmoid) is used to map the target value to... Interval.

[0067] In S15042, the safety risk control objective is set as a hard constraint, meaning that a candidate strategy is directly deemed infeasible if it violates a taboo or the risk exceeds a threshold. Therefore, a feasibility indicator function is defined: in, The risk threshold function is designed to vary with individual patient constraints, reflecting individual differences such as stricter risk thresholds for elderly patients and patients with comorbidities. By designing the risk threshold as an individual adaptive function, it avoids over-exclusion or negligence of risk caused by a uniform threshold.

[0068] exist In this approach, under the premise of satisfying hard constraints, the efficacy and compliance goals are jointly solved to obtain a comprehensive evaluation result of each candidate treatment strategy in a multi-objective space. To demonstrate inventiveness, this embodiment employs a constrained Pareto ordination and interpretable trade-off decision-making method: starting with strategies that satisfy... Selecting Pareto optimal sets from the set of strategies in, This is a tradeoff coefficient determined by adherence characteristics. When patient adherence is poor, the preference weight for the efficacy of high-intensity interventions is reduced, while the preference weight for executability is increased. When patient adherence is good and the severity is high, the efficacy weight can be increased to prioritize improving key indicators. This mechanism of adaptive change of the tradeoff coefficient with patient characteristics enables multi-objective decision-making to have individual differences and can form an explainable rationale (e.g., insufficient adherence - prioritizing executable strategies).

[0069] In S15044, a comprehensive score is generated based on the comprehensive evaluation results to characterize the overall fit of the strategy, and this score serves as the basis for selecting and ranking treatment plans. To further integrate the initial fit score obtained in step S1503, this embodiment defines the final comprehensive score as a fusion of the matching model score and the multi-objective compromise score: in, The fusion coefficient is... It reflects data-driven matching patterns. This reflects the trade-off between efficacy and compliance under the logic of clinical decision-making; when The strategy is directly eliminated when necessary. Through this fusion mechanism, this embodiment balances model learning ability and medical decision constraints, avoiding recommendation bias caused by relying solely on either side.

[0070] Finally, in step S1505, the candidate treatment strategy set is ranked according to the comprehensive score, and the treatment strategies that meet the preset safety constraints are selected from the ranking results to form a treatment plan. Furthermore, the treatment plan can be structured to output the main strategy, auxiliary strategies, and follow-up and adjustment rules, and provide intervention priorities. For example, in cases of moderate to severe OSA with high adherence, CPAP is prioritized, supplemented by weight management follow-up; in cases of insomnia accompanied by anxiety and average adherence, a simplified CBT-I pathway is prioritized, with phased follow-up trigger conditions set; when there is a risk of prior adverse reactions, the priority of drug strategies is reduced, and alternative non-drug intervention strategies are recommended, thus forming individualized, actionable treatment recommendations.

[0071] The parameters in the above formula have the following meanings: For a comprehensive semantic representation or its readout vector; For individual constraint feature vectors; This is a compliance feature vector; Match feature vectors to treatment plans; Candidate treatment strategies The strategy representation vector; For intelligent matching models; The initial fit score for the strategy; The target value for therapeutic effect; This is the target risk value; For compliance target values; Construct a constructor for the target feature; These are weight parameters; This is a risk threshold function; This is a feasibility indicator function; This is the set of Pareto optimal policies; For individualized trade-offs; To achieve a compromise score; Score is the fusion coefficient. This is the final overall score.

[0072] Through the above implementation method, the ingenuity of this embodiment is reflected in the following: not only does it introduce the learning matching capability of the intelligent matching model in the treatment plan generation stage, but it also explicitly incorporates safety risks into the decision-making process in the form of hard constraints, and adopts Pareto ranking and individualized trade-off coefficients to achieve an adaptive trade-off between efficacy and compliance under the premise that the hard constraints are satisfied, so that the output treatment plan has clinical rationality, safety controllability and executability, and can naturally support the generation of subsequent follow-up adjustment suggestions.

[0073] The second aspect, such as Figure 10 As shown, Figure 10 This application provides a schematic diagram of a sleep disorder diagnosis and treatment plan determination system based on a large language model, which includes: The acquisition unit 21 is used to acquire sleep-related multimodal data of the subject to be diagnosed and treated, wherein the aforementioned sleep-related multimodal data includes medical record text data, time-series physiological data, medical imaging data, and patient lifestyle and behavioral pattern data. Processing unit 22 is used to perform modality alignment and feature encoding processing on the above-mentioned sleep-related multimodal data, so as to map it into a multimodal representation under a unified semantic space; The generation unit 23 is used to input the above multimodal representation into a large language model enhanced with medical domain knowledge, and use a Transformer-based cross-modal attention mechanism to generate a comprehensive semantic representation that represents the symptom features of sleep disorders and their interrelationships. Inference unit 24 is used to infer the type and severity of sleep disorder based on the above comprehensive semantic representation, combined with the patient's past medical history and current sleep state, and to obtain a diagnostic result; The matching unit 25 is used to generate a treatment plan that matches the individual characteristics of the patient based on the above diagnostic results through an intelligent matching model. The treatment plan includes treatment strategy suggestions, intervention priorities, and follow-up adjustment suggestions.

[0074] In one feasible implementation, the second aspect can also perform the steps of the method described in any one of the first aspects.

[0075] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A method for determining a sleep disorder diagnosis and treatment plan based on a large language model, characterized in that, include: Acquire sleep-related multimodal data of the subject to diagnosis and treatment, wherein the sleep-related multimodal data includes medical record text data, time-series physiological data, medical imaging data, and patient lifestyle and behavioral pattern data; The sleep-related multimodal data is subjected to modality alignment and feature encoding to map it into a multimodal representation in a unified semantic space; The multimodal representation is input into a large language model enhanced with medical domain knowledge, and a comprehensive semantic representation representing the symptom features of sleep disorders and their interrelationships is generated using a Transformer-based cross-modal attention mechanism. Based on the comprehensive semantic representation, combined with the patient's past medical history and current sleep state, the type and severity of sleep disorder are inferred to obtain a diagnostic result; Based on the diagnostic results, a treatment plan matching the patient's individual characteristics is generated through an intelligent matching model. The treatment plan includes recommendations for treatment strategies, intervention priorities, and follow-up adjustment recommendations.

2. The method for determining sleep disorder diagnosis and treatment plans based on a large language model according to claim 1, characterized in that, The modality alignment and feature encoding processing of the sleep-related multimodal data to map it into a multimodal representation in a unified semantic space includes: The medical record text data is subjected to natural language processing based on medical named entity recognition and semantic relation extraction to extract descriptions of sleep-related symptoms, diagnostic conclusions, medication records and disease progression information, and encoded into the text semantic features through word vector model or context semantic coding model; The time-series physiological data are subjected to time synchronization processing based on a sliding time window, and noise suppression is performed using a bandpass filtering algorithm to extract time-series features reflecting sleep stages, number of awakenings, and sleep continuity. The medical image data is preprocessed based on image normalization and region of interest segmentation, and image representation features corresponding to upper airway structures, brain regions and other sleep disorder-related structures are extracted using a convolutional neural network. The patient's lifestyle and behavior pattern data were modeled using a hidden Markov model to generate behavioral characteristics that reflect circadian rhythms, sleep habit stability, and behavioral risk factors. Based on the cross-modal embedding alignment strategy, the text semantic features, the temporal features, the image representation features and the behavioral features are mapped to a unified semantic embedding space, and the multimodal representation is formed by feature weighted fusion.

3. The method for determining sleep disorder diagnosis and treatment plans based on a large language model according to claim 2, characterized in that, The process involves inputting the multimodal representation into a large language model enhanced with medical domain knowledge, and utilizing a Transformer-based cross-modal attention mechanism to generate a comprehensive semantic representation characterizing the symptom features of sleep disorders and their interrelationships, including: Construct a medical domain knowledge structure that includes the relationships between sleep disorder types, symptoms, physiological indicators, imaging features, and treatment methods; The medical knowledge structure is introduced as an external knowledge constraint into the reasoning process of the language model to perform semantic completion and consistency correction on the multimodal representation. Based on the cross-modal attention mechanism, the weights of the text semantic features, the temporal features, the image representation features, and the behavioral features are dynamically allocated in the reasoning process; By fusing the weighted modal features, a comprehensive semantic representation is generated that characterizes the sleep disorder symptoms and their intrinsic relationships.

4. The method for determining sleep disorder diagnosis and treatment plans based on a large language model according to claim 3, characterized in that, In the process of introducing the medical domain knowledge structure as an external knowledge constraint into the reasoning of the language model, semantic completion and consistency correction are performed on the multimodal representation, including: The medical knowledge structure is constructed as a knowledge graph structure including nodes and edges. The nodes of the knowledge graph structure are used to represent the disease type, symptoms, physiological indicators, imaging features and treatment methods of sleep disorders, and the edges of the graph structure are used to represent the causal relationship, correlation relationship or diagnosis and treatment constraint relationship between the nodes. During the reasoning process of the large language model, based on the symptom features and physiological indicator features identified in the multimodal representation, candidate knowledge nodes associated with them are retrieved in the knowledge graph structure to complete the missing medical semantic information in the multimodal representation. Based on the association strength and constraint relationship between the nodes, the consistency of the association rationality between different modal features in the multimodal representation is checked. When a feature combination that does not conform to the constraints of medical knowledge is detected, the weight of the corresponding feature is suppressed or corrected. The multimodal representation, after semantic completion and consistency correction, is fed back into the cross-modal attention mechanism, enabling the language big model to prioritize feature information that is more relevant to the sleep disorder and conforms to medical knowledge constraints during subsequent reasoning, in order to generate the comprehensive semantic representation.

5. The method for determining sleep disorder diagnosis and treatment plans based on a large language model according to claim 1, characterized in that, Based on the comprehensive semantic representation, combined with the patient's past medical history and current sleep state, the type and severity of the sleep disorder are inferred to obtain a diagnostic result, including: The comprehensive semantic representation is jointly analyzed with the patient's past medical history to determine the set of candidate types of the sleep disorder; Based on the temporal physiological characteristic changes corresponding to the current sleep state, the candidate type set is screened and confidence is evaluated. The severity of the sleep disorder is graded based on the symptom association strength and physiological index deviation reflected in the comprehensive semantic representation. The output includes the diagnostic results, which include the type of sleep disorder and the severity classification.

6. The method for determining sleep disorder diagnosis and treatment plans based on a large language model according to claim 5, characterized in that, The step of jointly analyzing the comprehensive semantic representation with the patient's past medical history to determine the candidate type set of the sleep disorder includes: Based on the symptom features, physiological indicator features and their correlations represented in the comprehensive semantic representation, a semantic feature vector representing the current abnormal sleep state of the patient is constructed. The patient's past medical history records historical diagnostic information, past symptom evolution information, and past treatment response information are analyzed in a structured manner to form a representation of medical history features; Based on the preset semantic template of sleep disorder type, the semantic feature vector and the medical history feature representation are respectively compared with the semantic template of sleep disorder type to calculate the similarity and obtain the matching score corresponding to each sleep disorder type. Based on the matching score and the comorbidity of diseases recorded in the patient's past medical history, as well as the exclusionary diagnostic constraints, sleep disorder types that do not meet the medical rationality constraints are eliminated. The sleep disorder types retained after the similarity calculation and the diagnostic constraint screening are determined as the candidate type set of the sleep disorder.

7. The method for determining sleep disorder diagnosis and treatment plans based on a large language model according to claim 6, characterized in that, The sleep disorder type semantic template is a pre-constructed structured semantic template for different sleep disorder types; The step of calculating the similarity between the semantic feature vector and the medical history feature representation and the semantic template of the sleep disorder type includes: For each type of sleep disorder, a multi-layered semantic template structure is constructed, including semantic sub-templates for core symptoms, semantic sub-templates for physiological indicators, semantic sub-templates for behavioral patterns, and semantic sub-templates for typical disease progression. The symptom features, physiological indicator features and behavioral features in the comprehensive semantic representation are respectively mapped to the corresponding semantic sub-template space to form a multi-dimensional matching vector corresponding to the sleep disorder type; The medical history features parsed from the patient's past medical history are mapped to the typical disease course evolution semantic sub-template to characterize the degree of matching between the patient's historical state and the typical evolution path of the sleep disorder type; Based on the similarity results between the multidimensional matching vector and the medical history feature representation at the semantic sub-template level, a weighted fusion strategy is used to calculate the comprehensive matching score corresponding to each sleep disorder type. The weight of each semantic sub-template is preset or adaptively adjusted according to the diagnostic criticality of the sleep disorder type, so that different sleep disorder types have different discrimination emphases in the similarity calculation process.

8. The method for determining sleep disorder diagnosis and treatment plans based on a large language model according to claim 1, characterized in that, Based on the diagnostic results, the process of generating a treatment plan that matches the patient's individual characteristics using an intelligent matching model includes: Based on the type and severity classification of sleep disorders in the diagnostic results, a corresponding set of candidate treatment strategies is determined, wherein the set of candidate treatment strategies includes drug treatment strategies, non-drug intervention strategies, and behavioral intervention strategies; Obtain individual constraint information related to the patient to be diagnosed and treated, wherein the individual constraint information includes the patient's age characteristics, past medication history, adverse reaction records, comorbidity information, and compliance assessment results; Based on the comprehensive semantic representation and the individual constraint information, a treatment plan matching feature vector is constructed, and the treatment plan matching feature vector is input into the intelligent matching model to evaluate the suitability of each treatment strategy in the candidate treatment strategy set. During the fit assessment process, a multi-objective optimization strategy is adopted to generate a comprehensive score corresponding to each treatment strategy; The candidate treatment strategies are ranked according to the comprehensive score, and the treatment strategies that meet the preset safety constraints are selected from the ranking results to form the treatment plan for the patient.

9. The method for determining a sleep disorder diagnosis and treatment plan based on a large language model according to claim 8, characterized in that, During the fit assessment process, a multi-objective optimization strategy is employed to generate a comprehensive score corresponding to each treatment strategy, including: For each candidate treatment strategy, construct a multidimensional optimization objective function that includes at least the treatment effectiveness objective, the safety risk control objective, and the patient compliance objective; Based on individual patient characteristics and diagnostic results, the safety risk control target is set as a hard constraint, while the treatment effectiveness target and the patient compliance target are set as optimizable targets. Under the premise of satisfying the hard constraints, a multi-objective decision-making algorithm is used to jointly solve the optimizable objectives to obtain the comprehensive evaluation results of each candidate treatment strategy in the multi-objective space. Based on the comprehensive evaluation results, a comprehensive score is generated to characterize the overall suitability of each candidate treatment strategy, and the comprehensive score is used as the basis for the decision-making of treatment plan selection and ranking.

10. A system for determining a sleep disorder diagnosis and treatment plan based on a large language model, characterized in that, include: The acquisition unit is used to acquire sleep-related multimodal data of the subject to be diagnosed and treated, wherein the sleep-related multimodal data includes medical record text data, time-series physiological data, medical imaging data, and patient lifestyle and behavioral pattern data; The processing unit is used to perform modality alignment and feature encoding processing on the sleep-related multimodal data to map it into a multimodal representation in a unified semantic space; The generation unit is used to input the multimodal representation into a large language model enhanced with medical domain knowledge, and use a Transformer-based cross-modal attention mechanism to generate a comprehensive semantic representation that characterizes the symptom features of sleep disorders and their interrelationships. The inference unit is used to infer the type and severity of sleep disorder based on the comprehensive semantic representation, combined with the patient's past medical history and current sleep state, and to obtain a diagnostic result; The matching unit is used to generate a treatment plan that matches the individual characteristics of the patient based on the diagnostic results through an intelligent matching model. The treatment plan includes treatment strategy suggestions, intervention priorities, and follow-up adjustment suggestions.

Images