Dialogue guide type multi-source fusion chronic obstructive pulmonary disease management system based on cloud native architecture
By adopting cloud-native architecture and dialogue-guided multi-source fusion technology in the COPD management system, integrating the Internet of Things and natural language processing, the problems of data recording and association in traditional COPD management methods are solved, and more efficient disease management and quality of life improvement are achieved.
Patent Information
- Application Number
- CN202411702224.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-26
- Publication Date
- 2025-03-25
AI Technical Summary
Traditional COPD management methods have problems such as inconvenient data recording, difficult to correlate multiple data source islands, and missing data recording, which affects the effective analysis and diagnosis and treatment of patient data.
The dialogue-guided multi-source fusion of COPD management system based on cloud-native architecture is adopted, and a variety of mobile and home devices are integrated through technologies such as the Internet of Things, cloud computing, natural language processing and pre-training large models to achieve multi-dimensional and multi-angle data collection and tracking of COPD patients.
It improves the management level of patients with COPD, improves the work efficiency of doctors, improves the quality of life of patients, and reduces medical costs.
Smart Images
Figure CN119694594A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of computer-aided diagnosis technology, and in particular to a conversation-guided multi-source fusion chronic obstructive pulmonary disease management system based on a cloud-native architecture. Background Art
[0002] Chronic obstructive pulmonary disease (COPD) is a common, long-term, chronic respiratory inflammatory disease of the respiratory system, characterized by airflow obstruction. It has become one of the important causes of increased disability and mortality. The Lancet, an international authoritative medical journal, published a large-scale population study in China, the Chinese Adult Lung Health Study (CPH), which revealed the prevalence of COPD in China and made it clear for the first time that the number of COPD patients in China is about 100 million. COPD has become a chronic disease "on par" with hypertension and diabetes, and constitutes a major disease burden on society.
[0003] At present, in the process of clinical diagnosis and treatment, due to the short average outpatient time for each patient and the short communication time between doctors and patients in the outpatient clinic, it is difficult to collect a complete history of patient symptom changes, resulting in inaccurate disease symptoms and risk assessment. In addition, COPD is often caused by other reasons such as smoking, infection, air pollution, occupational dust, chemical pollution, etc., and these factors are also related to the acute exacerbation of COPD. In addition, the acute exacerbation of COPD is also related to changes in atmospheric pressure and improper treatment. Therefore, although COPD is an incurable disease, through proper management and follow-up, the disease can be effectively controlled, and the patient's lifestyle and disease development can be understood, so that the treatment plan can be adjusted and optimized in time, the changes in factors related to the acute exacerbation of COPD can be grasped, the risk of acute exacerbation of COPD can be predicted, and adjustments can be made in time to avoid the occurrence of acute exacerbation of COPD and avoid unnecessary medical expenses and risks. However, traditional methods mainly use methods such as issuing paper record books, which have problems such as inconvenient data recording, difficulty in linking multiple data source islands, and missing recorded data, which affects the effective analysis of patient data and the diagnosis and treatment of patients. For example, even in relatively standardized clinical studies of COPD, a review of patient symptom diaries revealed that approximately 50% of COPD exacerbations were not reported. In addition, different factors associated with acute exacerbations are difficult to record and analyze, and it is difficult for doctors to keep track of changes in patients' conditions.
[0004] Advances in technologies such as the Internet of Things, cloud computing, mobile devices, and sensors have brought new solutions to the management and follow-up of COPD patients. The development of technologies such as the Internet of Things (IoT), mobile devices, and mobile Internet have enabled COPD patients to use various types of sensors to collect biomedical data, such as oxygen concentration, heart rate, and breathing rate. Through mobile devices equipped with COPD health management applications, patients can upload biomedical data recorded by various types of sensors, as well as additional physiological data such as respiratory symptoms, medication use, and exercise volume, as well as information such as location, to the cloud platform. Through cloud computing and Internet technology, the cloud platform can form an intelligent system with these sensors and mobile devices. This system can collect a large amount of biomedical data and analyze the collected information and data using machine learning algorithms or other analytical methods. Based on this system, doctors can detect the patient's physiological state in real time, so as to adjust the treatment plan or make emergency treatment in time.
[0005] The present invention proposes a COPD patient monitoring and management system based on technologies such as the Internet of Things, cloud computing, natural language processing, and pre-trained large models. The platform includes two components: a cloud platform and a patient local end. The cloud platform is based on a cloud-native architecture, and the system is based on the conversation-guided COPD management natural language conversation large model proposed in this patent and a multi-source data modal conversion and fusion method based on contrastive learning and a conversation-guided large model, to achieve analysis of various types of daily monitoring information of patients collected at the patient local end, and to achieve real-time monitoring of the physical condition of COPD patients, emergency reporting, self-management, and support for telemedicine services. Summary of the invention
[0006] The present invention first proposes a conversation-guided multi-source fusion COPD management system based on a cloud-native architecture, which consists of two parts: the patient's local end and the COPD management cloud platform:
[0007] The patient local end includes an Internet of Things patient monitoring module, a natural language dialogue module, and a data summary transmission module: wherein the Internet of Things patient monitoring module collects data on various risk factors related to COPD in the patient's daily life; the natural language dialogue module and the data summary transmission module are deployed on the patient's mobile phone in the form of a mobile APP; the natural language dialogue module communicates with the patient in the form of a dialogue, and collects the patient's daily activities, symptom manifestations, and medication monitoring information through dialogue questions or in the form of questionnaire forms during the communication and dialogue process; the patient data collected by the patient monitoring module and the natural language dialogue module will be transmitted to the COPD management cloud platform through the data summary transmission module; the natural language dialogue module is combined with the COPD management cloud platform to support interaction with the patient in the form of a dialogue and provide remote auxiliary medical services, answer questions raised by the patient, provide treatment suggestions, assist in remote follow-up, and can analyze the patient's monitoring data and patient baseline data based on the multi-source data modal conversion and fusion method and the dialogue-guided COPD management large model ChatCOPD deployed on the cloud platform, predict the patient's disease development trend, determine the risk level of acute exacerbation of COPD, and give corresponding suggestions and warnings;
[0008] The COPD management cloud platform evaluates the patient's baseline condition, establishes the patient's health record, and provides a personalized COPD management plan; at the same time, the cloud platform integrates and analyzes the patient's various medical data and daily monitoring data, promptly detects changes in the patient's condition, determines the probability of acute exacerbation emergencies, and provides doctors and patients with more comprehensive and accurate patient condition assessments and COPD management and treatment recommendations; the cloud platform adopts a cloud-native architecture design, combined with Serverless architecture, containerization technology and microservice architecture, and consists of a cloud database, a conversation-guided COPD management model ChatCOPD, a multi-source data modality conversion and fusion module, a patient baseline condition assessment module, and a patient monitoring and evaluation module; among which, the multi The source data modal conversion and fusion module is based on the multi-source data modal conversion and fusion method based on contrastive learning and dialogue-guided large model; the patient baseline condition assessment module calls the ChatCOPD model to analyze the patient baseline data and forms a patient baseline condition analysis report; the patient monitoring and assessment module is based on the multi-source data modal conversion and fusion module, and unifies the patient's daily activities, symptom manifestations, and medication monitoring information from the patient's local end including sequence and text modalities, as well as the assessment result report of the patient baseline assessment module into text modal data and realizes data fusion, and calls the dialogue-guided COPD management large model ChatCOPD to analyze the fused text data to realize symptom monitoring and evaluation, and acute exacerbation prediction and warning.
[0009] A conversation-guided COPD management model, ChatCOPD. The conversation-guided COPD management model, ChatCOPD, realizes the application of conversational artificial intelligence in the field of COPD management: the model is based on a pre-trained large language model, and is retrained on data such as patient medical records, medical history, symptom descriptions, treatment plans, follow-up and monitoring data, and other data on the entire process of COPD patient visits, treatments, and disease management, as well as doctor-patient Q&A texts, COPD-related literature, and diagnosis and treatment specifications, and fine-tuned in the form of conversation. As a natural language conversation model, the ChatCOPD model has good human-computer interaction capabilities, can communicate with patients in the form of natural language conversations, answer questions raised by patients, and provide COPD patients with personalized, accurate, and reliable diagnosis and treatment recommendations and health management plans, and can collect information such as patient symptoms, medication status, and somatic symptoms during conversations with patients. In addition, through training on a large number of COPD-related texts, the ChatCOPD model has a wealth of knowledge in the diagnosis, treatment and management of COPD, and thus has the ability to comprehensively analyze the condition of COPD patients: on the one hand, the ChatCOPD model can intelligently analyze and evaluate the severity of the patient's current condition based on the provided patient's basic information, signs, symptoms, electronic medical records, smoking history, historical examination results (text records of pulmonary function test results and CT and other imaging examination results, etc.), historical follow-up and medical records, treatment plans and other medical record data, establish the patient's health file, and provide patients with more accurate treatment. On the other hand, it can comprehensively analyze the information such as symptoms, medication, and physical symptoms collected through patient conversations, as well as the patient's daily activities, environmental information, and physical sign data collected through testing equipment, so as to achieve real-time monitoring and condition assessment of patients, timely discover changes in patients' conditions, quickly identify signs of worsening of patients' conditions, and achieve early warning of emergencies such as acute exacerbation of COPD, and provide patients with intervention and emergency treatment measures to reduce the risk of acute exacerbation of patients and improve the emergency response and treatment capabilities for emergencies such as acute exacerbation. Through this model, patients can more easily obtain disease-related knowledge, improve their self-management ability, and improve their quality of life. The ChatCOPD model can also work closely with medical professionals to provide doctors with auxiliary diagnosis and treatment suggestions, reduce the workload of doctors, improve the efficiency of medical services, and improve the quality of medical services. At the same time, through the analysis and learning of a large amount of case data, the model can continuously optimize and improve its own disease knowledge and management strategies, and further enhance its diagnosis and treatment capabilities for COPD. This model can be applied to all aspects of COPD management, including but not limited to symptom identification, disease diagnosis, treatment plan formulation, efficacy evaluation, patient condition tracking and evaluation, acute exacerbation prediction and warning, etc.
[0010] On this basis, the present invention proposes a multi-source data modality conversion and fusion method based on contrastive learning and a dialogue-guided large model, which converts the baseline conditions and daily monitoring conditions of COPD patients covering multiple modalities such as time series data and text data and multiple data sources into a unified text modality, and realizes the aggregation of multi-source data in the form of text, so that the multi-source multi-modal patient baseline and monitoring data such as the patient's baseline condition, daily monitoring data, daily symptom assessment, medication records, environmental information, etc. can be analyzed and evaluated using the dialogue-guided COPD management large model ChatCOPD, and the patient's disease progression and acute exacerbation can be judged and predicted. The multi-source data modality conversion and fusion method includes two components: a modality conversion method and a multi-source text fusion method.
[0011] The modality conversion part includes two stages: sequence-text comparison pre-training (Sequence CLIP) and sequence data text conversion (SeqCLIPCap).
[0012] The sequence-text contrastive pre-training (Sequence CLIP) is based on contrastive learning and cross-modal modality alignment (CLIP) and relies on sequence data and its annotations and report description text datasets. The Sequence CLIP method includes a text data encoder Enc Text , a sequence data encoder Enc Seq , and a contrastive learning module for feature alignment between the two encoders. The sequence data and its annotations and report description text datasets will be aligned in batches. The purpose of alignment is to make the feature pairs from the same sequence data-text pair have higher similarity. For each batch of sequence data Seq and text data Text, they will be converted into feature vectors F by the corresponding encoders. S and F T :
[0013] F S =Enc Seq (Seq),
[0014] F T =Enc Text (Text),
[0015] In order to achieve feature alignment, it is necessary to calculate the cosine similarity between each feature pair. Specifically, for the feature vector F S The i-th and F T The jth element in Their cosine similarity calculation formula is as follows:
[0016]
[0017] Among them, cosine_similarity is the formula for calculating cosine similarity. Before calculating cosine similarity, the feature vectors need to be normalized by L2 norm so that their modulus lengths are the same. Therefore, the formula can be further expanded as:
[0018]
[0019] Among them, ||x||2 represents the L2 norm of vector x. By calculating the cosine similarity between all feature pairs, a similarity matrix S can be obtained, where S(i,j) represents the i-th feature vector in the sequence data and the jth feature vector in the text data The similarity between .
[0020] In order to achieve the main goal of the Sequence CLIP cross-modal alignment, that is, the feature pairs from the same sequence data-text pair have a higher similarity, it is necessary to calculate a symmetric cross entropy loss function, that is, to calculate the cross entropy loss on the rows and columns of the similarity matrix S separately. First, for the cross entropy loss of logits on the rows, the following formula can be used to calculate:
[0021] labels=arange(n),
[0022] loss i =cross_entropy_loss(S,labels,axis=0),
[0023] Among them, cross_entropy_loss is the cross entropy loss function, and axis=0 means that the cross entropy loss is calculated for each column of S. labels is a vector from 0 to n, which means that the value on the diagonal of the similarity matrix should be the largest in the same column (and the same row later).
[0024] Next, for the cross entropy loss of logits on the column, the following formula can be used to calculate:
[0025] loss t =cross_entropy_loss(S,labels,axis=1)
[0026] Among them, axis = 1 means that the cross entropy loss is calculated for each row of the similarity matrix S. Finally, the symmetric loss function is the average of the cross entropy losses of the similarity matrix S on the rows and columns, that is:
[0027] loss=(loss i+loss t ) / 2
[0028] By minimizing this symmetric loss function, the parameters of the model can be optimized to achieve the task of cross-modal alignment between sequence data and its description text. The main purpose of the sequence-text comparative pre-training (Sequence CLIP) is to obtain a text data encoder and a sequence data encoder with a corresponding relationship. The sequence data encoder and the text data encoder respectively perform feature encoding on a section of sequence data and its corresponding text description, and there is a corresponding relationship between the obtained sequence feature encoding and the text feature encoding.
[0029] In the sequence data text conversion method (SeqCLIPCap) of the modality conversion part, the sequence data encoder obtained by the Sequence CLIP method is used to encode the sequence data to obtain the sequence CLIP feature F S The sequence data text conversion method is based on the CLIPCap method, but is different from the CLIPCap method for realizing image-to-text modality conversion in that the sequence data text conversion method converts the sequence CLIP feature F S The average, maximum, and minimum values of the sequence data obtained by statistical analysis methods p1, p2, ..., p n Combined with sequence CLIP and sequence statistical analysis features, the sequence description text is generated, thereby achieving a more accurate and reliable sequence data text mode conversion that can better reflect the original sequence data information. S Through a mapping model (MappingTransformer) mapped to the text mode, the feature sequence f1,f2,…,f in the text mode is obtained k , and the sequence statistical description information p1,p2,…,p n Combined with the autoregressive language model (such as GPT-2), it is converted into a text segment.
[0030] Given a time series x1,x2,…,x t , and its corresponding description text paragraph character sequence c1,c2,…,c l (All text sequences are completed to the same length, denoted as l), and the training objective of the sequence data text conversion method is:
[0031]
[0032] Where θ represents all trainable parameters in the model (including the parameters in the mapping model Mapping Transformer and the autoregressive language model), and i=1,…,N represents the i-th sequence-description text pair in the training data. The main purpose of the sequence data text conversion method is to utilize the cross-modal alignment capability of the Sequence CLIP to extract relevant features from the sequence data, and combine them with the basic information and statistical information of the sequence, and use the sequence CLIP features, sequence statistical information, and sequence basic information as prefix inputs of the autoregressive language model to generate subsequent text descriptions in an autoregressive manner. Therefore, the training objectives of the sequence data text conversion method can be refined as follows:
[0033]
[0034] Through the above modal conversion method, daily monitoring sequence data collected by wearable devices, IoT sensors, monitoring equipment and treatment instruments can be converted into corresponding descriptive texts. On this basis, in order to realize the fusion of daily monitoring data from different sources, as well as multi-source data such as patient baseline condition assessment, patient daily symptom assessment, medication records, etc., the multi-source text fusion method is based on the ChatCOPD large model, submits the multi-source text data to the COPD model for processing, and instructs the model to extract key information related to the patient's disease symptoms, disease progression, etc. from all different types and sources of text, generate a patient condition assessment report, and complete tasks such as patient baseline condition assessment and patient daily monitoring condition assessment.
[0035] Based on the conversation-guided COPD management model ChatCOPD and the multi-source data modality conversion and fusion method, the present invention proposes a conversation-guided multi-source fusion COPD management system based on a cloud-native architecture, which consists of a patient local terminal and a cloud service platform:
[0036] The patient's local end includes an IoT patient monitoring module, a natural language dialogue module, and a data summary transmission module. Among them, the IoT patient monitoring module includes various monitoring sensors, wearable devices, auxiliary treatment instruments, etc., which collect data on various risk factors related to COPD in the patient's daily life; the natural language dialogue module and the data summary transmission module are deployed on the patient's mobile phone in the form of a mobile APP, wherein the natural language dialogue module communicates with the patient in the form of a dialogue, and in the process of the communication dialogue, collects monitoring information such as the patient's daily activities, symptom manifestations, and medication status through dialogue questions or in the form of questionnaires. The patient data collected by the patient monitoring module and the natural language dialogue module will be transmitted to the COPD management cloud platform through the data summary transmission module. The natural language dialogue module is combined with the COPD management cloud platform to support interaction with patients in the form of dialogue and provide remote assisted medical services, answer questions raised by patients, provide treatment suggestions, assist in remote follow-up, etc., and can analyze the patient's monitoring data and patient baseline data based on the multi-source data modal conversion and fusion method and the dialogue-guided COPD management model ChatCOPD, predict the patient's disease development trend, determine the risk level of acute exacerbation of COPD, and give corresponding suggestions and warnings.
[0037] The COPD management cloud platform evaluates the patient's baseline condition, establishes the patient's health record, and provides a personalized COPD management plan; at the same time, the cloud platform integrates and analyzes the patient's various medical data and daily monitoring data, timely detects changes in the patient's condition, determines the probability of emergency situations such as acute exacerbation, and provides doctors and patients with more comprehensive and accurate patient condition assessments and COPD management and treatment recommendations. The cloud platform adopts a cloud-native architecture design, combining Serverless architecture, containerization technology and microservice architecture, and is composed of a cloud database, a conversation-guided COPD management model ChatCOPD, a multi-source data modality conversion and fusion module, a patient baseline condition assessment module, and a patient monitoring and assessment module. Among them, the multi-source data modal conversion and fusion module is based on the multi-source data modal conversion and fusion method based on contrastive learning and dialogue-guided large model; the patient baseline condition assessment module calls the ChatCOPD model to analyze the patient baseline data and form a patient baseline condition analysis report; the patient monitoring and evaluation module is based on the multi-source data modal conversion and fusion module, and the patient's daily activities, symptom manifestations, medication status and other monitoring information from the patient's local end in modalities such as sequences and texts, as well as the evaluation result report of the patient baseline assessment module are unified into text modal data and data fusion is realized, and the dialogue-guided COPD management large model ChatCOPD is called to analyze the fused text data to realize symptom monitoring and evaluation, acute exacerbation prediction and warning, etc.
[0038] The data collection devices and related data types covered by the patient local end of the cloud platform include:
[0039] Wearable devices: Bracelets for monitoring daily activities and vital signs of COPD patients, which collect exercise data such as the number of steps, exercise distance, exercise duration, and exercise intensity of patients, as well as vital sign data such as real-time heart rate and blood oxygen data around the clock, as well as the patient's breathing rate and breathing effort when the patient is in a resting state such as sleep and with less interference from exercise;
[0040] IoT sensors: including but not limited to indoor air quality sensors, which measure the carbon dioxide concentration and suspended particulate matter index of indoor air;
[0041] Monitoring equipment and instruments: including sleep monitors, oxygen concentrators and ventilators, etc., which continuously measure the patient's sleep quality, sleep time and sleep apnea index, blood oxygen changes, as well as the operation status and operating parameters of the oxygen concentrator or ventilator, and combine them with the patient's physiological data;
[0042] Text and form data: collect information such as patient symptoms, medication, and physical symptoms.
[0043] The technical effects to be achieved by the present invention are:
[0044] The primary feature of the system described in the present invention in the management of COPD is the integration of multiple mobile and home devices, incorporating multiple monitoring devices and monitoring information into the assessment and tracking of the condition of COPD patients, and realizing multi-dimensional and multi-angle data collection and tracking of COPD patients, which helps to improve the level of management of patients' conditions, improve the work efficiency of doctors, improve the treatment effect of COPD patients and the quality of life of patients, and reduce medical costs.
[0045] The system described in the present invention utilizes technologies such as wearable devices, the Internet of Things, and sensors to monitor toxic gases or particulate matter inhaled by patients, helping patients to promptly detect changes in indoor and outdoor air quality and environment, and collects physiological data such as heart rate, breathing rate, blood oxygen, etc. during patient activities and sleep, thereby helping patients and doctors understand changes in the patient's physical condition.
[0046] The system of the present invention uses a COPD patient management mobile application to help patients monitor their physiological data and collect information such as patient sensations, symptoms, medication, etc. The mobile application can also understand the location information of the patient's area and query the current and future predicted air quality changes in the patient's area based on the location information. In addition, the system is based on the conversation-guided COPD management big model ChatCOPD proposed in the present invention, and the multi-source data modal conversion and fusion method based on contrastive learning and the conversation-guided big model. It is trained through a large amount of COPD-related data. It can provide patients with personalized diagnosis and treatment plan recommendations based on the patient's symptom description, combined with multi-source and multi-modal information such as medical records, monitoring data, and environmental factors, as well as services such as disease tracking and acute exacerbation prediction and warning. The system adopts a cloud-native architecture design, combined with Serverless architecture, containerization technology and microservice architecture, so that the system can automatically manage the underlying computing resources and achieve high availability and elastic scalability of the system. The system covers different types of multi-source heterogeneous and multi-modal data, including baseline conditions such as patient electronic medical records, daily monitoring data such as blood oxygen and respiratory monitoring, daily symptom assessments such as wheezing and dyspnea, medication records, and environmental records such as air quality, which can achieve comprehensive management and monitoring of COPD patients.
[0047] The cloud service platform of the platform described in the present invention partially integrates patient baseline condition assessment, disease treatment plan recommendation and management plan planning, patient condition tracking monitoring and assessment, acute exacerbation risk prediction, remote assisted medical services, etc., and adopts cloud native architecture design, combined with Serverless architecture, containerization technology and microservice architecture, so that it can automatically manage the underlying computing resources, and achieve high availability and elastic scalability of the system. The system covers different types of data from multiple sources, heterogeneous and multimodal, including baseline conditions such as patient electronic medical records, daily monitoring data such as blood oxygen and respiratory monitoring, daily symptom assessments such as wheezing and dyspnea, medication records, environmental records such as air quality, etc., which can achieve comprehensive management and monitoring of COPD patients, and through the collected patient information, the COPD management large model ChatCOPD is used for comprehensive analysis to determine the risk of acute exacerbation of COPD in patients. When the risk of acute exacerbation increases and the patient has factors that are not conducive to the treatment and management of COPD, timely warnings are issued to patients and doctors, so as to achieve early intervention and try to avoid the occurrence of acute exacerbation of COPD. In addition, the cloud service platform, in conjunction with the patient's local mobile application, can provide services such as online patient follow-up, telemedicine services, and virtual doctor consultations, thereby improving the level of treatment and disease management for COPD patients and improving the efficiency of medical resource utilization. BRIEF DESCRIPTION OF THE DRAWINGS
[0048] Figure 1A conversation-guided, multi-source fusion COPD management system architecture based on cloud-native architecture;
[0049] Figure 2 The architecture of multi-source data modality conversion method based on contrastive learning; DETAILED DESCRIPTION
[0050] The following is a preferred embodiment of the present invention and combined with the accompanying drawings to further describe the technical solution of the present invention, but the present invention is not limited to this embodiment.
[0051] The present invention proposes a conversation-guided multi-source fusion chronic obstructive pulmonary disease management system based on a cloud-native architecture.
[0052] Overall system design:
[0053] The system architecture design is as follows Figure 1 As shown in the figure, it consists of a local patient terminal and a cloud service platform. The local patient terminal includes an IoT patient monitoring module, a natural language dialogue module, and a data aggregation and transmission module; the cloud service platform consists of a cloud database, a large dialogue-guided COPD management model ChatCOPD, a multi-source data modality conversion and fusion module, a patient baseline condition assessment module, and a patient monitoring and assessment module.
[0054] The patient's local end includes an IoT patient monitoring module, a natural language dialogue module, and a data summary transmission module. Among them, the IoT patient monitoring module includes various monitoring sensors, wearable devices, auxiliary treatment instruments, etc., which collect data on various risk factors related to COPD in the patient's daily life; the natural language dialogue module and the data summary transmission module are deployed on the patient's mobile phone in the form of a mobile APP, wherein the natural language dialogue module communicates with the patient in the form of a dialogue, and in the process of the communication dialogue, collects monitoring information such as the patient's daily activities, symptom manifestations, and medication status through dialogue questions or in the form of questionnaires. The patient data collected by the patient monitoring module and the natural language dialogue module will be transmitted to the COPD management cloud platform through the data summary transmission module. The natural language dialogue module is combined with the COPD management cloud platform to support interaction with patients in the form of dialogue and provide remote assisted medical services, answer questions raised by patients, provide treatment suggestions, assist in remote follow-up, etc., and can analyze the patient's monitoring data and patient baseline data based on the multi-source data modal conversion and fusion method and the dialogue-guided COPD management model ChatCOPD, predict the patient's disease development trend, determine the risk level of acute exacerbation of COPD, and give corresponding suggestions and warnings.
[0055] The COPD management cloud platform evaluates the patient's baseline condition, establishes the patient's health record, and provides a personalized COPD management plan; at the same time, the cloud platform integrates and analyzes the patient's various medical data and daily monitoring data, timely detects changes in the patient's condition, determines the probability of emergency situations such as acute exacerbation, and provides doctors and patients with more comprehensive and accurate patient condition assessments and COPD management and treatment recommendations. The cloud platform adopts a cloud-native architecture design, combining Serverless architecture, containerization technology and microservice architecture, and is composed of a cloud database, a conversation-guided COPD management model ChatCOPD, a multi-source data modality conversion and fusion module, a patient baseline condition assessment module, and a patient monitoring and assessment module. Among them, the multi-source data modal conversion and fusion module is based on the multi-source data modal conversion and fusion method based on contrastive learning and dialogue-guided large model; the patient baseline condition assessment module calls the ChatCOPD model to analyze the patient baseline data and form a patient baseline condition analysis report; the patient monitoring and evaluation module is based on the multi-source data modal conversion and fusion module, and the patient's daily activities, symptom manifestations, medication status and other monitoring information from the patient's local end in modalities such as sequences and texts, as well as the evaluation result report of the patient baseline assessment module are unified into text modal data and data fusion is realized, and the dialogue-guided COPD management large model ChatCOPD is called to analyze the fused text data to realize symptom monitoring and evaluation, acute exacerbation prediction and warning, etc.
[0056] The data collection devices and related data types covered by the patient local end of the multi-source fusion COPD management system include:
[0057] Wearable devices: Bracelets for monitoring daily activities and vital signs of COPD patients, which collect exercise data such as the number of steps, exercise distance, exercise duration, and exercise intensity of patients, as well as vital sign data such as real-time heart rate and blood oxygen data around the clock, as well as the patient's breathing rate and breathing effort when the patient is in a resting state such as sleep and with less interference from exercise;
[0058] IoT sensors: including but not limited to indoor air quality sensors, which measure the carbon dioxide concentration and suspended particulate matter index of indoor air;
[0059] Monitoring equipment and instruments: including sleep monitors, oxygen concentrators and ventilators, etc., which continuously measure the patient's sleep quality, sleep time and sleep apnea index, blood oxygen changes, as well as the operation status and operating parameters of the oxygen concentrator or ventilator, and combine them with the patient's physiological data;
[0060] Text and form data: collect information such as patient symptoms, medication, and physical symptoms.
[0061] Among them, the dialogue-guided COPD management model ChatCOPD realizes the application of conversational artificial intelligence in the field of COPD management: the model is based on a pre-trained large language model, and is retrained on the full process data of COPD patients' visits, treatment and disease management, such as patient medical records, medical history, symptom descriptions, treatment plans, follow-up and monitoring data, as well as doctor-patient dialogue question and answer texts, COPD-related literature and diagnosis and treatment specifications, and fine-tuned in the form of dialogue. As a natural language dialogue large model, the ChatCOPD model has good human-computer interaction capabilities, can communicate with patients in the form of natural language dialogue, answer questions raised by patients, and provide COPD patients with personalized, accurate and reliable diagnosis and treatment recommendations and health management plans, and can collect information such as patients' symptoms, medication status, and physical symptoms in conversations with patients. In addition, through training on a large number of COPD-related texts, the ChatCOPD model has a wealth of knowledge in the diagnosis, treatment and management of COPD, and thus has the ability to comprehensively analyze the condition of COPD patients: on the one hand, the ChatCOPD model can intelligently analyze and evaluate the severity of the patient's current condition based on the provided patient's basic information, signs, symptoms, electronic medical records, smoking history, historical examination results (text records of pulmonary function test results and CT and other imaging examination results, etc.), historical follow-up and medical records, treatment plans and other medical record data, establish the patient's health file, and provide patients with more accurate treatment. On the other hand, it can comprehensively analyze the information such as symptoms, medication, and physical symptoms collected through patient conversations, as well as the patient's daily activities, environmental information, and physical sign data collected through testing equipment, so as to achieve real-time monitoring and condition assessment of patients, timely discover changes in patients' conditions, quickly identify signs of worsening of patients' conditions, and achieve early warning of emergencies such as acute exacerbation of COPD, and provide patients with intervention and emergency treatment measures to reduce the risk of acute exacerbation of patients and improve the emergency response and treatment capabilities for emergencies such as acute exacerbation. Through this model, patients can more easily obtain disease-related knowledge, improve their self-management ability, and improve their quality of life. The ChatCOPD model can also work closely with medical professionals to provide doctors with auxiliary diagnosis and treatment suggestions, reduce the workload of doctors, improve the efficiency of medical services, and improve the quality of medical services. At the same time, through the analysis and learning of a large amount of case data, the model can continuously optimize and improve its own disease knowledge and management strategies, and further enhance its diagnosis and treatment capabilities for COPD. This model can be applied to all aspects of COPD management, including but not limited to symptom identification, disease diagnosis, treatment plan formulation, efficacy evaluation, patient condition tracking and evaluation, acute exacerbation prediction and warning, etc.
[0062] The COPD management cloud platform of the COPD management system adopts cloud-native architecture design, combined with Serverless architecture, containerization technology and microservice architecture, so that the system can automatically manage the underlying computing resources and achieve high availability and elastic scalability of the system. The cloud platform includes five components: cloud database, conversation-guided COPD management model ChatCOPD, multi-source data modality conversion and fusion module, patient baseline condition assessment module, and patient monitoring and assessment module.
[0063] Among them, the cloud database uses cloud storage and database technology, and the platform can achieve safe storage, effective management and rapid access to large amounts of data such as patient medical records, examination results and treatment plans;
[0064] The dialogue-guided COPD management model ChatCOPD is based on deep learning technology and natural language processing technology. On the one hand, it realizes the coordination between the front-end and the back-end with the natural language dialogue module in the patient local end of the COPD management system, conducts dialogues with patients, guides them to provide basic information, symptoms, medication status, physical symptoms and other information, and intelligently identifies and analyzes the patient's condition, and provides corresponding management suggestions and treatment plans; on the other hand, it provides support for other services in the cloud platform, such as patient baseline condition assessment, patient monitoring and evaluation, etc., and realizes functions such as intelligent analysis of COPD patient information through the natural language big model in a specific vertical field (COPD management field). The ChatCOPD module also uses containerization technology to encapsulate the model into a container, so that it can run in different computing environments and expand on demand, with high portability and flexibility.
[0065] The multi-source data modality conversion and fusion module is based on the multi-source data modality conversion and fusion method based on contrastive learning and dialogue-guided large models, converts all types of data under various data modalities covered by this system into text format, and realizes the fusion of multi-source data in text mode. The multi-source data modality conversion and fusion module adopts Serverless architecture, which can automatically manage the underlying computing resources to achieve high availability and elastic scalability; and adopts microservice architecture, the cloud platform decomposes complex large modules into multiple small modules, realizes modular design, and facilitates system maintenance and upgrades. The multi-source data modal conversion and fusion module provides support for the subsequent patient baseline condition assessment module and patient monitoring and assessment module: for the patient baseline condition assessment task, the multi-source data modal conversion and fusion module converts the patient's basic information, signs, symptoms, electronic medical records, smoking history, imaging examination results, pulmonary function test results, historical follow-up and medical records, treatment plans and other patient baseline conditions into text and realizes multi-source information aggregation for analysis by the patient baseline condition assessment module; for the patient monitoring record analysis and symptom assessment tasks, the multi-source data modal conversion and fusion module converts the patient's daily test data such as exercise data, heart rate, blood oxygen, respiratory monitoring, sleep monitoring, indoor air quality, daily condition assessments such as coughing, sputum, and dyspnea, medication records, and daily test data such as air quality in the area into text and realizes multi-source information fusion under a unified text modality for analysis by the patient monitoring and assessment module.
[0066] The patient baseline condition assessment module calls the multi-source data modality conversion and fusion module to convert the collected patient basic information, medical history, physical examination results and other data into text modality, and generates a generalized and summarized standardized text description through text information extraction and aggregation, and calls the dialogue-guided COPD management model ChatCOPD, combines the medical knowledge base, clinical guidelines, historical patient analysis results, etc., to evaluate and analyze the patient's condition, generate a patient baseline condition assessment report, provide doctors with a comprehensive patient condition assessment, and provide the patient monitoring and assessment module with the patient's basic condition assessment results, and provide basic data for the subsequent treatment plan formulation. This module adopts the Serverless architecture, which can automatically manage the underlying computing resources to achieve high availability and elastic scaling.
[0067] The patient monitoring and evaluation module calls the multi-source data modality conversion and fusion module to convert daily monitoring data (including but not limited to motion data, heart rate, blood oxygen, respiratory monitoring, etc. collected by wearable devices, indoor air quality, temperature and humidity, etc. collected by IoT sensors, sleep monitoring conditions, ventilator / oxygen generator operation and patient testing conditions, etc. collected by detection equipment and treatment instruments), patient daily symptom assessment (including but not limited to coughing, sputum, dyspnea, etc.), medication records, air quality in the area, and other multi-source multi-modal data containing multiple different modalities such as text and time series data into a unified text modality, and generate a patient daily monitoring situation description report through text summary and generalization. The obtained daily monitoring situation description report is together with the patient baseline condition assessment report obtained by the patient baseline condition assessment module, and is analyzed through the dialogue-guided COPD management model ChatCOPD, in combination with the medical knowledge base, clinical guidelines, historical patient analysis results, etc., to analyze the patient's daily condition changes, promptly detect abnormal conditions of the patient's condition, and predict in advance the occurrence of emergencies such as acute exacerbation of COPD. When the model determines that an acute exacerbation may occur, an alarm is sent to the patient and his or her doctor, and intervention and emergency treatment suggestions are provided to the patient.
[0068] like Figure 2 As shown, the multi-source data modality conversion and fusion method includes two components: a modality conversion method and a multi-source text fusion method, wherein the modality conversion part includes two stages: sequence-text contrastive pre-training (SequenceCLIP) and sequence data text conversion. The sequence-text contrastive pre-training (Sequence CLIP) is based on contrastive learning and cross-modal modality alignment algorithm (CLIP), and relies on sequence data and its annotations and report description text datasets. The Sequence CLIP method includes a text data encoder Enc Text , a sequence data encoder Enc Seq , and a contrastive learning module for feature alignment between the two encoders. The sequence data and its annotations and report description text datasets will be aligned in batches. The purpose of alignment is to make the feature pairs from the same sequence data-text pair have higher similarity. For each batch of sequence data Seq and text data Text, they will be converted into feature vectors F by the corresponding encoders. S and F T :
[0069] F S =Enc Seq (Seq),
[0070] F T =Enc Text (Text),
[0071] In order to achieve feature alignment, it is necessary to calculate the cosine similarity between each feature pair. Specifically, for the feature vector F S The i-th and F T The jth element in Their cosine similarity calculation formula is as follows:
[0072]
[0073] Among them, cosine_similarity is the formula for calculating cosine similarity. Before calculating cosine similarity, the feature vectors need to be normalized by L2 norm so that their modulus lengths are the same. Therefore, the formula can be further expanded as:
[0074]
[0075] Among them, ||x||2 represents the L2 norm of vector x. By calculating the cosine similarity between all feature pairs, a similarity matrix S can be obtained, where S(i,j) represents the i-th feature vector in the sequence data and the jth feature vector in the text data The similarity between .
[0076] In order to achieve the main goal of the Sequence CLIP cross-modal alignment, that is, the feature pairs from the same sequence data-text pair have a higher similarity, it is necessary to calculate a symmetric cross entropy loss function, that is, to calculate the cross entropy loss on the rows and columns of the similarity matrix S separately. First, for the cross entropy loss of logits on the rows, the following formula can be used to calculate:
[0077] labels=arange(n),
[0078] loss i =cross_entropy_loss(S, labels, axis=0),
[0079] Among them, cross_entropy_loss is the cross entropy loss function, and axis=0 means that the cross entropy loss is calculated for each column of S. labels is a vector from 0 to n, which means that the value on the diagonal of the similarity matrix should be the largest in the same column (and the same row later).
[0080] Next, for the cross entropy loss of logits on the column, the following formula can be used to calculate:
[0081] loss t=cross_entropy_loss(S, labels, axis=1)
[0082] Among them, axis = 1 means that the cross entropy loss is calculated for each row of the similarity matrix S. Finally, the symmetric loss function is the average of the cross entropy losses of the similarity matrix S on the rows and columns, that is:
[0083] loss=(loss i +loss t ) / 2
[0084] By minimizing this symmetric loss function, the parameters of the model can be optimized to achieve the task of cross-modal alignment between sequence data and its description text. The main purpose of the sequence-text comparative pre-training (Sequence CLIP) is to obtain a text data encoder and a sequence data encoder with a corresponding relationship. The sequence data encoder and the text data encoder respectively perform feature encoding on a section of sequence data and its corresponding text description, and there is a corresponding relationship between the obtained sequence feature encoding and the text feature encoding.
[0085] In the second stage of the modality conversion part in the multi-source data modality conversion and fusion method, that is, in the sequence data text conversion method, the sequence data encoder obtained by the above Sequence CLIP method will be used to encode the sequence data to obtain the sequence CLIP feature F S The sequence data text conversion method is based on the CLIPCap method, but is different from the CLIPCap method for realizing image-to-text modality conversion in that the sequence data text conversion method converts the sequence CLIP feature F S The average, maximum, and minimum values of the sequence data obtained by statistical analysis methods p1, p2, ..., p n Combined with sequence CLIP and sequence statistical analysis features, the sequence description text is generated, thereby achieving a more accurate and reliable sequence data text mode conversion that can better reflect the original sequence data information. S Through a mapping model (Mapping Transformer) mapping to the text mode, the feature sequence f1, f2, ..., f in the text mode is obtained. k , and the sequence statistical description information p1, p2, ..., p n Combined with the autoregressive language model (such as GPT-2), it is converted into a text segment.
[0086] Given a time series x1, x2, ..., x t, and its corresponding description text paragraph character sequence c1, c2, ..., c l (All text sequences are completed to the same length, denoted as l), and the training objective of the sequence data text conversion method is:
[0087]
[0088] Where θ represents all trainable parameters in the model (including the parameters in the mapping model Mapping Transformer and the autoregressive language model), and i=1,…,N represents the i-th sequence-description text pair in the training data. The main purpose of the sequence data text conversion method is to utilize the cross-modal alignment capability of the Sequence CLIP to extract relevant features from the sequence data, and combine them with the basic information and statistical information of the sequence, and use the sequence CLIP features, sequence statistical information, and sequence basic information as prefix inputs of the autoregressive language model to generate subsequent text descriptions in an autoregressive manner. Therefore, the training objectives of the sequence data text conversion method can be refined as follows:
[0089]
[0090] Through the above modal conversion method, daily monitoring sequence data collected by wearable devices, IoT sensors, monitoring equipment and treatment instruments can be converted into corresponding descriptive texts. On this basis, in order to realize the fusion of daily monitoring data from different sources, as well as multi-source data such as patient baseline condition assessment, patient daily symptom assessment, medication records, etc., the multi-source text fusion method is based on the ChatCOPD large model, submits the multi-source text data to the COPD model for processing, and instructs the model to extract key information related to the patient's disease symptoms, disease progression, etc. from all different types and sources of text, generate a patient condition assessment report, and complete tasks such as patient baseline condition assessment and patient daily monitoring condition assessment.
[0091] The COPD management cloud platform of the COPD management system adopts cloud-native architecture design, combined with Serverless architecture, containerization technology and microservice architecture, so that the system can automatically manage the underlying computing resources and achieve high availability and elastic scalability of the system. The cloud platform includes five components: cloud database, conversation-guided COPD management model ChatCOPD, multi-source data modality conversion and fusion module, patient baseline condition assessment module, and patient monitoring and assessment module.
[0092] Among them, the cloud database uses cloud storage and database technology, and the platform can achieve safe storage, effective management and rapid access to large amounts of data such as patient medical records, examination results and treatment plans;
[0093] The dialogue-guided COPD management big model ChatCOPD applies the content of claim 2, based on deep learning technology and natural language processing technology. On the one hand, it realizes the coordination of the front-end and back-end with the natural language dialogue module in the patient local end of the COPD management system, conducts dialogues with patients, guides patients to provide basic information, symptoms, medication status, physical symptoms and other information, and intelligently identifies and analyzes the patient's condition, and provides corresponding management suggestions and treatment plans; on the other hand, it provides support for other services in the cloud platform, such as patient baseline condition assessment, patient monitoring and evaluation, etc., and realizes functions such as intelligent analysis of COPD patient information through a natural language big model in a specific vertical field (COPD management field). The ChatCOPD module also uses containerization technology to encapsulate the model into a container, so that it can run in different computing environments and expand on demand, with high portability and flexibility.
[0094] The multi-source data modality conversion and fusion module is based on the multi-source data modality conversion and fusion method based on contrastive learning and dialogue-guided large models, converts all types of data under various data modalities covered by this system into text format, and realizes the fusion of multi-source data in text mode. The multi-source data modality conversion and fusion module adopts Serverless architecture, which can automatically manage the underlying computing resources to achieve high availability and elastic scalability; and adopts microservice architecture, the cloud platform decomposes complex large modules into multiple small modules, realizes modular design, and facilitates system maintenance and upgrades. The multi-source data modal conversion and fusion module provides support for the subsequent patient baseline condition assessment module and patient monitoring and assessment module: for the patient baseline condition assessment task, the multi-source data modal conversion and fusion module converts the patient's basic information, signs, symptoms, electronic medical records, smoking history, imaging examination results, pulmonary function test results, historical follow-up and medical records, treatment plans and other patient baseline conditions into text and realizes multi-source information aggregation for analysis by the patient baseline condition assessment module; for the patient monitoring record analysis and symptom assessment tasks, the multi-source data modal conversion and fusion module converts the patient's daily test data such as exercise data, heart rate, blood oxygen, respiratory monitoring, sleep monitoring, indoor air quality, daily condition assessments such as coughing, sputum, and dyspnea, medication records, and daily test data such as air quality in the area into text and realizes multi-source information fusion under a unified text modality for analysis by the patient monitoring and assessment module.
[0095] The patient baseline condition assessment module calls the multi-source data modality conversion and fusion module to convert the collected patient basic information, medical history, physical examination results and other data into text modality, and generates a generalized and summarized standardized text description through text information extraction and aggregation, and calls the dialogue-guided COPD management model ChatCOPD, combines the medical knowledge base, clinical guidelines, historical patient analysis results, etc., to evaluate and analyze the patient's condition, generate a patient baseline condition assessment report, provide doctors with a comprehensive patient condition assessment, and provide the patient monitoring and assessment module with the patient's basic condition assessment results, and provide basic data for the subsequent treatment plan formulation. This module adopts the Serverless architecture, which can automatically manage the underlying computing resources to achieve high availability and elastic scaling.
[0096] The patient monitoring and evaluation module calls the multi-source data modality conversion and fusion module to convert daily monitoring data (including but not limited to motion data, heart rate, blood oxygen, respiratory monitoring, etc. collected by wearable devices, indoor air quality, temperature and humidity, etc. collected by IoT sensors, sleep monitoring conditions, ventilator / oxygen generator operation and patient testing conditions, etc. collected by detection equipment and treatment instruments), patient daily symptom assessment (including but not limited to coughing, sputum, dyspnea, etc.), medication records, air quality in the area, and other multi-source multi-modal data containing multiple different modalities such as text and time series data into a unified text modality, and generate a patient daily monitoring situation description report through text summary and generalization. The obtained daily monitoring situation description report is together with the patient baseline condition assessment report obtained by the patient baseline condition assessment module, and is analyzed through the dialogue-guided COPD management model ChatCOPD, in combination with the medical knowledge base, clinical guidelines, historical patient analysis results, etc., to analyze the patient's daily condition changes, promptly detect abnormal conditions of the patient's condition, and predict in advance the occurrence of emergencies such as acute exacerbation of COPD. When the model determines that an acute exacerbation may occur, an alarm is sent to the patient and his or her doctor, and intervention and emergency treatment suggestions are provided to the patient.
Claims
1. A conversation-guided, multi-source fusion COPD management system based on cloud-native architecture, characterized by: It consists of two parts: the patient's local terminal and the COPD management cloud platform: The patient local end includes an Internet of Things patient monitoring module, a natural language dialogue module, and a data summary transmission module: wherein the Internet of Things patient monitoring module collects data on various risk factors related to COPD in the patient's daily life; the natural language dialogue module and the data summary transmission module are deployed on the patient's mobile phone in the form of a mobile APP; the natural language dialogue module communicates with the patient in the form of a dialogue, and collects the patient's daily activities, symptom manifestations, and medication monitoring information through dialogue questions or in the form of questionnaire forms during the communication and dialogue process; the patient data collected by the patient monitoring module and the natural language dialogue module will be transmitted to the COPD management cloud platform through the data summary transmission module; the natural language dialogue module is combined with the COPD management cloud platform to support interaction with the patient in the form of a dialogue and provide remote auxiliary medical services, answer questions raised by the patient, provide treatment suggestions, assist in remote follow-up, and can analyze the patient's monitoring data and patient baseline data based on the multi-source data modal conversion and fusion method and the dialogue-guided COPD management large model ChatCOPD deployed on the cloud platform, predict the patient's disease development trend, determine the risk level of acute exacerbation of COPD, and give corresponding suggestions and warnings; The COPD management cloud platform evaluates the patient's baseline condition, establishes the patient's health record, and provides a personalized COPD management plan; at the same time, the cloud platform integrates and analyzes the patient's various medical data and daily monitoring data, promptly detects changes in the patient's condition, determines the probability of acute exacerbation emergencies, and provides doctors and patients with more comprehensive and accurate patient condition assessments and COPD management and treatment recommendations; the cloud platform adopts a cloud-native architecture design, combined with Serverless architecture, containerization technology and microservice architecture, and consists of a cloud database, a conversation-guided COPD management model ChatCOPD, a multi-source data modality conversion and fusion module, a patient baseline condition assessment module, and a patient monitoring and evaluation module; among which, the multi The source data modal conversion and fusion module is based on the multi-source data modal conversion and fusion method based on contrastive learning and dialogue-guided large model; the patient baseline condition assessment module calls the ChatCOPD model to analyze the patient baseline data and forms a patient baseline condition analysis report; the patient monitoring and assessment module is based on the multi-source data modal conversion and fusion module, and unifies the patient's daily activities, symptom manifestations, and medication monitoring information from the patient's local end including sequence and text modalities, as well as the assessment result report of the patient baseline assessment module into text modal data and realizes data fusion, and calls the dialogue-guided COPD management large model ChatCOPD to analyze the fused text data to realize symptom monitoring and evaluation, and acute exacerbation prediction and warning.
2. The conversation-guided multi-source fusion COPD management system based on cloud-native architecture as claimed in claim 1, characterized in that: The dialogue-guided COPD management model ChatCOPD is based on a pre-trained large language model and is retrained on patient medical records, medical histories, symptom descriptions, treatment plans, follow-up and monitoring data, the entire process of COPD patient visits, treatments and disease management, as well as doctor-patient conversation question and answer texts, COPD-related literature and diagnosis and treatment standard data, and is fine-tuned in the form of dialogue. This enables the model to have strong semantic understanding, reasoning and expression capabilities in the field of COPD management, and to accurately understand and judge the severity of the patient's condition, thereby providing personalized and effective COPD management plans, and to estimate the progression of the patient's condition through continuous monitoring and analysis of the patient's symptoms, and to predict and warn of acute exacerbation emergencies.
3. The conversation-guided multi-source fusion COPD management system based on cloud-native architecture as claimed in claim 2, characterized in that: The multi-source data modality conversion and fusion method includes two components: a modality conversion method and a multi-source text fusion method; wherein the modality conversion method adopts a sequence-text contrast pre-training algorithm of contrastive learning and cross-modal modality alignment algorithm, and a sequence data text conversion method to convert baseline and monitoring data such as patient baseline condition assessment, daily monitoring data, daily symptom assessment, medication records, and air quality in the area in different modalities including time series and text into a unified text modality; In the multi-source text fusion method, the text data obtained above will be fused in text mode through the ChatCOPD model, and the fused multi-source text information will be analyzed through the ChatCOPD model.
4. The conversation-guided multi-source fusion COPD management system based on cloud-native architecture as claimed in claim 3, characterized in that: The sequence-text contrastive pre-training algorithm is based on contrastive learning and cross-modal modality alignment algorithms, and relies on sequence data and its annotations and report description text datasets; the sequence-text contrastive pre-training algorithm includes a text data encoder Enc Text , a sequence data encoder Enc Seq , and a contrastive learning module to achieve feature alignment between the two encoders; The sequence data and its annotations and report description text datasets will be aligned in batches; for each batch of sequence data Seq and text data Text, they will be converted into feature vectors F through the corresponding encoders. S and F T : F S =Enc Seq (Seq), F T =Enc Text (Text), Then calculate the cosine similarity between each feature pair. For the feature vector F S The i-th and F T The jth element in The cosine similarity calculation formula is as follows: Among them, cosine_similarity is the cosine similarity calculation formula ||x||2 represents the L2 norm of vector x; by calculating the cosine similarity between all feature pairs, a similarity matrix S can be obtained, where S(i,j) represents the i-th feature vector in the sequence data and the jth feature vector in the text data The similarity between Then calculate the symmetric cross entropy loss function, that is, calculate the cross entropy loss of the similarity matrix S on the rows and columns respectively: First, for the cross entropy loss of logits on the rows: labels=arange(n), loss i =cross_entropy_loss(S,labels,axis=0), Among them, cross_entropy_loss is the cross entropy loss function, axis=0 means that the cross entropy loss is calculated for each column of S, and labels is a vector from 0 to n, which means that the value on the diagonal of the similarity matrix is the largest in the same column; Next, calculate the cross entropy loss of logits on the columns: loss t =cross_entropy_loss(S,labels,axis=1) Among them, axis = 1 means that the cross entropy loss is calculated for each row of the similarity matrix S; finally, the symmetric loss function is the average value of the cross entropy loss of the similarity matrix S on the rows and columns, that is: loss=(loss i +loss t ) / 2 The task of achieving cross-modal alignment between sequence data and its description text.
5. The conversation-guided multi-source fusion COPD management system based on cloud-native architecture as claimed in claim 3, characterized in that: In the sequence data text conversion method, the sequence data encoder obtained by the sequence-text comparison pre-training algorithm method will be used to encode the sequence data to obtain the sequence CLIP feature F S , the sequence CLIP feature F S Through a mapping model, it is mapped to the text mode and the feature sequence f1,f2,…,f is obtained under the text mode. k , and the sequence statistical description information p1,p2,…,p n Combined,converted into text segments via an autoregressive language model; Given a time series x1,x2,…,x t , and its corresponding description text paragraph character sequence c1,c2,…,c l , the training objective of the sequence data text conversion method is: Where θ represents all trainable parameters in the model, i=1,…,N represents the i-th sequence-description text pair in the training data, and the training objective of the sequence data text conversion method can be refined as:
6. The conversation-guided multi-source fusion COPD management system based on cloud-native architecture as claimed in claim 3, characterized in that: The data collection devices and related data types covered by the patient local end of the cloud platform include: Wearable devices: Bracelets for monitoring daily activities and vital signs of COPD patients, which collect exercise data such as the number of steps, exercise distance, exercise duration, and exercise intensity of patients, as well as vital sign data such as real-time heart rate and blood oxygen data around the clock, as well as the patient's breathing rate and breathing effort when the patient is in a resting state such as sleep and with less interference from exercise; IoT sensors: including but not limited to indoor air quality sensors, which measure the carbon dioxide concentration and suspended particulate matter index in indoor air; Monitoring equipment and instruments: including sleep monitors, oxygen concentrators and ventilators. The sleep monitors continuously measure the patient's sleep quality, sleep time, sleep apnea index, blood oxygen changes, and the operation status and parameters of the oxygen concentrator or ventilator, and combine them with the patient's physiological data; Text and form data: collect information such as patient symptoms, medication, and physical symptoms.