Knowledge-driven multi-mode large model lung cancer postoperative rehabilitation guidance method and system

Abstract

The invention relates to a multi-mode large model lung cancer postoperative rehabilitation guidance method and system based on knowledge driving, and belongs to the technical field of postoperative guidance rehabilitation. The method comprises the following steps: (1) preprocessing and inputting patient data; (2) performing large model entity identification fine tuning, and identifying entities from patient data; (3) carrying out secondary fine tuning on the large model, extracting event and scene information from patient data, and generating related description; (4) analyzing the multi-modal data by the large model, and constructing an EES knowledge graph; (5) carrying out EES information matching through the EES-Match; and (6) a large model is fused with a rehabilitation guidance scheme. According to the method, a knowledge graph formed by EES is combined, the EES-Match is used for matching, and finally a plurality of rehabilitation guidance methods are generated and fused into a personalized rehabilitation guidance scheme through a large model.

Description

Technical Field

[0001] The present invention relates to a knowledge-driven multimodal large-scale model lung cancer postoperative rehabilitation guidance method and system, belonging to the technical field of postoperative rehabilitation guidance. Background Art

[0002] Lung cancer, also known as primary bronchogenic carcinoma, is the most common lung malignancy originating from the trachea, bronchial mucosa, or glands. Globally, lung cancer morbidity and mortality are extremely high and on the rise. Common treatments for lung cancer include surgery, radiotherapy, and chemotherapy. Postoperative rehabilitation is crucial for recovery. It not only helps restore lung function but also reduces the incidence of postoperative complications and significantly improves patients' quality of life. Postoperative rehabilitation includes nutritional support, psychological counseling, and regular medical checkups. To ensure effective recovery, patients require personalized rehabilitation guidance that fully considers individual differences, including age, gender, postoperative condition, and lifestyle. Effective rehabilitation guidance can help patients adjust their exercise and diet in a targeted manner, improve their mental health, and promptly identify potential health issues. Therefore, postoperative rehabilitation is not just a part of medical treatment but a comprehensive, long-term health management process.

[0003] Traditional rehabilitation guidance is usually provided by doctors and professional rehabilitation therapists, but this relies on manual assessment and individual judgment, which is subject to certain subjectivity and inconsistency. In recent years, with the continuous development of artificial intelligence (AI) and big data technologies, more and more intelligent systems have been introduced into the medical and rehabilitation fields. AI technology, especially the application of deep learning and large language models (LLMs), has been proven to play an important role in medical diagnosis, health monitoring, and rehabilitation management. For example, using AI systems to analyze patients' medical records, surgical reports, and personal health data can provide patients with more accurate and personalized rehabilitation guidance. In particular, the emergence of multimodal large models provides a powerful tool that can integrate multiple types of information such as text and images, thereby providing patients with more comprehensive rehabilitation plans.

[0004] While large models have achieved remarkable success in many tasks, particularly in natural language processing and image recognition, they also suffer from significant drawbacks, particularly in healthcare applications, which can impact their reliability and accuracy. A major issue is that large models are prone to "hallucinations," where the content generated by the model may be false and unfounded. For example, in the healthcare field, models may erroneously generate diagnoses or rehabilitation recommendations that have no medical basis, which is particularly dangerous for lung cancer patients recovering. Therefore, reducing hallucinations and improving model credibility are key challenges in current large model applications. Furthermore, the uninterpretability of large models is a pressing issue. While large models can provide accurate predictions and recommendations, their reasoning is often "black box," meaning users cannot understand how the model reaches a particular conclusion. This is particularly challenging in the healthcare field, as doctors and patients need to understand and trust the model's recommendations in order to act accordingly. Without sufficient explanation and transparency, model recommendations may be perceived as unreliable, even leading to misdiagnosis or incorrect treatment. Therefore, improving model interpretability is crucial to enhancing their value in healthcare and rehabilitation. To this end, the present invention is proposed. Summary of the Invention

[0005] In response to the shortcomings of the existing technology, the present invention provides a knowledge-driven multimodal large-scale model for lung cancer postoperative rehabilitation guidance method and system. Based on the patient's historical medical data, such as case records, surgical reports, large medical records and other multimodal text and image information, the method automatically identifies key entities (Entity), events (Event) and scenes (Scene) information, combines the knowledge graph composed of EES, and uses EES-Match for matching, ultimately generating multiple rehabilitation guidance methods and integrating them into personalized rehabilitation guidance plans through the large model.

[0006] The technical solutions of the present invention are as follows:

[0007] A knowledge-driven multimodal large-scale model-based rehabilitation guidance method for lung cancer surgery, with the following steps:

[0008] (1) Preprocessing and input of patient data;

[0009] (2) Fine-tuning of large-scale entity recognition. The entity recognition task aims to identify entities from patient data, such as patient name, disease type, surgical procedure, etc.

[0010] (3) Secondary fine-tuning of the large model to extract event and scene information from patient data and generate relevant descriptions;

[0011] (4) The large model analyzes multimodal data and constructs the EES knowledge graph;

[0012] (5) EES information matching through EES-Match;

[0013] (6) Large model integration rehabilitation guidance program.

[0014] According to the preferred embodiment of the present invention, in step (1), the specific steps are:

[0015] Collect multimodal information of patients, including text data (cases, surgical reports) and related imaging data, and perform data preprocessing on the collected data, including cleaning and formatting, to ensure its structured processing to adapt to subsequent analysis. Then, through multimodal data fusion, fuse the text data (cases, surgical reports) with imaging data (such as lung CT, electrocardiogram, etc.) to form a multimodal image (unified data input format).

[0016] According to the preferred embodiment of the present invention, in step (2), the specific steps are:

[0017] The large model uses the Qwen2.5-VL-7B-Instruct large model for recognition and fine-tuning. The Qwen2.5-VL-7B-Instruct large model includes a Vision Encoder layer and a Qwen2.5 LM Decoder layer. The multimodal image obtained in step (1) passes through the Vision Encoder layer. The Vision Encoder layer encodes the input multimodal image information, extracts the key visual features of the image, and converts these features into a representation form that the model can subsequently understand and process. After that, the image content is analyzed through the Qwen2.5 LM Decoder layer.

[0018] Fine-tuning enables the large model to accurately identify entities from the input data. The fine-tuning process is achieved by minimizing the following objective function:

[0019]

[0020] in, Represents the total loss function of the entity recognition task, which is used to measure the difference between the model prediction and the true label, N is the number of samples, y i is the true label (i.e., the category of the entity), is the entity category predicted by the large model for the sample, i represents the sample index subscript (1≤i≤N), indicating the i-th training sample, and CrossEntropy represents the cross entropy loss function, which measures the true distribution y i and the predicted distribution The difference between.

[0021] According to the preferred embodiment of the present invention, in step (3), the specific steps are:

[0022] After completing entity recognition, the large model is fine-tuned for a second time to extract event and scene information from multimodal images and generate relevant descriptions. The second fine-tuning is performed by minimizing the loss function:

[0023]

[0024] Among them, λ1 and λ2 are weight hyperparameters used to balance the contribution of events and scenes, y Ev，i and y Sc，i are real event and scene labels, and are the event and scene labels predicted by the large model.

[0025] According to the preferred embodiment of the present invention, in step (4), the specific steps are:

[0026] Using the large model trained in steps (2) and (3), the entity, event, and scene information in the patient case are identified based on the input text and image data. The text data is segmented, named entity recognized, and event extracted. The patient's medical background, surgical history, and other information are structured and extracted. The image data is analyzed using deep learning methods (such as convolutional neural networks) to extract entity and scene information related to lung health, such as lesions and surgical sites. By modeling the knowledge of lung diseases, treatment processes, and postoperative rehabilitation in the medical field, an EES (Entity-Event-Scene) knowledge graph is constructed. The EES knowledge graph includes entity query logic, event and scene query logic, and context matching logic.

[0027] According to the preferred embodiment of the present invention, in step (5), the specific steps are:

[0028] Matching is performed through the EES knowledge graph, using the entity, event, and scenario information extracted from the large model to find the corresponding context information in the knowledge graph. EES-Match includes three matching processes: Cypher1, Cypher2, and Cypher3, which are three knowledge graph languages ​​converted from the output of the large model. The specific process is as follows:

[0029] First, based on the entity information stored in the EES knowledge graph, use the Cypher1 query language to query the entity nodes that match the target entity:

[0030] Cypher 1←MATCH(e:Entity)WHERE e.name=entity_name RETURN e.name,e.type

[0031] Among them, MATCH(e:Entity) means matching the entity and using e as the identifier in subsequent query statements; WHERE e.name = entity_name means using name and entity_name as query conditions, and RETURN e.name, e.type means returning name and type as the return values. Then, query events and scenarios related to the entity, generate a similarity score based on the description, and use Cypher2 to obtain all related events and scenario nodes of the entity in the knowledge graph:

[0032] Cypher2←MATCH(e: Entity{name:entity_name})

[0033] OPTIONAL MATCH(e)-[:INVOLVED_IN]->(ev:Event)

[0034] OPTIONAL MATCH(ev)-[:OCOURS_IN]->(s:Scene)

[0035] RETURN ev.description AS Event, s.description AS Scene

[0036] Among them, MATCH(e:Entity{name:entity_name}) means querying the Entity whose attribute name has the value of entity_name, and using e as the identifier in subsequent statements; OPTIONAL indicates an optional matching condition; MATCH(e)-[:INVOLVED_IN]->(ev:Event) means looking for Event and Event with the relationship: INVOLVED_IN, and using ev to represent Event in subsequent query statements; MATCH(ev)-[:OCCURS_IN]->(s:Scene) means looking for Event and Scene nodes with the relationship: OCCURS_IN; RETURN indicates the subsequent return data, which includes ev.description and s.description.

[0037] Finally, context matching is performed. Combined with the event and scene information generated from EES, context-related information is obtained through EES-Match matching. The pseudo code of EES matching is shown in Cypher3. The output of the large model is converted into a knowledge graph matching language to obtain the context-related information node Context:

[0038] Cypher3←MATCH(e:Entity{name:entity_name}),

[0039] (ev:Event{description:event_description}),

[0040] (s:Scene{description:scenedescription})

[0041] MATCH(c:Context)

[0042] WHERE(c)-[:DESCRIBES]->(e)

[0043] AND(c)-[:DESCRIBES]->(ev)AND(c)-[:DESCRIBES]->(s)

[0044] Among them, ev:Event{description:event_description} means matching Event whose description attribute is event_description; s:Scene{description:scene_description} means matching Scene whose description attribute is scene_description; MATCH(c:Context) means matching Context; (c)-[:DESCRIBES]->(e) means querying Context and Entity whose relationship is: DESCRIBES; (c)-[:DESCRIBES]->(ev) means querying Context and Event whose relationship is: DESCRIBES; (c)-[:DESCRIBES]->(s) means querying Context and Scene whose relationship is: DESCRIBES.

[0045] According to the preferred embodiment of the present invention, in step (6), specifically, the large model selects the best combination from multiple candidate rehabilitation guidance plans and integrates them into a final personalized rehabilitation plan. The plan integrates the individual differences of the patient, the type of surgery, the postoperative rehabilitation needs, and the historical medical data to ensure the personalization and targeting of the rehabilitation plan. Finally, the integrated rehabilitation guidance plan is presented to the doctor and the patient in natural language or other forms through the multimodal generative model for implementation.

[0046] A knowledge-driven, multimodal, large-scale lung cancer postoperative rehabilitation guidance system, including:

[0047] Data processing module, used for preprocessing and inputting patient data;

[0048] Recognition fine-tuning module, used for large-scale model entity recognition fine-tuning to identify entities from patient data;

[0049] The secondary fine-tuning module is used for secondary fine-tuning of the large model, extracting event and scene information from patient data and generating relevant descriptions;

[0050] The analysis module is used to analyze multimodal data with large models and build the EES knowledge graph;

[0051] Information matching module, used to perform EES information matching through EES-Match;

[0052] Program guidance module, used to integrate rehabilitation guidance programs into large models.

[0053] The beneficial effects of the present invention are:

[0054] By combining the knowledge graph (EES-KG) and the large model, the present invention can provide lung cancer patients with more accurate, reliable, and explanatory rehabilitation guidance. As a structured knowledge representation method, the knowledge graph can systematically store entities (such as patients' medical records, surgical reports, rehabilitation records, etc.), events (such as postoperative recovery progress, complications, etc.), and scenarios (such as patients' lifestyles, environments, etc.). In the present invention, the EES information in the knowledge graph will serve as the basis for model reasoning and will be matched through EES-Match to provide patients with tailored rehabilitation advice. In this way, the possibility of hallucinations caused by the large model can be effectively reduced, and its reliability and accuracy can be enhanced.

[0055] Furthermore, combined with the structured information of the knowledge graph, large models can provide a more transparent and explainable reasoning process. When outputting rehabilitation guidance, they can not only provide the final recommendation but also provide the basis for reasoning. For example, when recommending a certain diet or exercise plan, the system can explain the medical and rehabilitation principles behind it, allowing patients and medical staff to understand the model's decision-making process.

[0056] This invention combines the advantages of large models and knowledge graphs to provide accurate and personalized rehabilitation guidance for patients after lung cancer surgery. At the same time, it overcomes the hallucination problem of large models, improves the interpretability of the model, provides strong support for patients' postoperative recovery, and provides new solutions for intelligent decision-making in other medical fields. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1 Flow chart of the method of the present invention. DETAILED DESCRIPTION

[0058] The present invention will be further described below with reference to embodiments and accompanying drawings, but is not limited thereto.

[0059] Example 1:

[0060] This embodiment provides a knowledge-driven multimodal large-scale model-based rehabilitation guidance method for lung cancer surgery, which includes the following steps:

[0061] (1) Preprocessing and input of patient data: Collect multimodal information of patients, including text data (cases, surgical reports) and related imaging data, and preprocess the collected data, including cleaning and formatting, to ensure its structured processing to adapt to subsequent analysis. Then, through multimodal data fusion, fuse the text data (cases, surgical reports) with imaging data (such as lung CT, electrocardiogram, etc.) to form a multimodal image (unified data input format).

[0062] (2) Fine-tuning of large-scale entity recognition. The entity recognition task aims to identify entities from patient data, such as patient name, disease type, surgical procedure, etc.

[0063] The large model uses the Qwen2.5-VL-7B-Instruct large model for recognition and fine-tuning. The Qwen2.5-VL-7B-Instruct large model is an existing model, including a Vision Encoder layer and a Qwen2.5 LM Decoder layer. The multimodal image obtained in step (1) passes through the Vision Encoder layer. The Vision Encoder layer encodes the input multimodal image information, extracts the key visual features of the image, and converts these features into a representation form that the model can subsequently understand and process. After that, the image content is analyzed through the Qwen2.5 LM Decoder layer.

[0064] Fine-tuning enables the large model to accurately identify entities from the input data. The fine-tuning process is achieved by minimizing the following objective function:

[0065]

[0066] in, Represents the total loss function of the entity recognition task, which is used to measure the difference between the model prediction and the true label, N is the number of samples, y i is the true label (i.e., the category of the entity), is the entity category predicted by the large model for the sample, i represents the sample index subscript (1≤i≤N), indicating the i-th training sample, and CrossEntropy represents the cross entropy loss function, which measures the true distribution y i and the predicted distribution The difference between.

[0067] (3) Secondary fine-tuning of the large model to extract event and scene information from patient data and generate relevant descriptions;

[0068] After completing entity recognition, the large model is fine-tuned for a second time to extract event and scene information from multimodal images and generate relevant descriptions. The second fine-tuning is performed by minimizing the loss function:

[0069]

[0070] Among them, λ1 and λ2 are weight hyperparameters used to balance the contribution of events and scenes, y Ev，i and y Sc，i are real event and scene labels, and are the event and scene labels predicted by the large model.

[0071] (4) The large model analyzes multimodal data and constructs the EES knowledge graph;

[0072] Using the large model trained in steps (2) and (3), the entity, event, and scene information in the patient case are identified based on the input text and image data. The text data is segmented, named entity recognized, and event extracted. The patient's medical background, surgical history, and other information are structurally extracted. The image data is analyzed using deep learning methods (such as convolutional neural networks) to extract entity and scene information related to lung health, such as lesions and surgical sites. By modeling the knowledge of lung diseases, treatment processes, and postoperative rehabilitation in the medical field, an EES (Entity-Event-Scene) knowledge graph is constructed. The EES knowledge graph includes entity query logic, event and scene query logic, and context matching logic. The relevant logic code is as follows:

[0073]

[0074] Entity query logic

[0075]

[0076] Event and Scene query logic

[0077]

[0078] Context matching logic

[0079] The knowledge graph includes relationships between patient entities (such as patients, doctors), events (such as surgical procedures, postoperative observations), and related scenarios (such as postoperative recovery environment, living habits).

[0080] (5) EES information matching through EES-Match;

[0081] Matching is performed through the EES knowledge graph, using the entity, event, and scenario information extracted from the large model to find the corresponding context information in the knowledge graph. EES-Match includes three matching processes: Cypher1, Cypher2, and Cypher3, which are three knowledge graph languages ​​converted from the output of the large model. The specific process is as follows:

[0082] First, based on the entity information stored in the EES knowledge graph, use the Cypher1 query language to query the entity nodes that match the target entity:

[0083] Cypher1←MATCH(e:Entity)WHERE e.name=entity_name RETURN e.name,e.type

[0084] Among them, MATCH(e:Entity) means matching the entity and using e as the identifier in subsequent query statements; WHERE e.name = entity_name means using name and entity_name as query conditions, and RETURN e.name, e.type means returning name and type as the return values. Then, query events and scenarios related to the entity, generate a similarity score based on the description, and use Cypher2 to obtain all related events and scenario nodes of the entity in the knowledge graph:

[0085] Cypher2←MATCH(e: Entity{name:entity_name})

[0086] OPTIONAL MATCH(e)-[:INVOLVED_IN]->(ev:Event)

[0087] OPTIONAL MATCH(ev)-[:OCOURS_IN]->(s:Scene)

[0088] RETURN ev.description AS Event, s.description AS Scene

[0089] Among them, MATCH(e:Entity{name:entity_name}) means querying the Entity whose attribute name has the value of entity_name, and using e as the identifier in subsequent statements; OPTIONAL indicates an optional matching condition; MATCH(e)-[:INVOLVED_IN]->(ev:Event) means looking for the Entity and Event with the relationship: INVOLVED_IN, and using ev to represent Event in subsequent query statements; MATCH(e)-[:INVOLVED_IN]->(s:Scene) means looking for Event and Scene nodes with the relationship: OCCURS_IN; RETURN indicates the subsequent return data, which includes ev.description and s.description.

[0090] Finally, context matching is performed. Combined with the event and scene information generated from EES, context-related information is obtained through EES-Match matching. The pseudo code of EES matching is shown in Cypher3. The output of the large model is converted into a knowledge graph matching language to obtain the context-related information node Context:

[0091] Cypher3←MATCH(e:Entity{name:entity_name}),

[0092] (ev:Event{description:event_descrtption}),

[0093] (s:Scene{description:scene_description})

[0094] MATCH(c:Context)

[0095] WHERE(c)-[:DESCRIBES]->(e)

[0096] AND(c)-[:DESCRIBES]->(ev)AND(c)-[:DESCRIBES]->(s)

[0097] Among them, ev:Event{description:event_description} means matching the Event whose description attribute is event_description; s:Scene{description:scene_description} means matching the Scene whose description attribute is scene_description; MATCH(c:Context) means matching the Context; (c)-[:DESCRIBES]->(e) means querying the Context and Entity whose relationship is: DESCRIBES; (c)-[:DESCRIBES]->(ev) means querying the Context and Event whose relationship is: DESCRIBES; (c)-[:DESCRIBES]->(s) means querying the Context and Scene whose relationship is: DESCRIBES.

[0098] (6) The large model integrates rehabilitation guidance plans. The large model selects the best combination from multiple candidate rehabilitation guidance plans and integrates them into a final personalized rehabilitation plan. This plan integrates the patient's individual differences, surgery type, postoperative rehabilitation needs, and historical medical data to ensure the personalization and targeting of the rehabilitation plan. Finally, the integrated rehabilitation guidance plan is presented to doctors and patients in natural language or other forms through a multimodal generative model for implementation.

[0099] Example 2:

[0100] A knowledge-driven, multimodal, large-scale lung cancer postoperative rehabilitation guidance system, including:

[0101] Data processing module, used for preprocessing and inputting patient data;

[0102] Recognition fine-tuning module, used for large-scale model entity recognition fine-tuning to identify entities from patient data;

[0103] The secondary fine-tuning module is used for secondary fine-tuning of the large model, extracting event and scene information from patient data and generating relevant descriptions;

[0104] The analysis module is used to analyze multimodal data with large models and build the EES knowledge graph;

[0105] Information matching module, used to perform EES information matching through EES-Match;

[0106] Program guidance module, used to integrate rehabilitation guidance programs into large models.

Claims

1. A knowledge-driven multimodal large-scale model lung cancer postoperative rehabilitation guidance method, characterized by: Here are the steps: (1) Preprocessing and input of patient data; (2) Large model entity recognition fine-tuning, which aims to identify entities from patient data; (3) Secondary fine-tuning of the large model to extract event and scenario information from patient data and generate relevant descriptions; (4) The large model analyzes multimodal data and constructs the EES knowledge graph; (5) EES information matching through EES-Match; (6) Large model integration rehabilitation guidance program.

2. The knowledge-driven multimodal large-scale model lung cancer postoperative rehabilitation guidance method according to claim 1, characterized in that: In step (1), the specific steps are: Collect multimodal information of patients, including text data and related imaging data, perform data preprocessing on the collected data, including cleaning and formatting, and then fuse the text data with the imaging data through multimodal data fusion to form a multimodal image.

3. The knowledge-driven multimodal large-scale model lung cancer postoperative rehabilitation guidance method according to claim 2, characterized in that: In step (2), the specific steps are: The large model uses the Qwen2.5-VL-7B-Instruct large model for recognition and fine-tuning. The Qwen2.5-VL-7B-Instruct large model includes a Vision Encoder layer and a Qwen2.5 LM Decoder layer. The multimodal image obtained in step (1) passes through the Vision Encoder layer. The Vision Encoder layer encodes the input multimodal image information and extracts the key visual features of the image. After that, the image content is analyzed through the Qwen2.5 LM Decoder layer. Through fine-tuning, the large model is made to accurately identify entities from the input data. The fine-tuning process is achieved by minimizing the following objective function: in, Represents the total loss function of the entity recognition task, which is used to measure the difference between the model prediction and the true label, N is the number of samples, y i is the true label, is the entity category predicted by the large model for the sample, i represents the sample index subscript (1≤i≤N), indicating the i-th training sample, and CrossEntropy represents the cross entropy loss function, which measures the true distribution y i and the predicted distribution The difference between.

4. The knowledge-driven multimodal large-scale model lung cancer postoperative rehabilitation guidance method according to claim 3, characterized in that: In step (3), the specific steps are: After completing entity recognition, the large model is fine-tuned for a second time to extract event and scene information from multimodal images and generate relevant descriptions. The second fine-tuning is performed by minimizing the loss function: Among them, λ1 and λ2 are weight hyperparameters used to balance the contribution of events and scenes, y Ev，i and y Sc，i are real event and scene labels, and are the event and scene labels predicted by the large model.

5. The knowledge-driven multimodal large-scale model lung cancer postoperative rehabilitation guidance method according to claim 4, characterized in that: In step (4), the specific steps are: Using the large model trained in steps (2) and (3), the entity, event, and scene information in the patient case are identified based on the input text and image data. The text data is segmented, named entity recognized, and event extracted. The patient's medical background and surgical history information are structured and extracted. The image data is analyzed using deep learning methods to extract entity and scene information related to lung health. By modeling the knowledge of lung diseases, treatment processes, and postoperative rehabilitation in the medical field, an EES knowledge graph is constructed. The EES knowledge graph includes entity query logic, event and scene query logic, and context matching logic.

6. The knowledge-driven multimodal large-scale model lung cancer postoperative rehabilitation guidance method according to claim 5, characterized in that: In step (5), the specific steps are: Matching is performed through the EES knowledge graph, using the entity, event, and scenario information extracted from the large model to find the corresponding contextual information in the knowledge graph. EES-Match includes three matching processes: Cypher1, Cypher2, and Cypher3, which are three knowledge graph languages ​​converted from the output of the large model. The specific process is as follows: First, based on the entity information stored in the EES knowledge graph, use the Cypher1 query language to query the entity nodes that match the target entity: Cypher1←MATCH(e:Entity)WHERE e.name=entity_name RETURN e.name,e.type Among them, MATCH(e:Entity) means matching the entity and using e as the identifier in subsequent query statements; WHERE e.name = entity_name means using name and entity_name as query conditions, and RETURN e.name, e.type means returning name and type as the return values. Then, query events and scenarios related to the entity, generate a similarity score based on the description, and use Cypher2 to obtain all related events and scenario nodes of the entity in the knowledge graph: Cypher2←MATCH(e: Entity{name: entity_name}) OPTIONAL MATCH(e)-[:INVOLVED_IN]->(ev:Event) OPTIONAL MATCH(ev)-[:OCCURS_IN]->(s:Scene) RETURN ev.description AS Event,s.description AS Scene Among them, MATCH(e:Entity{name:entity_name}) means querying the entity whose attribute name has the value of entity_name, and uses e as the identifier in subsequent statements; OPTIONAL indicates an optional matching condition; MATCH(e)-[:INVOLVED_IN]->(ev:Event) means searching for the entity and event with the relationship: INVOLVED_IN, and using ev to represent Event in subsequent query statements; MATCH(ev)-[:OCCURS_IN]->(s:Scene) means searching for Event and Scene nodes with the relationship: OCCURS_IN; RETURN indicates the subsequent return data, which includes ev.description and s.description; Finally, context matching is performed. Combined with the event and scene information generated from EES, context-related information is obtained through EES-Match matching. The pseudo code of EES matching is shown in Cypher3. The output of the large model is converted into a knowledge graph matching language to obtain the context-related information node Context: Cypher3←MATCH(e:Entity{name:entity_name}), (ev:Event{description:event_description}), (s:Scene{description:scene_description}) MATCH(c:Context) WHERE(c)-[:DESCRIBES]->(e) AND(c)-[:DESCRIBES]->(ev)AND(c)-[:DESCRIBES]->(s) Among them, ev:Event{description:event_description} means matching the Event whose description attribute is event_description; s:Scene{description:scene_description} means matching the Scene whose description attribute is scene_description; MATCH(c:Context) means matching the Context; (c)-[:DESCRIBES]->(e) means that the query relationship is: DESCRIBES's Context and Entity; (c)-[:DESCRIBES]->(ev) means that the query relationship is: DESCRIBES's Context and Event; (c)-[:DESCRIBES]->(s) means that the query relationship is: DESCRIBES's Context and Scene.

7. The knowledge-driven multimodal large-scale model lung cancer postoperative rehabilitation guidance method according to claim 6, characterized in that: In step (6), specifically, the large model selects the best combination from multiple candidate rehabilitation guidance plans and integrates them into the final personalized rehabilitation plan.

8. A knowledge-driven multimodal large-scale lung cancer postoperative rehabilitation guidance system, characterized by: include: Data processing module, used for preprocessing and inputting patient data; Recognition fine-tuning module, used for large-scale model entity recognition fine-tuning to identify entities from patient data; The secondary fine-tuning module is used for secondary fine-tuning of the large model, extracting event and scene information from patient data and generating relevant descriptions; The analysis module is used to analyze multimodal data with large models and build the EES knowledge graph; Information matching module, used to perform EES information matching through EES-Match; Program guidance module, used to integrate rehabilitation guidance programs into large models.

Images