Digital liver cancer whole course management method and device based on large model agent

Abstract

This disclosure provides a digital intelligent management method and device for the entire course of liver cancer based on a large-scale intelligent model: It acquires input multimodal information, including question information and medical image information related to the target object. The question information describes information related to liver cancer in the target object, and the medical image information includes information obtained from liver cancer-related examinations of the target object. Based on the question information, it determines the target task to be processed, which includes at least one of medical history association and image analysis based on medical image information. Based on the result of executing the target task, it outputs structured analysis information of liver cancer corresponding to the target object. In existing liver cancer management, doctors need to consider a large amount of medical data from different sources and modalities, which is time-consuming. The implementation of this disclosure can provide medical staff with structured multimodal data processing results to support clinical decision-making, thereby improving the efficiency and level of medical management.

Description

Technical Field

[0001] This disclosure relates to the fields of artificial intelligence and medical technology. Specifically, this disclosure relates to a digital intelligent management method, system and related equipment for the whole course of liver cancer based on a large model intelligent agent. Background Technology

[0002] With the explosive growth of medical knowledge and the multimodal nature of diagnostic and treatment data (electronic medical record text, images, surgical videos, etc.), doctors and patients face severe challenges in the full-course management of complex diseases such as liver cancer. These challenges include insufficient specialty depth: general models cannot handle specialty tasks, liver cancer surgery relies on surgeon experience for subtyping, and there is a lack of standardized operating procedures. Furthermore, knowledge fragmentation exists: guidelines, consensus statements, and 3D model data are scattered, and knowledge about hepatobiliary and pancreatic diseases is dispersed across paper documents and isolated databases, resulting in lengthy retrieval times. Existing text extraction tools (such as conventional OCR) struggle to accurately reproduce the layout and semantic information of medical documents, leading to the loss or distortion of key data (such as laboratory indicator tables and imaging descriptive diagrams), affecting subsequent analysis, limiting doctors' accumulation of specialty knowledge, and impacting their work. Summary of the Invention

[0003] The purpose of this disclosure is to at least solve one of the aforementioned technical defects. The technical solution provided by the embodiments of this disclosure is as follows: The first aspect is a digital and intelligent approach to the whole-course management of liver cancer based on large-scale model intelligent agents, including: The input multimodal information includes question information and medical image information related to the target object. The question information is used to describe information related to liver cancer of the target object, and the medical image information includes information obtained from liver cancer-related examinations of the target object. Based on the problem information, a target task to be processed is determined, and the target task includes at least one of linking medical history with the constructed knowledge base and image analysis based on the medical image information. Based on the results of performing the target task, the structured analysis information of liver cancer corresponding to the target object is output.

[0004] In one feasible embodiment, if the target task includes associating medical history with a constructed knowledge base, then the execution of the target task includes at least one of the following: Based on the problem information, first information related to liver cancer of the target object is obtained through a locally pre-built first knowledge base, wherein the first knowledge base includes content built based on liver cancer knowledge; Based on the aforementioned problem information, second information related to liver cancer is obtained through a second knowledge base pre-built in the cloud. The second knowledge base includes content built based on public medical knowledge.

[0005] In one feasible embodiment, the first knowledge base is constructed through the following operations: Differentiated parsing for multimodal documents; Based on the parsing results, the document is divided into directory-guided segments; Using a large language model, several question-answer pairs are generated for each segment, which include the content of that segment and are related to other parts of the document. The pre-trained text model encodes each text segment into a feature vector for storage.

[0006] In one feasible embodiment, the step of obtaining first information related to liver cancer of the target object based on the problem information through a locally pre-built first knowledge base includes: The problem information is encoded to obtain a problem feature vector; Based on the question feature vector, fragments and question-answer pairs related to the target object are obtained from the first knowledge base.

[0007] In one feasible embodiment, the target task performed, which involves image analysis based on the medical image information, includes at least one of the following: Based on the aforementioned medical image information, central hepatocellular carcinoma is identified to obtain third information; Based on the medical image information, at least one of the following is performed: lesion identification, microvascular invasion (MVI) sign analysis, and satellite lesion identification.

[0008] In one feasible embodiment, the step of performing at least one of lesion identification, microvascular invasion (MVI) sign analysis, and satellite lesion identification based on the medical image information includes: Based on the medical image information, multi-scale feature extraction is performed to generate an image feature map of the liver region; Based on the medical image information, semantic segmentation is performed, and the contour coordinate array and volume measurement value of the lesion region are output. Based on the medical image information, texture analysis is performed to output a quantitative score reflecting the probability of microvascular invasion; Based on the medical image information, a neighborhood search is performed to output the location and confidence level of the suspected satellite lesion. In one feasible embodiment, the step of identifying central hepatocellular carcinoma based on the medical image information to obtain third information includes: Based on the aforementioned medical image information, individualized Fang's classification of central hepatocellular carcinoma is performed to obtain the classification results; Based on the medical image information, the liver condition of the target subject is assessed, and the assessment result is obtained; Based on the evaluation results, a similarity match is performed with a preset subtyping feature library, and at least one subtyping label with the highest matching degree and its probability vector are output.

[0009] In one feasible embodiment, the medical image information includes image information from CT and / or MRI arterial and portal venous phase sequences; Based on the medical image information, the Fang's classification of central hepatocellular carcinoma is performed to obtain the classification results, including: Image features are extracted based on the medical image information using a convolutional neural network; Based on the image features, the classification results are determined by matching them with the classification rules in the preset classification rule library. The classification rules include mass type, infiltrative type and mixed type. The classification results include at least one of individualized hepatic artery classification, individualized portal vein classification, individualized right hepatic portal vein classification or individualized hepatic vein classification.

[0010] In one feasible embodiment, prior to performing the central type classification process, the method further includes: Based on the medical image information, three-dimensional imaging of the liver of the target object is performed to obtain three-dimensional image data; The method further includes: Based on the aforementioned third information, a three-dimensional visualization classification of central hepatocellular carcinoma is performed, and at least one of the following operations is executed based on the processing result: The corresponding 3D model is converted into a solid model using 3D printing equipment. This solid model includes a physical model of the tumor and surrounding tissues. Displayed via virtual reality devices; Perform a simulated surgery and obtain the simulated surgical results.

[0011] Secondly, embodiments of this disclosure provide a digital intelligent liver cancer full-course management system based on a large-scale model intelligent agent, including: A multimodal input interface is used to acquire input multimodal information, which includes question information and medical image information related to the target object. The question information is used to describe information related to liver cancer of the target object, and the medical image information includes information obtained from liver cancer-related examinations of the target object. An intent parsing module is used to determine the target task to be processed based on the question information. The target task includes at least one of associating medical history with a constructed knowledge base and image analysis based on the medical image information. The task scheduling engine, based on the ReAct architecture, is used to break down the task chain and call the corresponding modules according to the intent parsing results. The knowledge base retrieval module is used to obtain first information related to liver cancer of the target object based on the question information through a first knowledge base pre-built locally; and to obtain second information related to liver cancer through a second knowledge base pre-built in the cloud, wherein the second knowledge base includes content built based on public medical knowledge. The image analysis module is used to perform image analysis on the medical image information; The structured report generation module is used to output structured analysis information of liver cancer corresponding to the target object based on the results of performing the target task.

[0012] Thirdly, embodiments of this disclosure provide an electronic device including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the method provided in the first aspect and any of its embodiments.

[0013] Fourthly, embodiments of this disclosure provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method described in the first aspect and any of its embodiments.

[0014] Fifthly, embodiments of this disclosure provide a computer program product, including a computer program that, when executed by a processor, implements the steps of the method provided in the first aspect and any of its embodiments.

[0015] The beneficial effects of the technical solutions provided in this disclosure are: This disclosure provides a digital intelligent management method for the entire course of liver cancer based on a large-scale intelligent model. Specifically, it can acquire input multimodal information, which may include question information and medical image information related to the target object. The question information describes information related to liver cancer in the target object, and the medical image information includes information obtained from liver cancer-related examinations of the target object. Based on this, the target task to be processed can be determined through the question information. The target task may include at least one of associating medical history with a constructed knowledge base and image analysis based on medical image information. Furthermore, based on the result of executing the target task, structured analysis information of liver cancer corresponding to the target object can be output. The implementation of this disclosure can be adapted to process liver cancer specialty tasks (such as liver cancer image analysis), providing decision-making information to medical staff. It can also associate diagnostic and treatment data through a pre-constructed knowledge base, improving information processing capabilities, enhancing specialty full-course management capabilities, and thus improving the level of medical care. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of this disclosure, the accompanying drawings used in the description of the embodiments of this disclosure will be briefly introduced below.

[0017] Figure 1 A flowchart of a digital intelligent liver cancer full-course management method based on a large model intelligent agent provided in this disclosure embodiment; Figure 2 A flowchart illustrating a management scenario for initial screening and diagnosis provided in this disclosure embodiment; Figure 3 A schematic diagram of a three-dimensional visualization assessment path for central hepatocellular carcinoma provided in an embodiment of this disclosure; Figure 4 This is a schematic diagram illustrating a closed-loop management process for the entire liver cancer surgery, as provided in an embodiment of this disclosure. Figure 5 A schematic diagram of the architecture of a digital intelligent liver cancer full-course management system based on a large model intelligent agent provided in this embodiment of the disclosure; Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present disclosure. Detailed Implementation

[0018] The embodiments of this disclosure are described below with reference to the accompanying drawings. It should be understood that the embodiments described below with reference to the accompanying drawings are exemplary descriptions for explaining the technical solutions of the embodiments of this disclosure, and do not constitute a limitation on the technical solutions of the embodiments of this disclosure.

[0019] Those skilled in the art will understand that, unless specifically stated otherwise, the singular forms “a,” “an,” “the,” and “the” used herein may also include the plural forms. It should be further understood that the terms “comprising” and “including” as used in embodiments of this disclosure mean that the corresponding feature can be implemented as the presented feature, information, data, step, operation, element, and / or component, but do not exclude implementation as other features, information, data, step, operation, element, component, and / or combinations thereof supported by the art. It should be understood that when we say that an element is “connected” or “coupled” to another element, the one element can be directly connected or coupled to the other element, or it can mean that the one element and the other element are connected through an intermediate element. Furthermore, “connected” or “coupled” as used herein can include wireless connection or wireless coupling. The term “and / or” as used herein indicates at least one of the items defined by the term, for example, “A and / or B” or “A, B” indicates implementation as “A,” or implementation as “B,” or implementation as “A and B.”

[0020] In this disclosure, the terms "module" or "unit" refer to a computer program or part of a computer program that has a predetermined function and works with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can be part of an overall module or unit that includes the functionality of that module or unit.

[0021] To make the objectives, technical solutions, and advantages of this disclosure clearer, the embodiments of this disclosure will be described in further detail below with reference to the accompanying drawings.

[0022] The following description of several exemplary embodiments illustrates the technical solutions of this disclosure and the technical effects produced by these solutions. It should be noted that the following embodiments can be referenced, learned from, or combined with each other. Identical terms, similar features, and similar implementation steps in different embodiments will not be repeated.

[0023] The methods provided in this disclosure may relate to one or more fields in the technical fields of speech, language, image, video, or data intelligence.

[0024] The core objective of the whole-course management system is to break down the barriers of time, space, and profession in traditional healthcare, integrating and structuring all of a patient's medical and health information from pre-hospital to post-hospital stages, and presenting it to medical staff, patients, and administrators in a clear and operable manner. One of the system's key features is its ability to deliver structured output. This structured output paradigm goes beyond simply listing data; it presents intelligently processed, correlated, and visualized decision support information.

[0025] Furthermore, compared to other diseases, the management of liver cancer throughout its entire course has very distinct and unique characteristics. This is mainly due to the highly malignant nature of liver cancer itself, the special nature of the liver organ, and the unique regional epidemiological characteristics of liver cancer. Many diseases are managed after symptoms appear or diagnosis is made, focusing on controlling the existing disease. Early symptoms of liver cancer are extremely subtle, and approximately 70%-80% of patients are diagnosed at an advanced stage, missing the optimal opportunity for radical treatment. Therefore, the primary and most unique characteristic of liver cancer management throughout its entire course is its extreme reliance on early screening of high-risk groups. Treatment for other diseases typically only considers how to remove lesions (such as tumor resection), with relatively low requirements on the organ's basic function. However, more than 90% of liver cancers occur on the basis of cirrhosis, and patients' liver function reserve (Child-Pugh score) is already fragile. Therefore, the treatment of liver cancer must not only consider "killing the tumor" but also "protecting the liver." In the comprehensive management of liver cancer, internists (especially hepatologists and infectious disease specialists) need to monitor liver function throughout the entire process. Through antiviral therapy, liver protection, and correction of coagulation function, they create a window of opportunity for patients to undergo surgery, interventional procedures, or systemic treatment, and help liver function recover after treatment. Liver cancer exhibits extreme heterogeneity; the same patient requires completely different strategies at different stages of the disease (early, middle, late, and post-recurrence). Surgery, interventional radiology, oncology, radiation oncology, radiology, and pathology departments need to customize personalized plans for each patient. Even with radical resection of liver cancer, the recurrence rate within 3 years post-surgery is as high as 50%-70%. Early recurrence is often asymptomatic, and once symptoms (such as abdominal pain and jaundice) appear, it is usually already in an advanced stage. Therefore, comprehensive management of liver cancer emphasizes the seamless extension from "in-hospital treatment" to "out-of-hospital follow-up." Through tools such as specialist management, apps, and intelligent follow-up systems, a rigorous follow-up plan is enforced (such as a check-up every 3 months for 1-2 years post-surgery) to ensure that recurrence is detected as early as possible.

[0026] Therefore, the whole-course management of liver cancer requires starting with prevention and screening, continuing with individualized treatment centered on MDT (Multidisciplinary Team), and extending to long-term psychological, nutritional, and recurrence monitoring. Its complexity, dynamism, and high dependence on multidisciplinary collaboration make the provision of a more intelligent management system of great significance.

[0027] The following describes the digital intelligent management method for the entire course of liver cancer based on a large-scale intelligent agent, as provided in the embodiments of this disclosure. Optionally, the method provided in the embodiments of this disclosure can be achieved through... Figure 5 The system architecture shown is implemented.

[0028] Specifically, such as Figure 1 As shown, the method provided in this embodiment includes steps S101 to S103: S101. Obtain input multimodal information, the multimodal information including question information and medical image information related to a specified or unspecified target object, the question information being used to describe information related to liver cancer of the target object, and the medical image information including information obtained from liver cancer-related examinations of the target object.

[0029] S102. Based on the problem information, determine the target task to be processed. The target task includes at least one of associating medical history with the constructed knowledge base and image analysis based on the medical image information.

[0030] S103. Based on the results of performing the target task, output liver cancer analysis information corresponding to the target object.

[0031] Optionally, the system can provide a natural language interactive interface (such as web forms, voice input systems, etc.) to collect questions from users (such as medical staff) related to liver cancer of the target group (such as patients). These questions can cover multiple dimensions, including symptom descriptions, medical history inquiries, and consultations on recommended treatment plans. Optionally, the question information can be text or voice information. The target group involved in this embodiment does not specifically refer to a single patient, but can also be a group of people with one or more similar indicators, thereby enabling full-course management of liver cancer for single or several specified target groups.

[0032] Optionally, the system can also provide a medical image information acquisition interface, utilizing medical imaging equipment (such as CT scanners, MRI equipment, ultrasound diagnostic instruments, etc.) to perform liver cancer-related examinations on the target object and acquire corresponding medical image data. This data can include images that reflect key information such as liver morphology, structure, and blood flow, such as plain and enhanced CT scans of the liver, and dynamic enhanced MRI images of the liver. Optionally, to improve the accuracy of data processing, the acquired image data can also be standardized to ensure data consistency and usability.

[0033] For example, such as Figure 5 As shown, this embodiment of the disclosure provides users with an entry point for inputting multimodal data such as images or 3D data, as well as language questions.

[0034] Optionally, when the problem information involves content related to the patient's medical history, symptom development, or other disease-related aspects, the large-scale model agent can define the target task as medical history association. In this case, the large-scale model agent can analyze the disease's development patterns and potential complications by interacting with a pre-built liver cancer knowledge base, retrieving the patient's historical medical reports, and pathological information similar to the patient's symptoms and medical history. This provides historical reference and experience support for the subsequent development of a comprehensive disease management plan.

[0035] Optionally, in determining the target task to be processed based on problem information, embodiments of this disclosure provide a ReAct task engine (which may involve, for example, Figure 5 The AI-Agent, as shown, processes information in two ways. Firstly, it performs reasoning, such as parsing user intent and breaking down the task chain (e.g., a user uploads a CT image and asks "Is there a tumor?", which can be broken down into: a) image anomaly detection; b) if an anomaly is found, linking it to the patient's medical history; c) generating a diagnostic opinion). Secondly, it performs actions, such as automatically calling preset tool APIs based on the task chain (e.g., calling the "Liver CT Lesion Detection API," "LiRADs Grading API," and "NCCN Guideline Query API") to complete the target task. Tool calls adhere to privacy rules, and sensitive operations can be performed at the edge.

[0036] Optionally, when the problem information involves image-related content such as the interpretation of medical images and the identification of lesion features, the large-scale intelligent model can perform image analysis tasks. Utilizing deep learning algorithms and computer vision technology, it can automatically analyze and process the input medical images, identify lesion areas within the liver, and calculate characteristic parameters such as the size and shape of the lesions. For example, image segmentation algorithms can be used to determine the location and extent of liver tumors, and texture analysis technology can be used to assess the malignancy of the tumor, providing relatively objective image-based evidence for the entire course of the disease.

[0037] Optionally, the large model agent can comprehensively analyze the results of history association tasks and / or image analysis tasks, such as by using... Figure 5 The expert toolkit shown, combined with clinical guidelines and expert consensus on liver cancer diagnosis and treatment, assesses the severity, stage, and type of a patient's condition, generates an individualized liver cancer management plan for the entire course of the disease, and outputs corresponding structured liver cancer analysis information (such as...). Figure 5 The returned results are shown below. For example, the tool's returned results (such as multimodal feedback, language suggestions, etc.) can be integrated into a unified response (such as a PDF report + image annotation display).

[0038] In this embodiment, large amounts of multimodal information can be processed and analyzed rapidly, providing medical staff with more suitable information for the entire course of disease management. This reduces the time and effort required for manual evaluation, improves decision-making efficiency, and by comprehensively considering the patient's medical history, symptoms, imaging characteristics, and other information, personalized full-course management plans can be output for each patient, improving targeting and effectiveness and enhancing patient prognosis. Furthermore, this embodiment supports multimodal input / output and ReAct task chains to meet diverse clinical scenarios, and the system design covers the entire process of liver cancer screening, diagnosis, treatment, and prognosis. Through personalized knowledge services and tool invocation, it achieves true digital and intelligent end-to-end assistance.

[0039] In a feasible embodiment, if the target task includes associating medical history with a constructed knowledge base, then the execution of the target task includes at least one of the following steps A1 and A2: Step A1: Based on the problem information, obtain first information related to liver cancer of the target object through a locally pre-built first knowledge base. The first knowledge base includes content built based on liver cancer knowledge.

[0040] Step A2: Based on the problem information, obtain second information related to liver cancer through a second knowledge base pre-built in the cloud. The second knowledge base includes content built based on public medical knowledge.

[0041] Optional, a locally pre-built first knowledge base (such as...) Figure 5 The local knowledge base shown is designed to build upon existing knowledge about liver cancer, covering comprehensive professional content from etiology, pathogenesis, and pathophysiology to clinical manifestations, diagnostic methods, treatment approaches, and prognostic assessment. For example, it records the unique pathological features and biological behaviors of different types of liver cancer (such as hepatocellular carcinoma and cholangiocarcinoma), providing a solid theoretical basis for precise diagnosis and treatment. Optionally, it can also include liver cancer diagnosis and treatment information for each patient.

[0042] Optionally, the pre-built second knowledge base in the cloud includes content based on public medical knowledge, which can encompass a wider range of medical information, such as basic medical knowledge (anatomy, physiology, biochemistry, etc.), knowledge of other related diseases (such as hepatitis B, hepatitis C, and other liver diseases closely related to liver cancer), and general medical research methods and statistical knowledge. This information helps to understand liver cancer from multiple perspectives and levels, providing more comprehensive support for the whole-course management of liver cancer.

[0043] In this embodiment, on the one hand, rapid access can be achieved through a local knowledge base. The first knowledge base is stored on a local server or device, allowing medical staff to quickly access the necessary information without relying on a network connection. This ensures timely access even in situations with poor network conditions or emergencies. For example, during surgery or in emergency situations, relevant information can be quickly retrieved from the local knowledge base as supplementary support. On the other hand, real-time updates and sharing of information are achieved through a cloud-based knowledge base, enabling the timely delivery of the latest medical research findings and clinical experience to medical staff. For example, when a hospital in a certain region encounters a rare case of liver cancer, it can access similar pathological experiences and treatment plans from hospitals in other regions through the cloud-based knowledge base.

[0044] In a feasible embodiment, considering that existing retrieval methods based on simple vector similarity perform poorly when associating complex medical concepts across chapters and documents (e.g., associating "postoperative AFP change trend after liver cancer ablation" with the "postoperative follow-up" and "tumor markers" sections in a guideline), the relevance of retrieval results is low. This disclosure provides a knowledge base construction scheme to address the problems of incomplete information extraction and low structuring in medical documents (such as scanned documents, mixed text and image documents), thereby improving the accuracy of cross-chapter and cross-document medical knowledge retrieval and association capabilities. The constructed knowledge base can employ artificial intelligence models, such as deep learning-based OCR and language model fusion technology, to parse information from medical documents and construct neural network models. This optimizes the accuracy of character recognition and semantic understanding, improves the completeness of information extraction and the accuracy of information recognition, and reduces information omissions caused by complex layout or ambiguous characters. Optionally, knowledge graph construction technology can be introduced to map the extracted medical information to a predefined entity-relationship framework, forming structured data, which is beneficial for improving the coverage of cross-document knowledge association and increasing response speed.

[0045] For example, this disclosure provides a structured local knowledge base for liver cancer, as shown in Table 1 below: Table 1

[0046] Optionally, the first knowledge base is constructed through the following steps A01 to A04: Step A01: Perform differential parsing on multimodal documents.

[0047] Step A02: Based on the parsing results, perform directory-guided segmentation on the document.

[0048] Step A03: Using a large language model, generate several question-answer pairs based on each segment, which include the content of that segment and are related to other parts of the document.

[0049] Step A04: Encode each text segment into a feature vector and store it using a pre-trained text model.

[0050] Optionally, the system provides a multimodal document processing engine to perform differentiated parsing of multimodal documents. For example, it provides appropriate parsing methods for documents of different modalities: 1. For parsable PDFs: Convert to HTML, parse the DOM tree, remove irrelevant style tags, and extract the pure semantic text.

[0051] 2. Scan PDF: The Visual-Language Model (VLM) is invoked to perform: a) Page segmentation (recognizing text blocks, tables, charts, and titles); b) Reading order prediction; c) High-precision OCR of text regions; d) Recognizing cell coordinates, borders, and merging relationships in table regions, and outputting structured data with HTML tags; e) Generating descriptive text for medical image regions (e.g., "Liver CT arterial phase shows an approximately 2cm enhancing nodule in the left lobe").

[0052] 3. PPT: Convert each slide to an image, and use VLM to extract text content and natural language summaries of key charts / flowcharts.

[0053] Optionally, structured output can be used: tables are saved in HTML format; images are embedded as knowledge base placeholders with their descriptive text, and the original image links are retained for later retrieval.

[0054] Optionally, this disclosure also provides an intelligent segmentation and semantic enhancement module to perform steps A02 to A04 as described above. For example, in the directory-guided segmentation process, the document is segmented according to its chapter title hierarchy (e.g., "1. Overview", "1.1 Epidemiology"), with the length of each segment controlled within a preset threshold (e.g., 4096 tokens). In the global question-and-answer (QA) pair generation process, for each segment, a large language model (e.g., Qwen-plus) is called to generate several (e.g., 5-10) question-and-answer pairs (QA) that cover the core content of the segment and are related to other parts of the document. For example, for the segment "Indications for TACE treatment of liver cancer", the following QA is generated: "Q: Are Child-Pugh B patients suitable for TACE? A: Liver function reserve needs careful assessment; see the relevant descriptions in the 'Liver Function Assessment' and 'TACE Complications' sections." In vectorization and retrieval processing, pre-trained models (such as GPT and BERT variants) can be used to encode segmented text into vectors and store them in an efficient vector database (such as FAISS). Subsequently, in semantic reordering processing, when a user asks a question, the question is encoded and the top-K (the top K most relevant matching segments) are retrieved from the vector database. The reordering model (such as Qwen3-Reranker-8B) is then invoked to reorder the relevance of the K segments based on a deep understanding of the question's semantics, outputting the most relevant (e.g., 1-3) segments and their associated QA pairs.

[0055] In this embodiment, information fidelity can be improved, the structured restoration rate of scanned documents or PPTs can be increased, medical knowledge can be completely stored in the database, and the accuracy of retrieval can also be improved. Through the semantic enhancement strategy of directory segmentation and global QA generation, the accuracy of cross-chapter question answering can be improved.

[0056] Optionally, in step A1, based on the problem information, first information related to liver cancer of the target object is obtained through a locally pre-built first knowledge base, including steps A11 to A12: Step A11: Encode the problem information to obtain the problem feature vector.

[0057] Step A12: Based on the question feature vector, obtain the fragments and question-answer pairs related to the target object in the first knowledge base.

[0058] In this embodiment, encoding the question information to obtain a question feature vector allows for in-depth analysis of the question, extracting key features closely related to liver cancer and the target individual, such as the target individual's age, liver cancer type, and specific symptoms. The encoding process transforms this key information into specific dimensional values ​​within the vector, accurately describing the key points of the question. When retrieving relevant segments and question-answer pairs from the first knowledge base based on the question feature vector, it ensures matching highly relevant information, avoiding the acquisition of a large amount of irrelevant content and improving the accuracy of information retrieval. Furthermore, the encoded feature vector is concise and standardized, facilitating rapid retrieval and filtering in the knowledge base, significantly reducing the search scope and time. Automated information retrieval reduces human intervention in the information retrieval process, minimizing manual labor and the influence of subjective factors. Through encoding and vector matching, the system can automatically complete information filtering and extraction, improving the efficiency of information retrieval.

[0059] In one feasible embodiment, the target task performed includes image analysis based on the medical image information, comprising at least one of the following steps B1 to B2: Step B1: Based on the medical image information, perform processing for central liver cancer to obtain third information.

[0060] Optionally, the processing of central hepatocellular carcinoma may include image preprocessing, liver region segmentation, extracting features of the hepatocellular carcinoma lesions from the segmented liver region using image feature extraction methods, and then using a classification algorithm to subtype the central hepatocellular carcinoma based on the extracted features. Accurate subtyping of central hepatocellular carcinoma can provide medical staff with effective information for the entire course of disease management, helping to develop corresponding personalized full-course management plans tailored to the target population.

[0061] For example, in image preprocessing, input medical images (such as CT and MRI images) can be preprocessed to eliminate noise, artifacts, and other interfering factors, thereby improving image quality. For instance, Gaussian filtering and median filtering can be used to smooth the image and reduce random noise; histogram equalization can be used to enhance image contrast, making structures such as liver tissue and lesions more clearly distinguishable. Image standardization can also be performed to unify the image size and grayscale range for subsequent processing. For example, the image size can be adjusted to a fixed pixel size, and the grayscale values ​​can be normalized to the range [0,1].

[0062] For example, in liver region segmentation, image segmentation algorithms (such as thresholding, region growing, deep learning segmentation models, etc.) can be used to accurately segment the liver region from medical images. For instance, the UNet model based on deep learning is trained on a large number of labeled liver images, enabling it to automatically learn the morphological and textural features of the liver, so as to achieve liver segmentation through a pre-trained model.

[0063] For example, in the classification of central hepatocellular carcinoma, features of the hepatocellular carcinoma lesions can be extracted using image feature extraction methods within the segmented liver region. Based on the extracted features, classification algorithms (such as support vector machines, random forests, deep learning classification models, etc.) are used to classify central hepatocellular carcinoma. For instance, a convolutional neural network (CNN) based on deep learning, trained on a large number of central hepatocellular carcinoma classification samples, can automatically learn the feature differences between different classifications, thereby achieving accurate classification of the images in the example.

[0064] Step B2: Based on the medical image information, perform at least one of the following: lesion detection, microvascular invasion (MVI) sign analysis, and satellite lesion detection.

[0065] Optionally, lesion detection can use methods such as sliding window or selective search to generate possible lesion candidate regions in the image, then extract features from each candidate region, and use a classifier to determine whether the corresponding candidate region is a lesion, thereby achieving automatic lesion detection.

[0066] Optionally, microvascular invasion (MVI) sign analysis can analyze image features to identify those related to microvascular invasion, such as the blurring of tumor margins, the morphology and distribution of blood vessels around the tumor, etc., and calculate corresponding quantitative indicators to assess the likelihood of MVI, such as the distance between the tumor and surrounding blood vessels (the shorter the distance, the greater the likelihood of MVI), the density of blood vessels, etc. Then, machine learning or deep learning models can be used to predict MVI by combining the extracted image features and quantitative indicators.

[0067] Optionally, satellite lesion detection can first perform background removal and other operations on the liver image to highlight the liver tissue and potential satellite lesions. Then, appropriate small lesion detection algorithms can be used to detect satellite lesions, such as multi-scale analysis and local contrast enhancement, analyzing images at different scales to enhance the contrast between small lesions and surrounding tissues and improve the detection rate of satellite lesions. Subsequently, the detected satellite lesions can be verified and confirmed, such as by combining other image sequences (e.g., CT or MRI images from different periods) or clinical information to improve the accuracy of satellite lesion detection.

[0068] Optionally, in step B1, based on the medical image information, central hepatocellular carcinoma is classified to obtain third information, including steps B11 to B13: Step B11: Based on the medical image information, perform individualized Fang's classification of central hepatocellular carcinoma to obtain the classification results.

[0069] Step B12: Based on the medical image information, assess the liver condition of the target object and obtain the assessment result.

[0070] Step B13: Based on the classification results and the evaluation results, obtain third information for the diagnosis of central hepatocellular carcinoma.

[0071] Optional, such as Figure 3 As shown in the embodiments of this disclosure, during the whole-course management of central hepatocellular carcinoma (HCC) classification, the system performs individualized Fang's classification (such as individualized hepatic artery classification, individualized portal vein classification, individualized right portal vein classification, and individualized hepatic vein classification) to identify the specific type and characteristics of the tumor. Simultaneously, the system assesses the patient's overall liver condition, including individualized Fang's segmentation of the liver, individualized liver volume calculation, vascular-centric assessment, and individualized biliary reconstruction, providing comprehensive information for subsequent diagnosis by medical personnel. Combining the individualized classification and liver assessment results, the system makes a diagnosis of central HCC and generates a detailed assessment report.

[0072] Optionally, the Fang classification can be used to clarify the relationship between the tumor and blood vessels, providing a basis for surgical approach selection, such as planning resection margins. The management of the Fang classification facilitates individualized treatment, providing decision support for medical staff to develop targeted surgical plans based on the patient's specific classification, thus improving the overall management of the disease. Specifically, individualized hepatic artery classification can clarify the blood supply source of the tumor, providing a basis for interventional procedures or vascular ligation. Individualized portal vein classification can assess the extent of tumor invasion of the portal vein, guiding the determination of the surgical scope and the management of portal vein tumor thrombi. Individualized right hepatic portal vein classification can clarify the anatomical variations of the right hepatic portal vein and the relationship between the tumor and the right hepatic portal vein for right hepatic tumors, guiding surgeries such as right hemihepatectomy or right trilobectomy. Individualized hepatic vein classification can assess the extent of tumor invasion of the hepatic veins, dynamically guiding hepatic vein management and calculating the remaining liver volume.

[0073] In this embodiment, on the one hand, the Fang's classification is used to classify central hepatocellular carcinoma. This method comprehensively considers factors such as the growth pattern of the hepatocellular carcinoma within the liver, its relationship with blood vessels, and the boundary characteristics of the tumor. Compared with existing technologies, this improves the accuracy of classification and more accurately identifies different types of central hepatocellular carcinoma. On the other hand, the liver condition of the target subject is assessed, which can include multiple dimensions such as liver morphology, function, and the presence of other diseases (e.g., cirrhosis, fatty liver). This assessment of liver condition helps to more accurately determine the environment and background of the hepatocellular carcinoma, enriching the information for the entire course of disease management. Based on this, the classification results and assessment results are combined to obtain the third information for the assessment of central hepatocellular carcinoma, which can provide a reference for the formulation of personalized full-course management plans and improve the level of medical management and decision-making.

[0074] Optionally, the medical image information includes image information from CT and / or MRI arterial and portal venous phase sequences.

[0075] Optionally, step B11, which involves performing individualized Fang's classification of central hepatocellular carcinoma based on the medical image information to obtain the classification result, includes steps B111 to B112: Step B111: Extract image features based on the medical image information using a convolutional neural network.

[0076] Step B112: Based on the image features, match them with the classification rules in the preset classification rule library to determine the classification result. The classification rules include mass type, infiltrative type and mixed type. The classification result includes at least one of individualized hepatic artery classification, individualized portal vein classification, individualized right hepatic portal vein classification or individualized hepatic vein classification.

[0077] Optionally, the convolutional neural network can employ a Transformer or TransUNet architecture trained on liver cancer imaging data.

[0078] Optionally, in the classification of central hepatocellular carcinoma, the input can include enhanced CT / MRI arterial and portal venous phase sequences. Then, a convolutional neural network can be used to extract image features (such as capsule integrity, enhancement homogeneity, and necrosis percentage). Based on the extracted image features, a classification rule base is matched (e.g., mass type: intact capsule > 90%; infiltrative type: diffuse enhancement; mixed type: meets dual features), and the classification result is output based on the matching result (e.g., individualized hepatic artery classification, individualized portal vein classification, individualized right portal vein classification, and individualized hepatic vein classification). Optionally, key image evidence annotations can also be output simultaneously.

[0079] Optionally, before performing the central typing process, the method further includes step B01: performing three-dimensional imaging of the liver of the target object based on the medical image information to obtain three-dimensional image data.

[0080] Optional, such as Figure 3 As shown, CT and MRI technologies can be used to perform three-dimensional imaging of a patient's liver, generating detailed three-dimensional image data. Based on this three-dimensional image data, individualized Fang's classification of central hepatocellular carcinoma and assessment of liver condition are performed.

[0081] Optionally, the method provided in this disclosure further includes C1: performing three-dimensional visualization typing of central hepatocellular carcinoma based on the third information, and performing at least one of the following steps C11 to C13 based on the processing result: Step C11: Using a 3D printing device, the corresponding 3D model is converted into a solid model, which includes a physical model of the tumor and surrounding tissues.

[0082] Step C12: Display via virtual reality device.

[0083] Step C13: Perform the simulated surgery and obtain the results of the simulated surgery.

[0084] Optionally, the system can further perform three-dimensional visualization and classification of central hepatocellular carcinoma to assist in a more intuitive understanding of the tumor's location, size, and morphology. For example, the system can be linked with other systems, such as converting the three-dimensional model into a physical model, creating a physical model of the tumor and surrounding tissues using 3D printing technology for preoperative planning or doctor-patient communication; using virtual reality technology to display the three-dimensional model, allowing doctors to observe and analyze the tumor from multiple angles in a virtual environment, improving the accuracy of surgical planning; and based on the three-dimensional model and virtual reality technology, the system can simulate the surgical process, performing a simulated surgery to help doctors rehearse surgical steps in advance.

[0085] Augmented Reality (AR) is a technology that integrates virtual information (such as text, images, music, and videos) with the real world. In practice, real-world data is captured using cameras and sensors, and then the virtual information is mixed and overlaid with the real-world data to generate a new virtual image. This virtual image is then presented to the user through display devices (such as AR glasses and mobile phones), allowing the user to interact with the virtual information through intelligent methods (such as voice recognition and gesture recognition) (AR interaction).

[0086] The solution provided in this disclosure achieves intelligent closed-loop management of liver cancer through the following three processes: Procedure 1: Primary Liver Cancer Assessment Procedure, such as... Figure 2 As shown, the system first obtains the patient's chief complaint information. Based on this information, it plans which blood tests and imaging examinations can be performed. Then, the system performs a comprehensive analysis, combining the actual test results (blood tests) with CT and MRI image data. Using a convolutional neural network model algorithm, it locates and identifies suspicious lesions and generates an assessment report.

[0087] Step 2: Three-dimensional visualization assessment pathway for central hepatocellular carcinoma, such as... Figure 3 As shown, the specific details can be found in the descriptions of steps B11 to B13 and steps C11 to C13 in the above embodiments.

[0088] Step 3: Closed-loop management of the entire liver cancer surgery process, such as... Figure 4 As shown, the system can assist surgeons in developing detailed surgical plans by analyzing the types of augmented and mixed reality-guided laparoscopic liver resections. This includes information such as the surgical path, resection extent, and challenges and technical requirements of augmented reality combined with ICG molecular fluorescence navigation in laparoscopic liver resection. The system can also be linked to an intraoperative navigation system, using information from the intraoperative navigation system and surgical reports, combined with the patient's postoperative clinical data, to conduct postoperative assessments.

[0089] Below are some feasible application examples.

[0090] Application Example 1: Preoperative Auxiliary Analysis and Management for Physicians User input: The surgeon enters the following into the system input device at the ward workstation: "Patient Zhang XX is scheduled for liver cancer surgery. Please analyze his latest enhanced CT scan to check the tumor location, size, vascular invasion, and any satellite lesions. Also, please retrieve his pathology report and last postoperative follow-up record." Afterward, the full-process management system of this application automatically retrieves the patient's latest abdominal enhanced CT DICOM image stored locally.

[0091] The system processing flow is as follows: (a) Speech Recognition and Intent Parsing (Edge): The ASR module converts speech into text. Intelligent agent parses intent: requirements include image analysis (tumor location, size, MVI, satellite lesions) and accessing / associating text reports (pathology, postoperative follow-up).

[0092] (b) Document Retrieval and Association (Edge + Cloud): Retrieve the pathology report (PDF) and postoperative follow-up record (Word) of patient "Zhang XX" from local encrypted storage. If it is a scanned PDF pathology report, call the local VLM module to parse its structured information (including key diagnostic text and any tabular data).

[0093] Specifically, the user's question "tumor location, size..." is encoded, and relevant document fragments and associated QA documents for that patient are queried from the local knowledge base. Simultaneously, non-sensitive, generalized parts of the question (such as "liver cancer satellite lesion imaging features") are sent to the cloud-based knowledge base to retrieve public medical knowledge. A rearrangement model is then used to integrate the most relevant results returned from both the local and cloud-based databases.

[0094] (c) Image analysis task breakdown and execution (ReAct-edge): Reasoning: Task chain = [1. CT liver lesion detection and segmentation; 2. Measure lesion size and location; 3. Analyze MVI signs; 4. Detect satellite lesions].

[0095] Action: Sequentially invoke the dedicated image analysis API deployed on the edge workstation or hospital intranet: Liver_Lesion_Segmentation_API(CT_Image) → Returns the segmentation mask and lesion coordinates.

[0096] Lesion_Measurement_API(Segmentation_Result) → Returns the maximum diameter, volume, and liver segment position.

[0097] MVI_RadScore_API(CT_Image, Segmentation_Result) → Returns the MVI probability and image feature description.

[0098] Satellite_Detection_API(CT_Image, Main_Lesion_Coord) → Returns the number and location of satellite foci.

[0099] (d) Results integration and report generation (edge): The intelligent agent receives: 1) locally retrieved patient structured pathology reports ("moderately differentiated hepatocellular carcinoma") and key indicators for postoperative follow-up; 2) various results returned by the image analysis API; 3) relevant summaries on "prognostic significance of satellite lesions" retrieved from the cloud / local knowledge base.

[0100] The above information is integrated and input into a large language model (which can be deployed in a lightweight manner at the edge or securely accessed in the cloud) to generate a structured preoperative analysis report.

[0101] The system output (multimodal-edge) includes the following: Text report (displayed on the workstation screen): Preoperative imaging assessment report (Patient: Zhang XX) Lesion: A single mass in the right posterior lobe of the liver (S7 segment), with a maximum diameter of approximately 3.5 cm. A 3D reconstruction location diagram can be provided (either directly or via a link).

[0102] MVI assessment: Based on imaging features (RadScore 0.78), it indicates a high risk of microvascular invasion (probability >80%).

[0103] Satellite lesion: A suspicious satellite lesion (approximately 0.8 cm in diameter) was found next to the main lesion (within 2 cm). It is recommended to focus on exploring it during surgery.

[0104] Related information: Pathology (202X-XX-XX): Moderately differentiated hepatocellular carcinoma; Postoperative (202X-XX-XX) follow-up AFP: 120ng / mL (not reduced to normal).

[0105] Overall assessment: The tumor is resectable, but the high risk of MVI and satellite lesions suggest a high risk of postoperative recurrence. It is recommended to conduct thorough intraoperative ultrasound examination and develop a plan for intensive postoperative follow-up and adjuvant therapy. See Section XX of the "Guidelines for the Diagnosis and Treatment of Primary Liver Cancer (202X Edition)" for details.

[0106] Visualization (Edges): An interactive view that overlays lesion segmentation contours and satellite lesion location markers onto CT images.

[0107] In this embodiment, considering data security, all original DICOM images, pathology reports, and structured reports of the patient are stored and processed locally on the processing workstation (edge ​​device). External queries may only involve secure encryption when generating report summaries that reference public guidelines. Complete reports can be encrypted and exported as PDFs for archiving in the electronic medical record system. Furthermore, regarding report export and archiving, after the doctor confirms the report, the system packages the structured text report, key image annotations, and audio recordings into encrypted ZIP files and standard PDF documents, storing them on the hospital's local server or secure cloud storage, and synchronizing them to the electronic medical record system.

[0108] Furthermore, considering that multiple assessments may be conducted throughout the entire course of liver cancer management, to improve management quality, each assessment report will also include supplementary management recommendations following the current assessment. These recommendations include, but are not limited to, other relevant supplementary testing items, multidisciplinary collaboration suggestions, key patient education content, and prognostic influencing factors. These supplementary management contents may not be essential basic items, but they can broaden surgeons' focus on specialty knowledge and doctor-patient communication methods. The generation of these supplementary management contents is based on the aforementioned first and / or second knowledge bases, but the emphasis is on assisting surgeons and medical teams in strengthening the "standardized, individualized, and full-cycle" characteristics of liver cancer management from the perspectives of clinical precision, teamwork, patient safety, and scientific management.

[0109] Application Example 2: Surgical Planning and Postoperative Assessment User input: The surgeon inputs on the system input device in the operating room: "Patient Zhang XX, central hepatocellular carcinoma of the right lobe, 3D printed model and VR pre-visualization have been completed, real-time navigation of vascular distribution and surgical field risk areas is required during the operation." The system processing flow is as follows: Retrieve preoperative data: Extract key vascular anatomy information and combine it with a local knowledge base to perform three-dimensional visualization of central hepatocellular carcinoma classification (types I to V), providing preoperative planning and enhanced and mixed reality navigation prompts for laparoscopic liver resection (1. Using the superior and inferior vena cava fossa and the bottom of the gallbladder as marker points, perform image fusion and registration in combination with the shape of the liver edge; 2. Using the main trunk or bifurcation of the portal vein, the hepatic artery or the abdominal aorta as registration markers for real-time image fusion navigation, visualizing the positional relationship between the hepatic artery and the main trunk of the portal vein within the first hepatic hilum; 3. Navigate the dissection of the proper hepatic artery and the right hepatic (left) artery using a three-dimensional reconstruction model; 4. Dissect the main trunk of the portal vein and the right (left) branch of the portal vein, ligate and detach the right (left) hepatic artery and the right (left) branch of the portal vein; 5. Use positive or negative staining fluorescence imaging of the liver resection line; 6. Perform liver parenchyma detachment along the fluorescent line marking, and perform real-time fusion and interactive navigation of the hepatic veins and their tributaries.) Postoperative assessment: Based on the surgical report, postoperative clinical information and pathology report, generate rehabilitation suggestions and follow-up plans (such as "AFP check 1 month after surgery, enhanced MRI every 3 months").

[0110] The system output is as follows: Postoperative report: Integrating imaging and pathology data to generate a personalized rehabilitation plan.

[0111] This disclosure provides a digital and intelligent management method for the entire course of liver cancer based on a large-scale intelligent model. This method integrates a digital and intelligent liver cancer specialty module (such as a central intelligent classification system and a multimodal intelligent agent architecture deployed at the edge, including staging, central classification, preoperative planning, and intraoperative guidance) with a structured knowledge base of hepatobiliary and pancreatic diseases (covering more than 10 sub-domains such as expert consensus, group standards, and 3D visualization), combined with a multimodal document processing engine, edge-deployed intelligent agents, and dynamic knowledge management technology, to achieve closed-loop management of the entire liver cancer diagnosis and treatment process from initial screening to postoperative evaluation. The method employs several approaches. Firstly, it utilizes a Visual Language Model (VLM) to precisely parse medical documents, preserving the semantics of tables and images. Secondly, an intelligent agent based on the ReAct architecture schedules specialized toolchains (such as lesion detection and identification, and 3D visualization classification of central hepatocellular carcinoma) at the edge. Thirdly, it constructs a digital and intelligent local knowledge base for hepatocellular carcinoma (e.g., 7 categories of digital intelligence + 10 categories of 3D visualization) to support precise knowledge retrieval. Fourthly, it can also generate 3D surgical planning schemes preoperatively (highlighting the difficulties and technical requirements of laparoscopic liver resection using augmented reality combined with ICG molecular fluorescence navigation). The implementation of this application can assist physicians in the entire process of hepatocellular carcinoma management, especially providing structured support for 3D visualization classification (types I-V), intraoperative navigation, and postoperative evaluation of central hepatocellular carcinoma.

[0112] The output results of the methods provided in this disclosure are all intermediate processed data and do not directly provide or are equivalent to disease diagnosis conclusions. The final clinical judgment should be made by qualified medical personnel in combination with the patient's specific situation.

[0113] This disclosure provides an electronic device including a memory, a processor, and a computer program stored in the memory. The processor executes the computer program to implement the steps of the method provided in any optional embodiment of this disclosure.

[0114] In one alternative embodiment, an electronic device is provided, such as Figure 6 As shown, Figure 6The illustrated electronic device 4000 includes a processor 4001 and a memory 4003. The processor 4001 and the memory 4003 are connected, for example, via a bus 4002. Optionally, the electronic device 4000 may further include a transceiver 4004, which can be used for data interaction between the electronic device and other electronic devices, such as sending and receiving data. It should be noted that in practical applications, the transceiver 4004 is not limited to one type, and the structure of the electronic device 4000 does not constitute a limitation on the embodiments of this disclosure.

[0115] Processor 4001 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It may implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with this disclosure. Processor 4001 may also be a combination that implements computational functions, such as including one or more microprocessor combinations, a combination of a DSP and a microprocessor, etc.

[0116] Bus 4002 may include a pathway for transmitting information between the aforementioned components. Bus 4002 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. Bus 4002 can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 6 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0117] The memory 4003 may be ROM (Read Only Memory) or other types of static storage devices capable of storing static information and instructions, RAM (Random Access Memory) or other types of dynamic storage devices capable of storing information and instructions, or EEPROM (Electrically Erasable Programmable Read Only Memory), CD-ROM (Compact Disc Read Only Memory) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media, other magnetic storage devices, or any other medium capable of carrying or storing computer programs and capable of being read by a computer, without limitation herein.

[0118] The memory 4003 is used to store computer programs that execute embodiments of the present disclosure, and is controlled by the processor 4001 to execute them. The processor 4001 is used to execute the computer programs stored in the memory 4003 to implement the steps shown in the foregoing method embodiments.

[0119] This disclosure provides a computer-readable storage medium storing a computer program, which, when executed by a processor, can implement the steps and corresponding content of the aforementioned method embodiments.

[0120] This disclosure also provides a computer program product, including a computer program that, when executed by a processor, can implement the steps and corresponding content of the aforementioned method embodiments.

[0121] It should be understood that although arrows indicate various operation steps in the flowcharts of the embodiments of this disclosure, the order in which these steps are implemented is not limited to the order indicated by the arrows. Unless explicitly stated herein, in some implementation scenarios of the embodiments of this disclosure, the implementation steps in each flowchart can be executed in other orders as required. Furthermore, some or all of the steps in each flowchart may include multiple sub-steps or multiple stages based on the actual implementation scenario. Some or all of these sub-steps or stages can be executed at the same time, and each sub-step or stage can also be executed at different times. In scenarios where execution times differ, the execution order of these sub-steps or stages can be flexibly configured as required, and the embodiments of this disclosure do not limit this.

[0122] The above description is only an optional implementation method for some implementation scenarios of this disclosure. It should be noted that for those skilled in the art, other similar implementation methods based on the technical concept of this disclosure without departing from the technical concept of this disclosure also fall within the protection scope of the embodiments of this disclosure.

Claims

1. A digital intelligent management method for the entire course of liver cancer based on a large-scale intelligent model agent, characterized in that, include: The input multimodal information includes question information and medical image information related to the target object. The question information is used to describe information related to liver cancer of the target object, and the medical image information includes information obtained from liver cancer-related examinations of the target object. Based on the problem information, a target task to be processed is determined, and the target task includes at least one of linking medical history with the constructed knowledge base and image analysis based on the medical image information. Based on the results of performing the target task, the structured analysis information of liver cancer corresponding to the target object is output.

2. The method according to claim 1, characterized in that, If the target task includes associating medical history with a constructed knowledge base, then the execution of the target task includes at least one of the following: Based on the problem information, first information related to liver cancer of the target object is obtained through a locally pre-built first knowledge base, wherein the first knowledge base includes content built based on liver cancer knowledge; Based on the aforementioned problem information, second information related to liver cancer is obtained through a second knowledge base pre-built in the cloud. The second knowledge base includes content built based on public medical knowledge.

3. The method according to claim 2, characterized in that, The first knowledge base was constructed through the following operations: Differentiated parsing for multimodal documents; Based on the parsing results, the document is divided into directory-guided segments; Using a large language model, several question-answer pairs are generated for each segment, which include the content of that segment and are related to other parts of the document. The pre-trained text model encodes each text segment into a feature vector for storage.

4. The method according to claim 3, characterized in that, Based on the problem information, the first information related to liver cancer of the target object is obtained through a locally pre-built first knowledge base, including: The problem information is encoded to obtain a problem feature vector; Based on the question feature vector, fragments and question-answer pairs related to the target object are obtained from the first knowledge base.

5. The method according to claim 1, characterized in that, The target task being performed includes image analysis based on the medical image information, comprising at least one of the following: Based on the aforementioned medical image information, central hepatocellular carcinoma is identified to obtain third information; Based on the medical image information, at least one of the following is performed: lesion identification, microvascular invasion (MVI) sign analysis, and satellite lesion identification.

6. The method according to claim 5, characterized in that, The step of performing at least one of the following based on the medical image information: lesion identification, microvascular invasion MVI sign analysis, and satellite lesion identification: Based on the medical image information, multi-scale feature extraction is performed to generate an image feature map of the liver region; Based on the medical image information, semantic segmentation is performed, and the contour coordinate array and volume measurement value of the lesion region are output. Based on the medical image information, texture analysis is performed to output a quantitative score reflecting the probability of microvascular invasion; Based on the medical image information, a neighborhood search is performed to output the location and confidence level of the suspected satellite lesion.

7. The method according to claim 5, characterized in that, The process of identifying central hepatocellular carcinoma based on the medical image information to obtain third information includes: Based on the aforementioned medical image information, individualized Fang's classification of central hepatocellular carcinoma is performed to obtain the classification results; Based on the medical image information, the liver condition of the target subject is assessed, and the assessment result is obtained; Based on the evaluation results, a similarity match is performed with a preset subtyping feature library, and at least one subtyping label with the highest matching degree and its probability vector are output.

8. The method according to claim 7, characterized in that, The medical image information includes image information from CT and / or MRI arterial and portal venous phase sequences; Based on the medical image information, the Fang's classification of central hepatocellular carcinoma is performed to obtain the classification results, including: Image features are extracted based on the medical image information using a convolutional neural network; Based on the image features, the classification results are determined by matching them with the classification rules in the preset classification rule library. The classification rules include mass type, infiltrative type and mixed type. The classification results include at least one of individualized hepatic artery classification, individualized portal vein classification, individualized right hepatic portal vein classification or individualized hepatic vein classification.

9. The method according to claim 7 or 8, characterized in that, Before performing the central type classification process, the following is also included: Based on the medical image information, three-dimensional imaging of the liver of the target object is performed to obtain three-dimensional image data; The method further includes: Based on the aforementioned third information, a three-dimensional visualization classification of central hepatocellular carcinoma is performed, and at least one of the following operations is executed based on the processing result: The corresponding 3D model is converted into a solid model using 3D printing equipment. This solid model includes a physical model of the tumor and surrounding tissues. Displayed via virtual reality devices; Perform a simulated surgery and obtain the simulated surgical results.

10. A digital intelligent management system for the entire course of liver cancer based on a large-scale intelligent agent model, characterized in that, include: A multimodal input interface is used to acquire input multimodal information, which includes question information and medical image information related to the target object. The question information is used to describe information related to liver cancer of the target object, and the medical image information includes information obtained from liver cancer-related examinations of the target object. An intent parsing module is used to determine the target task to be processed based on the question information. The target task includes at least one of associating medical history with a constructed knowledge base and image analysis based on the medical image information. The task scheduling engine, based on the ReAct architecture, is used to break down the task chain and call the corresponding modules according to the intent parsing results. The knowledge base retrieval module is used to obtain first information related to liver cancer of the target object based on the question information and through a first knowledge base pre-built locally. It also obtains secondary information related to liver cancer through a pre-built second knowledge base in the cloud, which includes content built based on public medical knowledge; The image analysis module is used to perform image analysis on the medical image information; The structured report generation module is used to output structured analysis information of liver cancer corresponding to the target object based on the results of performing the target task.

11. An electronic device comprising a memory, a processor, and a computer program stored in the memory, characterized in that, The processor executes the computer program to implement the steps of the method according to any one of claims 1 to 9.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 9.

13. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 9.

Images