Brain disease assessment method and device, electronic equipment and storage medium

Abstract

The invention provides a brain disease assessment method and device, electronic equipment, a storage medium and a computer program product, and relates to the technical field of computers. The method comprises the steps of obtaining multiple pieces of target data; the target data is examination data of a target object about different modes of the brain; performing multi-modal feature extraction and fusion on the target data to obtain target features; in the process of generating an evaluation result according to the target feature, introducing a retrieval mechanism based on a brain disease knowledge base to perform enhanced retrieval on the target feature to obtain the evaluation result of the brain disease; the evaluation result comprises the brain disease condition and the treatment suggestion of the target object. According to the method and the device, the problem of low accuracy of brain disease assessment in related technologies is solved.

Description

Technical Field

[0001] The present application relates to the field of medical technology. Specifically, the present application relates to a brain disease assessment method, device, electronic device and storage medium. Background Art

[0002] With the rapid development of artificial intelligence (AI) technology, AI models have shown great potential in the medical field, particularly in diagnosing and recommending treatments for brain diseases. However, existing AI models often rely on the characteristics of existing data and may lack sufficient professionalism and authority.

[0003] From the above, we can see that how to improve the accuracy of brain disease assessment remains to be solved. Summary of the Invention

[0004] This application provides a brain disease assessment method, device, electronic device, and storage medium, which can solve the problem of low accuracy of brain disease assessment in related technologies. The technical solution is as follows:

[0005] According to one aspect of the present application, a brain disease assessment method includes: acquiring multiple target data; the target data are examination data of different modalities of the target object's brain; performing multimodal feature extraction and fusion on each of the target data to obtain target features; in the process of generating an assessment result based on the target features, introducing a retrieval mechanism based on a brain disease knowledge base to perform enhanced retrieval on the target features to obtain the assessment result of the brain disease; the assessment result includes the brain disease status of the target object and treatment recommendations.

[0006] According to one aspect of the present application, a brain disease assessment device includes: a data acquisition module for acquiring a plurality of target data; the target data are examination data of a target object in different modalities regarding the brain; a feature processing module for performing multimodal feature extraction and fusion on each of the target data to obtain target features; a disease assessment module for introducing a retrieval mechanism based on a brain disease knowledge base to perform enhanced retrieval on the target features in the process of generating an assessment result based on the target features to obtain the assessment result of the brain disease; the assessment result includes the brain disease condition of the target object and treatment recommendations.

[0007] According to one aspect of the present application, an electronic device includes at least one processor and at least one memory, wherein the memory stores a computer program, and when the computer program is executed by the processor, the brain disease assessment method as described above is implemented.

[0008] According to one aspect of the present application, a storage medium stores a computer program thereon, and when the computer program is executed by one or more processors, the brain disease assessment method as described above is implemented.

[0009] According to one aspect of the present application, a computer program product includes a computer program, which implements the brain disease assessment method described above when executed by one or more processors.

[0010] The beneficial effects of the technical solution provided by this application are:

[0011] In this technical solution, automated multimodal data processing and feature fusion reduce manual intervention and improve diagnostic efficiency. This method can simulate a multidisciplinary consultation model, fully leveraging the complementary nature of various medical data types to enhance diagnostic comprehensiveness and interpretable decision-making. It provides more accurate, professional, and efficient brain disease diagnosis and treatment recommendations, effectively addressing the low accuracy of brain disease assessments in related technologies. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for describing the embodiments of the present application. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without inventive efforts.

[0013] Figure 1 It is a schematic diagram of the implementation environment involved in this application;

[0014] Figure 2 is a hardware structure diagram of a server according to an exemplary embodiment;

[0015] Figure 3 is a flow chart showing a method for evaluating a brain disease according to an exemplary embodiment;

[0016] Figure 4 yes Figure 3 A flowchart of step 330 in one embodiment corresponding to the embodiment;

[0017] Figure 5 yes Figure 3 The steps before step 350 in the corresponding embodiment are a flowchart of an embodiment;

[0018] Figure 6 yes Figure 3 A flowchart of an embodiment corresponding to step 350 in an embodiment;

[0019] Figures 7 to 9This is a schematic diagram of a specific implementation of a brain disease assessment method in an application scenario;

[0020] Figure 10 is a structural block diagram of a brain disease assessment device according to an exemplary embodiment;

[0021] Figure 11 The figure is a structural block diagram of an electronic device according to an exemplary embodiment. DETAILED DESCRIPTION

[0022] The following describes in detail embodiments of the present application, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present application, and are not to be construed as limiting the present application.

[0023] It will be understood by those skilled in the art that, unless expressly stated otherwise, the singular forms "a", "an", "said" and "the" used herein may also include the plural forms. It should be further understood that the term "comprising" used in the specification of the present disclosure refers to the presence of the features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof. It should be understood that when we refer to an element as being "connected" or "coupled" to another element, it may be directly connected or coupled to the other element, or there may be intermediate elements. In addition, "connected" or "coupled" as used herein may include wireless connections or wireless couplings. The term "and / or" used herein includes all or any units and all combinations of one or more associated listed items.

[0024] As mentioned earlier, existing AI models often rely on the characteristics of existing data and may lack sufficient professionalism and authority.

[0025] While existing artificial intelligence (AI) models have made some progress in diagnosing and recommending treatments for brain diseases, they still have the following major shortcomings:

[0026] (1) Insufficient professionalism and authority: Existing AI models often rely on the characteristics of existing data and may lack sufficient professionalism and authority. For example, there are significant differences in diagnostic criteria between doctors, which affects the quality and consistency of AI model training data.

[0027] (2) Difficulty in fusing multimodal data: Brain disease diagnosis involves data from multiple modalities, such as behavioral videos, imaging data (EEG, MEG, fMRI, PET, NIRS, etc.), genomes, clinical records, etc. Existing AI models face challenges in effectively fusing these heterogeneous data, making it difficult to fully capture the patient's condition.

[0028] From the above, it can be seen that the relevant technology still has the defect of low accuracy in brain disease assessment.

[0029] To this end, the brain disease assessment method provided in this application can effectively improve the accuracy of brain disease assessment. Accordingly, the brain disease assessment method is suitable for a brain disease assessment device, which can be deployed in an electronic device. The electronic device can be a computer device configured with a von Neumann architecture, for example, the computer device includes a desktop computer, a laptop computer, a server, etc.

[0030] In order to make the objectives, technical solutions and advantages of this application clearer, the implementation methods of this application will be further described in detail below with reference to the accompanying drawings.

[0031] Figure 1 This is a schematic diagram of an implementation environment involved in a brain disease assessment method. It should be noted that this implementation environment is only an example adapted to the present invention and should not be considered as providing any limitation on the scope of application of the present invention.

[0032] The implementation environment includes a collection end 110 and a service end 130 .

[0033] Specifically, the acquisition terminal 110 can also be considered as an examination device, including but not limited to electronic devices used for medical examinations such as CT equipment and MRI equipment. For example, the acquisition terminal 110 is a CT device.

[0034] Server 130 can be an electronic device such as a desktop computer, laptop computer, or server. It can also be a computer cluster consisting of multiple servers, or even a cloud computing center consisting of multiple servers. Server 130 is used to provide backend services, including, but not limited to, brain disease assessment services.

[0035] A network communication connection is pre-established between the server 130 and the collection terminal 110 via a wired or wireless method, and data transmission between the server 130 and the collection terminal 110 is achieved via the network communication connection. The transmitted data includes but is not limited to target data and the like.

[0036] In one application scenario, through the interaction between the acquisition terminal 110 and the server 130 , the acquisition terminal 110 obtains a target image by inspecting the target object, and uploads the target image to the server 130 to request the server 130 to provide brain disease assessment services.

[0037] For the server 130, after receiving the target data uploaded by the acquisition terminal 110, it calls the brain disease assessment service, performs multimodal feature extraction and fusion on each target data, and obtains the target feature; in the process of generating the assessment result according to the target feature, a retrieval mechanism based on the brain disease knowledge base is introduced to perform enhanced retrieval on the target feature to obtain the assessment result of the brain disease; the assessment result includes the brain disease condition of the target object and treatment recommendations, thereby solving the problem of low accuracy of brain disease assessment in related technologies.

[0038] See also Figure 2 , Figure 2 This is a hardware structure diagram of a server according to an exemplary embodiment. Figure 1 A server 130 is shown in an implementation environment.

[0039] It should be noted that the server is only an example adapted for this application and cannot be considered to provide any limitation on the scope of use of this application. The server cannot be interpreted as needing to rely on or must have Figure 2 One or more components of exemplary server 200 are shown.

[0040] The hardware structure of the server 200 may vary greatly due to different configurations or performances, such as Figure 2 As shown, the server 200 includes a power supply 210 , an interface 230 , at least one memory 250 , and at least one central processing unit (CPU) 270 .

[0041] Specifically, the power supply 210 is used to provide operating voltage for each hardware device on the server 200 .

[0042] The interface 230 includes at least one wired or wireless network interface 231 for interacting with external devices. Figure 1 The interaction between the collection end 110 and the service end 130 in the implementation environment is shown.

[0043] Of course, in other examples adapted by this application, the interface 230 may further include at least one serial-to-parallel conversion interface 233, at least one input-output interface 235, and at least one USB interface 237, etc. Figure 2 As shown, this does not constitute a specific limitation.

[0044] The memory 250 serves as a carrier for resource storage and can be a read-only memory, random access memory, disk or optical disk, etc. The resources stored thereon include an operating system 251, application 253 and data 255, etc. The storage method can be temporary storage or permanent storage.

[0045] Among them, the operating system 251 is used to manage and control the hardware devices and application programs 253 on the server 200 to enable the central processing unit 270 to calculate and process the massive data 255 in the memory 250. It can be WindowsServerTM, Mac OS XTM, UnixTM, LinuxTM, FreeBSDTM, etc.

[0046] The application 253 is a computer program formed by computer-readable instructions based on the operating system 251 to perform at least one specific task, and may include at least one module ( Figure 2 Each module may include corresponding computer-readable instructions. For example, the brain disease assessment device may be considered as an application 253 deployed on the server 200.

[0047] The data 255 may be photos, pictures, etc. stored in a disk, or may be target data, etc. stored in the memory 250 .

[0048] The central processing unit 270 may include one or more processors and is configured to communicate with the memory 250 via at least one communication bus to read the computer program stored in the memory 250, thereby performing operations and processing on the massive amount of data 255 in the memory 250. For example, the brain disease assessment method may be implemented by the central processing unit 270 reading the application program 253 stored in the memory 250.

[0049] In addition, the present application can also be implemented through hardware circuits or hardware circuits combined with software. Therefore, the implementation of the present application is not limited to any specific hardware circuits, software, or a combination of the two.

[0050] See also Figure 3 , the embodiment of the present application provides a method for evaluating brain diseases, which is applicable to electronic devices, for example, the electronic device may be Figure 1 The server 130 in the implementation environment is shown, and the hardware structure of the electronic device can be as follows Figure 2 shown.

[0051] In the following method embodiments, for ease of description, the execution subject of each step of the method is taken as an electronic device as an example for illustration, but this does not constitute a specific limitation.

[0052] like Figure 3 As shown, the method may include the following steps:

[0053] Step 310: Acquire multiple target data.

[0054] The target data is examination data of the target object in different modalities regarding the brain.

[0055] First of all, it should be noted that brain disease assessment requires support from multiple modalities of examination data, such as behavioral video data, imaging data (EEG, MEG, fMRI, PET, NIRS, etc.), genomic data, clinical record data, etc., which are not limited here.

[0056] It is understandable that the target data are real and sufficient in quantity, which can better reflect the brain status of the target object, thereby providing strong support for subsequent brain disease assessment.

[0057] In one possible implementation, before step 310, the following steps may be included: acquiring original data; performing denoising processing on the original data to obtain original data from which noise has been removed; and performing standardization processing on the original data from which noise has been removed to obtain the target data.

[0058] The raw data refers to the unprocessed examination data of the target object regarding different brain modalities.

[0059] Denoising refers to removing irrelevant information, errors, or interference from raw data to improve the signal-to-noise ratio. Standardization refers to unifying data formats, dimensions, and distributions.

[0060] Specifically, the original data needs to be preprocessed according to its modality. For example, image data can be denoised and standardized through image processing technology; genetic data can be quality controlled (noise reduction) and standardized through bioinformatics methods; and clinical records can be text cleaned (noise reduction) and standardized through natural language processing technology.

[0061] Through the above process, the original data is denoised and standardized to improve the quality of the target data, remove noise interference, unify the format, dimension and distribution of the target data, and ensure the key foundation for the reliability of subsequent analysis.

[0062] In step 330 , multimodal feature extraction and fusion are performed on each target data to obtain target features.

[0063] First of all, it should be noted that examination data of different modalities have completely different forms, dimensions, and semantics. Therefore, in order to better reflect the brain state of the target object, multimodal feature extraction and fusion of examination data of different modalities are required.

[0064] In one possible implementation, such as Figure 4As shown, step 330 may further include the following steps:

[0065] In step 331 , target data of different modalities are extracted respectively to obtain corresponding modal features.

[0066] It is understandable that target data of different modalities can reflect the brain state of the target subject in different forms. However, due to the different data types of target data of different modalities, feature extraction needs to be performed on them separately to obtain the corresponding modality features.

[0067] In one possible implementation, the target data includes at least one of text data, gene data, image data, and multimedia data. Through a multimodal feature extraction model, feature extraction is performed on the text data, gene data, image data, and multimedia data respectively to generate corresponding modal features.

[0068] Regarding the multimodal feature extraction model, it can refer to designing a special model (such as CNN processing images and LMM processing text) for target data of different modalities (such as text data, genetic data, image data, multimedia data).

[0069] Specifically, multimedia data can use a multimodal feature extraction model to extract modal features of images or videos; imaging data can use a multimodal feature extraction model to extract detailed information to obtain corresponding modal features; genetic data can use a multimodal feature extraction model to extract sequence features to obtain corresponding modal features; text data, such as clinical records, can use a multimodal feature extraction model to extract text features to obtain corresponding modal features.

[0070] In step 333 , the features of each modality are fused according to a multimodal fusion method to obtain target features with unified expression.

[0071] First, multimodal fusion methods involve fusing features from various modalities to generate a unified target feature, such as a textual or graphical representation. It's understandable that fusing features from various modalities into a unified target feature facilitates subsequent analysis and processing.

[0072] It is understandable that a single modality may not be able to fully reflect the condition (for example, the imaging features of early dementia are not obvious, but behavioral videos have shown abnormalities). Unified expression can integrate complementary information.

[0073] In one possible implementation, the attention mechanism is used to fuse the features of each modality to generate target features in text form; or, the multimodal graph fusion model is used to fuse the features of each modality to generate target features in graph structure.

[0074] Regarding target features in text form, the attention mechanism is used to dynamically assign weights and highlight important modal features (such as giving higher weights to MRI image features). Different modal features can be weighted and fused to generate target features in a unified text form.

[0075] Regarding target features in the form of graph structures, multimodal graph fusion models (e.g., graph neural networks (GNNs)) can be used to fuse features from different modalities. Specifically, a graph structure can be constructed, with nodes representing modal features and edges representing inter-modal associations (e.g., pathological associations between genes and imaging). This generates target features with a unified graph structure to capture the complex relationships between different modalities. Graph structures can enhance feature expression capabilities and capture potential cross-modal associations.

[0076] Through this process, target data from multiple modalities—including multimedia, imaging, genetic, and clinical records—is effectively integrated to comprehensively capture the patient's condition and improve diagnostic accuracy. Multimodal fusion leverages the complementarity between target data from different modalities, improving evaluation performance and mitigating the impact of low-quality and erroneous data from a single modality.

[0077] Step 350 , in the process of generating the evaluation result according to the target feature, a retrieval mechanism based on the brain disease knowledge base is introduced to perform enhanced retrieval on the target feature to obtain the evaluation result of the brain disease.

[0078] The assessment results include the target subject's brain disease status and treatment recommendations. The target subject can be a patient at risk for a brain disease. A brain disease knowledge base can be established using medical books and literature. This knowledge base can include a large amount of medical literature, case reports, and treatment guidelines, including information on brain disease diagnostic criteria, brain treatment plans, and brain pathological mechanisms.

[0079] The retrieval mechanism can be either a retrieval enhancement mechanism or a graph-based retrieval enhancement generation mechanism, which is not limited here. It should be noted that the type of retrieval enhancement mechanism used is related to the structure of the target data and the brain disease knowledge base. For example, if the target data and the expertise stored in the brain disease knowledge base are graph-structured, then a graph-based retrieval enhancement generation mechanism can be used to enhance the retrieval of target features.

[0080] Regarding the retrieval enhancement mechanism, it can be the retrieval enhancement generation (RAG) technology. The retrieval enhancement generation (RAG) technology includes three steps: retrieval, enhancement and generation. The retrieval stage can retrieve relevant documents or paragraphs from the brain disease knowledge base based on the input target data; the enhancement stage combines the retrieved information with the original input to form a richer context; the generation stage uses this enhanced context to produce the final output evaluation result.

[0081] Regarding the graph-based retrieval enhancement generation mechanism, it can refer to the introduction of target features in the form of graph structure in the process of generating evaluation results to improve the accuracy and interpretability of the generated evaluation results.

[0082] Through the above process, by building a brain disease knowledge base, we can provide more professional and authoritative brain disease diagnosis and treatment recommendations, continuously update and adapt to new medical knowledge and clinical practices, and maintain the timeliness and accuracy of diagnosis and treatment recommendations. The brain disease knowledge base can be continuously updated, avoiding the high cost of model training and fine-tuning.

[0083] See also Figure 5 In an exemplary embodiment, before step 350, the method may further include the following steps:

[0084] Step 410: Obtain medical knowledge data about brain diseases.

[0085] Among them, medical knowledge data includes medical knowledge information about brain diseases such as medical books, medical literature, case reports and treatment guidelines.

[0086] Step 430: Use natural language processing methods to perform text vectorization processing on the medical knowledge data to obtain a medical knowledge vector.

[0087] It is understandable that medical knowledge data (such as literature, case reports, and guidelines) are mostly unstructured texts containing complex terms, abbreviations, and long-tail expressions.

[0088] For example, "Donepezil," "Donepezil," and "Aricept" (trade name) all have the same meaning. Another example is "hippocampal atrophy is associated with Aβ deposition," which requires semantic understanding rather than simple keyword matching.

[0089] Then, natural language processing methods can be used to unify the terminology of medical knowledge data and to perform semantic understanding of medical knowledge data.

[0090] In one possible implementation, an embedding model can be used to convert the text information corresponding to the medical knowledge data into a medical knowledge vector representation.

[0091] Step 450: Establish a knowledge base based on the medical knowledge vectors to generate a brain disease knowledge base.

[0092] It should be noted that it is also possible to build a graph-based medical knowledge base, converting information such as medical knowledge vectors into a graph structure to generate a brain disease knowledge base to support subsequent more complex reasoning and queries.

[0093] Step 470 , detecting whether the brain disease knowledge base needs to be updated. If yes, returning to the step of obtaining medical knowledge data about brain diseases.

[0094] Regarding whether the knowledge base for detecting brain diseases needs to be updated, it can be updated regularly or triggered by conditions, such as the publication of new medical data, etc., which is not limited here.

[0095] It should be noted that the timeliness and accuracy of the knowledge base can be ensured by combining automated crawlers and manual review.

[0096] See also Figure 6 In an exemplary embodiment, step 350 may include the following steps:

[0097] Step 351 , according to the retrieval enhancement generation method, knowledge retrieval is performed in the brain disease atlas knowledge base using the target feature to obtain target knowledge matching the target feature.

[0098] Regarding the target knowledge matched with the target feature, the cosine similarity between each relevant knowledge in the brain disease atlas knowledge base and the target feature can be calculated. If the cosine similarity indicates that the two are similar (for example, more than 90%), then the match is considered successful.

[0099] Among them, target knowledge refers to the relevant professional knowledge retrieved from the knowledge base based on target features.

[0100] Step 353: Use the large language model to understand the target knowledge and generate an evaluation result.

[0101] Regarding understanding, it means converting target knowledge into machine-operable semantic representations through the semantic modeling, reasoning, and generation capabilities of large language models, and completing knowledge association, logical inference, and decision support based on this.

[0102] In one possible implementation, a retrieval module can utilize retrieval-enhanced generation techniques (such as LightRAG) to search the target features against a brain disease knowledge base to obtain relevant information. Specifically, vector retrieval methods, such as cosine similarity-based retrieval, can be used to quickly locate the target knowledge. The generation module can input the target knowledge into a large language model to generate corresponding evaluation results. Specifically, relevant expertise retrieved from the knowledge base using multimodal feature text is used as input to the large language model to ensure the accuracy and professionalism of the generated content.

[0103] Through the above process, by building a local brain disease vector knowledge base and combining it with a large language model, we can provide more professional and authoritative brain disease diagnosis and treatment recommendations. RAG technology enhances the model's knowledge acquisition ability and generation accuracy by providing the latest information to the LLM.

[0104] Combined with the above examples, automated multimodal data processing and feature fusion can reduce manual intervention and improve diagnostic efficiency. This method can simulate a multidisciplinary consultation model, fully leveraging the complementary nature of various medical data, improving diagnostic comprehensiveness and decision-making interpretability, and providing more accurate, professional, and efficient brain disease diagnosis and treatment recommendations.

[0105] It should be noted that the brain disease assessment method can also be integrated into practical application systems, such as medical diagnosis support systems, intelligent medical assistants, etc., and a friendly user interface can be designed to facilitate use by doctors and patients.

[0106] Specifically, brain disease assessment applications may include:

[0107] The feature processing module is used to obtain multiple target data, perform multimodal feature extraction and fusion on each target data, and obtain target features.

[0108] Retrieval module: Using retrieval-enhanced generation technology (such as LightRAG), the fused target features are searched against the local brain disease knowledge base to obtain relevant information. Vector search methods, such as cosine similarity-based search, are used to quickly locate target knowledge.

[0109] Generation Module: Combined with the large language model, it generates corresponding evaluation results. The target knowledge retrieved from the brain disease knowledge base based on the target feature is used as the input of the large language model to ensure the accuracy and professionalism of the generated content.

[0110] Regarding the system architecture for brain disease assessment applications: Each module and the brain disease knowledge base will be integrated into practical application systems, such as medical diagnosis support systems and intelligent medical assistants. The system architecture adopts a microservice design to ensure the independence and maintainability of each module.

[0111] Regarding the user interface, a user-friendly interface can be designed to facilitate use by doctors and patients. The interface supports multiple input methods, such as voice, text, and images, to meet the needs of different users.

[0112] Regarding system deployment: Deploy the system in the cloud or on a local server to ensure data security and privacy protection. Use encryption technology and access control to prevent unauthorized access.

[0113] Through the above process, brain disease assessment methods can be integrated into practical application systems, such as medical diagnosis support systems and intelligent medical assistants. By integrating with clinical workflows, they can provide real-time brain disease diagnosis and treatment recommendations, assist doctors in decision-making, and improve the efficiency and quality of medical services.

[0114] Figure 7This is a schematic diagram of a specific implementation of a brain disease assessment method in an application scenario, namely, epilepsy assessment.

[0115] Now combined Figure 7 Explain the epilepsy assessment scenario:

[0116] The process for this application scenario begins with data preprocessing. This application scenario uses the publicly available Qwen2.5-VL-7B-Instruct model to process target patient data and implement feature extraction and fusion. The target data can include epilepsy treatment information and patient test data. The model efficiently extracts text from scanned images while also enabling image understanding. Each page of a PDF of epilepsy treatment information consists of text, images, and charts. This step involves using hint engineering to extract text from images, converting visually comprehensible information like images and charts into textual data suitable for subsequent analysis, thereby obtaining the target data.

[0117] Next, LightRAG technology is applied, and the knowledge base generated by combining epilepsy treatment information is combined, which enables the system to accurately retrieve and generate relevant content based on the target data. In the embedding part, the system uses the public BAAI / bge-m3 model to ensure the generation of high-quality vector representations, which is critical for the model to understand and generate responses. Finally, the public Qwen2.5-72B-Instruct-AWQ, a large language model (the quantized model has a small number of parameters but powerful performance), is used to generate accurate and context-relevant responses. This model ensures that the output evaluation results are highly relevant and complex, and are suitable for the advanced needs of medical or technical tasks.

[0118] The epilepsy auxiliary classification and treatment recommendation knowledge graph constructed is as follows Figure 8 shown.

[0119] Figure 9 Comparison of adjunctive treatment recommendations. StarLab, a brain disease assessment method implemented for this application scenario, demonstrates that its adjunctive treatment recommendations for brain diseases are more accurate. Compared to GPT-o3-mini-high, StarLab's assessment results include first-line treatments, alternative treatments, contraindicated medications, and precautions, providing more refined treatment recommendations.

[0120] It should be noted that this solution is not only applicable to the diagnosis and treatment recommendations of brain diseases, but can also be extended to other medical fields, such as: Cancer diagnosis and treatment: integrating imaging data, genomic data and clinical records to provide personalized cancer diagnosis and treatment plans. Cardiovascular disease management: combining electrocardiograms, ultrasound images and patient medical history to assist in early screening and risk assessment of cardiovascular diseases. Genetic disease prediction: using genomic data and family medical history to predict the risk of genetic diseases and provide prevention and intervention recommendations. Mental illness assessment: integrating behavioral videos, psychological assessments and clinical records to assist in the diagnosis of mental illness and the formulation of treatment plans.

[0121] In this application scenario, fragmented medical knowledge is transformed into dynamic decision support capabilities through a closed loop of multimodality → retrieval → generation → graph, providing reliable tools for the individualized treatment of complex diseases such as epilepsy.

[0122] It should be understood that although the steps in the flowcharts of the accompanying drawings are shown in sequence as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some of the steps in the flowcharts of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a portion of the sub-steps or stages of other steps.

[0123] The following are embodiments of the apparatus of the present application, which can be used to perform the brain disease assessment method involved in the present application. For details not disclosed in the apparatus embodiments of the present application, please refer to the method embodiments of the brain disease assessment method involved in the present application.

[0124] See also Figure 10 In an embodiment of the present application, a brain disease assessment device 900 is provided, including but not limited to: a data acquisition module 910, a feature processing module 930, and a disease assessment module 950.

[0125] The data acquisition module 910 is used to acquire a plurality of target data; the target data is the examination data of the target object in different modalities regarding the brain;

[0126] The feature processing module 930 is used to extract and fuse multimodal features of each target data to obtain target features;

[0127] The disease assessment module 950 is used to introduce a retrieval mechanism based on a brain disease knowledge base to perform enhanced retrieval on the target features in the process of generating an assessment result based on the target features, so as to obtain the assessment result of the brain disease; the assessment result includes the brain disease status of the target object and treatment recommendations.

[0128] In an exemplary embodiment, the feature processing module 930 is further used to extract the target data of different modalities respectively to obtain corresponding modal features; and perform feature fusion on each of the modal features according to a multimodal fusion method to obtain the target features with unified expression.

[0129] In an exemplary embodiment, the target data includes at least one of text data, gene data, image data, and multimedia data;

[0130] The feature processing module 930 is further configured to perform feature extraction on the text data, the gene data, the image data, and the multimedia data respectively through a multimodal feature extraction model to generate corresponding modal features.

[0131] In an exemplary embodiment, the feature processing module 930 is further used to use the attention mechanism to fuse the modal features to generate target features in text form; or, use the multimodal graph fusion model to fuse the modal features to generate target features in graph structure.

[0132] In an exemplary embodiment, the device also includes: a knowledge base construction module; the knowledge base construction module is also used to obtain medical knowledge data about brain diseases; use natural language processing methods to perform text vectorization on the medical knowledge data to obtain medical knowledge vectors; establish a knowledge base based on the medical knowledge vectors to generate the brain disease knowledge base; detect whether the brain disease knowledge base needs to be updated, and if so, return to the step of obtaining medical knowledge data about brain diseases.

[0133] In an exemplary embodiment, the disease assessment module 950 is further used to perform knowledge retrieval in the brain disease atlas knowledge base using the target features according to a graph-based retrieval enhancement generation method to obtain target knowledge that matches the target features; and use a large language model to understand the target knowledge and generate an assessment result.

[0134] In an exemplary embodiment, the data acquisition module 910 is further configured to acquire original data; perform denoising processing on the original data to obtain original data from which noise has been removed; and perform standardization processing on the original data from which noise has been removed to obtain the target data.

[0135] It should be noted that the brain disease assessment device provided in the above embodiment only uses the division of the above-mentioned functional modules as an example when performing brain disease assessment. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the brain disease assessment device will be divided into different functional modules to complete all or part of the functions described above.

[0136] In addition, the brain disease assessment device and the brain disease assessment method provided in the above embodiments belong to the same concept, and the specific manner in which each module performs operations has been described in detail in the method embodiments and will not be repeated here.

[0137] See also Figure 11 In an embodiment of the present application, an electronic device 4000 is provided. The electronic device 4000 may include: a desktop computer, a laptop computer, a server, etc.

[0138] exist Figure 11 In the embodiment, the electronic device 4000 includes at least one processor 4001 and at least one memory 4003.

[0139] Data exchange between the processor 4001 and the memory 4003 can be achieved via at least one communication bus 4002. The communication bus 4002 may include a path for transmitting data between the processor 4001 and the memory 4003. The communication bus 4002 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, for example. The communication bus 4002 may be divided into an address bus, a data bus, a control bus, and the like. For ease of illustration, only one thick line is used in the figure, but this does not mean that there is only one bus or only one type of bus.

[0140] Optionally, the electronic device 4000 may further include a transceiver 4004, which may be used for data exchange between the electronic device and other electronic devices, such as data transmission and / or data reception. It should be noted that in actual applications, the number of transceivers 4004 is not limited to one, and the structure of the electronic device 4000 does not constitute a limitation on the embodiments of the present application.

[0141] Processor 4001 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It may implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. Processor 4001 may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, and the like.

[0142] The memory 4003 can be a ROM (Read Only Memory) or other types of static storage devices that can store static information and instructions, a RAM (Random Access Memory) or other types of dynamic storage devices that can store information and instructions, or an EEPROM (Electrically Erasable Programmable Read Only Memory), a CD-ROM (Compact Disc Read Only Memory) or other optical disk storage, optical disk storage (including compact disc, laser disc, optical disc, digital versatile disc, Blu-ray disc, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store a computer program in the form of instructions or data structures and can be accessed by the electronic device 400, but is not limited to these.

[0143] The memory 4003 stores a computer program, and the processor 4001 can read the computer program stored in the memory 4003 through the communication bus 4002 .

[0144] The computer program is executed by one or more processors 4001 to implement the brain disease assessment method in the above embodiments.

[0145] In addition, an embodiment of the present application provides a storage medium on which a computer program is stored. The computer program is executed by one or more processors to implement the brain disease assessment method as described above.

[0146] A computer program product is provided in an embodiment of the present application, including a computer program, which is executed by one or more processors to implement the brain disease assessment method as described above.

[0147] Compared with related technologies, this solution reduces manual intervention and improves diagnostic efficiency through automated multimodal data processing and feature fusion. This method can simulate a multidisciplinary consultation model, fully leveraging the complementary nature of various medical data types to enhance diagnostic comprehensiveness and interpretable decision-making. It provides more accurate, professional, and efficient brain disease diagnosis and treatment recommendations, effectively addressing the low accuracy of brain disease assessments found in related technologies.

[0148] The above description is only part of the implementation methods of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A method for evaluating brain diseases, characterized in that: include: Get multiple target data; The target data is examination data of the target object in different modalities related to the brain; Performing multimodal feature extraction and fusion on each of the target data to obtain target features; In the process of generating the evaluation results based on the target features, a retrieval mechanism based on the brain disease knowledge base is introduced to perform enhanced retrieval on the target features to obtain the evaluation results of the brain disease; the evaluation results include the brain disease status of the target object and treatment recommendations.

2. The method according to claim 1, wherein The performing multimodal feature extraction on the target data to obtain target features includes: Extracting the target data of different modalities respectively to obtain corresponding modal features; The modal features are fused according to a multimodal fusion method to obtain the target features with unified expression.

3. The method according to claim 2, wherein The target data includes at least one of text data, gene data, image data and multimedia data; The target data of different modalities are extracted respectively to obtain corresponding modal features, including: Through a multimodal feature extraction model, feature extraction is performed on the text data, the gene data, the image data and the multimedia data respectively to generate corresponding modal features.

4. The method according to claim 2, wherein The step of fusing the modal features according to the multimodal fusion method to obtain the target feature includes: Using the attention mechanism to fuse the features of each modality to generate target features in text form; or, The multimodal graph fusion model is used to fuse the modal features to generate target features of the graph structure.

5. The method according to claim 1, wherein Before searching the brain disease knowledge base using the target feature, the method further includes: Access medical knowledge data on brain diseases; Performing text vectorization processing on the medical knowledge data using a natural language processing method to obtain a medical knowledge vector; Establishing a knowledge base based on the medical knowledge vector to generate the brain disease knowledge base; Detect whether the brain disease knowledge base needs to be updated. If yes, return to the step of obtaining medical knowledge data about brain diseases.

6. The method according to claim 1, wherein In the process of generating the evaluation result according to the target feature, a retrieval mechanism based on a brain disease knowledge base is introduced to perform enhanced retrieval on the target feature to obtain the evaluation result of the brain disease, including: According to a graph-based retrieval enhancement generation method, the target feature is used to perform knowledge retrieval in the brain disease atlas knowledge base to obtain target knowledge matching the target feature; The target knowledge is understood using a large language model to generate an evaluation result.

7. The method according to any one of claims 1 to 6, wherein: The acquiring of a plurality of target data includes: Get the original data; Performing denoising on the original data to obtain original data from which noise has been removed; The original data from which the noise is removed is normalized to obtain the target data.

8. A brain disease assessment device, characterized in that: include: A data acquisition module is used to acquire multiple target data; The target data is examination data of the target object in different modalities related to the brain; A feature processing module is used to extract and fuse multimodal features of each target data to obtain target features; The disease assessment module is used to introduce a retrieval mechanism based on a brain disease knowledge base to perform enhanced retrieval on the target features in the process of generating an assessment result based on the target features, so as to obtain the assessment result of the brain disease; the assessment result includes the brain disease status of the target subject and treatment recommendations.

9. An electronic device comprising at least one processor and at least one memory, wherein: The memory stores a computer program, wherein when the computer program is executed by the processor, the brain disease assessment method according to any one of claims 1 to 7 is implemented.

10. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by one or more processors, the brain disease assessment method according to any one of claims 1 to 7 is implemented.

Images