A multi-granularity electroencephalogram-text alignment and clinical reasoning method based on expert prior guidance

Abstract

This application relates to a multi-granularity EEG-text alignment and clinical reasoning method based on expert prior guidance. The method includes: first, acquiring multimodal EEG-text raw data; then, mapping unstructured reports into structured triples using a structured clinical expert ontology; next, extracting EEG spatiotemporal frequency features and text semantic features using a dual-stream coding architecture; constructing a dimension-matched expert mask matrix by combining anatomical priors; injecting this mask matrix into a cross-attention mechanism to obtain fine-grained alignment features; subsequently, mapping these features to the semantic embedding space of a generative large language model using a cross-modal projection matrix and constructing global prompts; finally, fusing the global prompts with EEG local abnormality evidence into the large model, and generating a natural language diagnostic report through clinical logical reasoning. This method can improve the interpretability and clinical compliance of assisted diagnosis, effectively solving the technical problems of the contradiction between accuracy and generalization, and the separation of signal and semantics in the field of EEG analysis.

Images