Graph-driven intelligent medical image retrieval method and system

Abstract

The present application discloses a graph-driven intelligent medical image retrieval method and system, which relates to the field of image retrieval. It first encodes the text query input by the user by constructing a medical knowledge graph and generates a graph-driven enhanced embedding vector containing advanced medical concepts and a query-related graph concept set. Subsequently, this enhanced vector is used to perform a preliminary vector similarity search to quickly screen out the Top‑K preliminary candidate images. Finally, based on the query-related graph concept set, these preliminary candidate images are secondary sorted by graph concepts, and the deep medical semantics of the image are accurately matched by calculating the concept overlap. This two-stage retrieval and graph deep fusion mechanism ensures that the retrieval results are not only visually similar, but also highly relevant in medical concepts, thereby significantly improving the accuracy and intelligence level of medical image retrieval.

Description

Technical Field

[0001] The present application relates to the field of image retrieval, and more specifically, to a graph-driven intelligent medical image retrieval method and system. Background Art

[0002] In the healthcare sector, with the rapid development of medical imaging technology, various imaging data types, such as CT, MRI, and X-rays, have exploded in volume, providing a rich source of information for clinical diagnosis and scientific research. However, efficiently and accurately retrieving images highly relevant to specific clinical needs from this massive amount of image data has become a major challenge. This is primarily due to the prevalent semantic gap in medical image retrieval. Specifically, the underlying visual features of medical images, such as pixel intensity, texture, and shape, are low-level information that computers can easily extract; however, doctors (users) rely on higher-level medical concepts such as nodules, infiltrates, space-occupying lesions, and increased lung markings when making diagnoses. Simple visual feature similarity often cannot equate to true similarity in medical diagnosis or pathology. This makes it difficult for traditional content-based image retrieval systems to accurately understand the user's underlying medical intent. The resulting semantic gap between retrieval results and actual clinical needs makes truly intelligent retrieval difficult to achieve.

[0003] Some existing medical image retrieval methods focus on using deep learning techniques to extract multimodal features from images for vector matching, or attempt to search based on text keywords in imaging reports. While these methods have improved retrieval efficiency to some extent, they generally fail to effectively bridge the aforementioned semantic gap. For example, retrieval based on visual features is prone to encountering different images for the same condition or the same image for different conditions when faced with image variations across different patients, different equipment, or different scanning parameters, resulting in insufficient medical relevance of the retrieval results. Furthermore, retrieval based on text keywords, limited by the ambiguity of natural language and the complexity of medical terminology, struggles to capture the deep semantic connections and logical relationships between medical concepts. Even though some systems attempt to incorporate some structured knowledge, if this knowledge is not deeply integrated into the retrieval process and truly drives the process, the system may degenerate into traditional low-level feature matching, failing to accurately understand the user query and express the medical conceptualization of the imaging content, thus limiting its practical application in auxiliary diagnosis, teaching, and scientific research.

[0004] In view of the above challenges, there is an urgent need for a more intelligent and accurate medical image retrieval method that can deeply understand user intent and bridge the semantic gap. Summary of the Invention

[0005] In view of the above-mentioned limitations of the existing methods, according to one aspect of the present application, a graph-driven intelligent medical image retrieval method is provided, which includes: constructing a medical knowledge graph; obtaining a user text query; based on the medical knowledge graph, performing semantic enhancement embedding encoding on the user text query to obtain a user text query graph-driven enhanced embedding vector and a query-related graph concept set; inputting the user text query graph-driven enhanced embedding vector into a vector retrieval engine, and the vector retrieval engine extracting the Top-K images most similar to the user text query graph-driven enhanced embedding vector from a semantic index library as a preliminary candidate list; based on the query-related graph concept set, performing graph concept secondary sorting on the preliminary candidate list to obtain a sorted list of related medical images; and displaying the sorted list of related medical images.

[0006] According to another aspect of the present application, a graph-driven intelligent medical image retrieval system is provided, which includes: a medical knowledge graph construction module for constructing a medical knowledge graph; a user text query acquisition module for acquiring a user text query; a user text query encoding module for performing semantic enhancement embedding encoding on the user text query based on the medical knowledge graph to obtain a user text query graph-driven enhanced embedding vector and a query-related graph concept set; a preliminary candidate list generation module for inputting the user text query graph-driven enhanced embedding vector into a vector retrieval engine, and the vector retrieval engine extracts the Top-K images most similar to the user text query graph-driven enhanced embedding vector from a semantic index library as a preliminary candidate list; a preliminary candidate list sorting module for performing graph concept secondary sorting on the preliminary candidate list based on the query-related graph concept set to obtain a sorted related medical image list; and a related medical image list display module for displaying the sorted related medical image list.

[0007] Compared with the existing technology, the present application provides a graph-driven intelligent medical image retrieval method and system, which first performs deep semantic enhancement embedding encoding on the text query input by the user by constructing a medical knowledge graph. This not only identifies and links the medical terms in the query to the graph entity, but also generates a graph-driven enhanced embedding vector containing advanced medical concepts and a query-related graph concept set. This directly bridges the gap between text queries and medical concepts. Subsequently, this enhanced vector is used to perform a preliminary vector similarity search to quickly screen out the Top-K preliminary candidate images. Finally, based on the query-related graph concept set, these preliminary candidate images are secondary sorted by graph concepts, and the deep medical semantics of the image are accurately matched by calculating the concept overlap. This two-stage retrieval and graph deep fusion mechanism ensures that the retrieval results are not only visually similar, but also highly relevant in medical concepts, thereby significantly improving the accuracy and intelligence level of medical image retrieval. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] The above and other purposes, features, and advantages of the present application will become more apparent through a more detailed description of the embodiments of the present application in conjunction with the accompanying drawings. The accompanying drawings are intended to provide a further understanding of the embodiments of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the present application and do not constitute a limitation of the present application. In the drawings, the same reference numerals generally represent the same components or steps.

[0009] Figure 1 This is a flowchart of a graph-driven intelligent medical image retrieval method according to an embodiment of the present application.

[0010] Figure 2 Schematic diagram of data flow of the graph-driven intelligent medical image retrieval method according to an embodiment of the present application.

[0011] Figure 3 This is a flowchart of step S3 in the graph-driven intelligent medical image retrieval method according to an embodiment of the present application.

[0012] Figure 4 This is a flowchart of step S4 in the graph-driven intelligent medical image retrieval method according to an embodiment of the present application.

[0013] Figure 5 This is a block diagram of a graph-driven intelligent medical image retrieval system according to an embodiment of the present application. DETAILED DESCRIPTION

[0014] The following describes embodiments of the present disclosure in more detail with reference to the accompanying drawings. While the drawings illustrate certain embodiments of the present disclosure, it should be understood that the present disclosure can be implemented in various forms and should not be construed as limited to the embodiments described herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are for illustrative purposes only and are not intended to limit the scope of protection of the present disclosure.

[0015] In response to the problems in the above-mentioned background technology, this application proposes a graph-driven intelligent medical image retrieval method. Figure 1 This is a flowchart of a graph-driven intelligent medical image retrieval method according to an embodiment of the present application. Figure 2 Schematic diagram of data flow of the graph-driven intelligent medical image retrieval method according to the embodiment of the present application. Figure 1 and Figure 2As shown, the graph-driven intelligent medical image retrieval method according to the embodiment of the present application includes: S1, constructing a medical knowledge graph; S2, obtaining a user text query; S3, based on the medical knowledge graph, performing semantic enhancement embedding encoding on the user text query to obtain a user text query graph-driven enhanced embedding vector and a query-related graph concept set; S4, inputting the user text query graph-driven enhanced embedding vector into a vector retrieval engine, and the vector retrieval engine extracts the Top-K images most similar to the user text query graph-driven enhanced embedding vector from a semantic index library as a preliminary candidate list; S5, based on the query-related graph concept set, performing graph concept secondary sorting on the preliminary candidate list to obtain a sorted related medical image list; S6, displaying the sorted related medical image list.

[0016] In step S1, a medical knowledge graph is constructed. It should be understood that the core challenge facing traditional methods in the field of medical image retrieval is the semantic gap: the significant discrepancy between the low-level visual features recognized by computers and the high-level medical concepts understood by physicians. Relying solely on features such as pixels and textures for matching, or searching solely through keywords, fails to accurately capture the underlying medical intent of user queries. This leads to a mismatch between retrieval results and actual clinical needs, preventing true intelligence from being achieved. To bridge this gap and empower the retrieval process with the power of medical expertise, constructing a medical knowledge graph is essential. Notably, a medical knowledge graph organizes and represents medical entities and their interrelationships in a structured form, integrating dispersed medical knowledge into a rigorous knowledge network. By pre-constructing this graph, subsequent retrieval processes can deeply understand and enhance user queries based on rich medical semantic information. This enables a shift from surface-level feature matching to deeper conceptual associations, ensuring the medical relevance and accuracy of retrieval results. This is the cornerstone of intelligent medical image retrieval.

[0017] In particular, in one achievable method of the present application, step S1 is processed as follows: first, a variety of original medical data are collected, including but not limited to authoritative medical textbooks, clinical diagnosis and treatment guidelines, electronic medical record texts, medical literature abstracts, and existing medical ontologies, such as the International Classification of Diseases (ICD) coding system, medical subject headings, and systematically named medical coding systems.

[0018] Next, information extraction is performed, a core step in knowledge graph construction. It primarily involves medical named entity recognition (MNE) and relation extraction. For MNE recognition, a deep learning-based sequence labeling model is employed, such as a fine-tuned model based on a pretrained language model (such as BERT or RoBERTa). This model takes a text sequence as input and, through a multi-layer self-attention mechanism and a feedforward network, generates context-sensitive embedding representations for each token. These embeddings are then fed into a classification head—a linear layer—to predict the entity label for each token, such as B-disease, I-disease, or O. To ensure the validity of the label sequence, a conditional random field (CRF) layer is added after the classification head. This layer learns the transition probabilities between labels, thereby optimizing the overall label sequence. The model's weights and bias parameters are trained through supervised learning on a large corpus of annotated medical text. For example, given the input text of a patient presenting with lung nodules and cough, the model will identify and label lung nodules as the disease entity and cough as the symptom entity.

[0019] Based on entity recognition, relationship extraction aims to identify semantic relationships between identified entities. This is done using a deep learning-based model, which can also be based on a pre-trained language model. The model takes a sentence containing identified entities as input and obtains the sentence's contextual embedding using a pre-trained language model. Subsequently, a representation of the entity pair is extracted through specific pooling operations, such as the embedding of the entity's initial token or average pooling of all entity token embeddings, and concatenated with the overall sentence representation. This concatenated vector is fed into a multi-layer perceptron (MLP) classifier, which uses a softmax activation function to output the possible relationship types between the entity pairs, such as the probability distribution of "cause," "locate," and "treat." The model's weights and bias parameters are also trained through supervised learning on medical corpus containing annotated entity relationships. For example, given the sentence "pulmonary nodules cause cough," the model will identify a cause-cause relationship between pulmonary nodules and cough.

[0020] Extracted entities and relationships are initially stored as triples (entity 1, relationship, entity 2). To ensure knowledge accuracy and consistency, knowledge fusion and alignment are required. This involves merging synonymous entities from different data sources, resolving entity ambiguities, and adjudicating conflicting information. For example, if "lung mass" and "lung lesion" refer to the same concept in different sources, they are mapped to a unified entity identifier. Ultimately, these structured medical entities and relationships are stored in a graph database, forming a medical knowledge graph.

[0021] In step S2, the user text query is obtained. It is understandable that when faced with massive amounts of medical imaging data, users often use natural language to describe the medical concepts, symptoms or imaging features of their interest, such as searching for CT images with ground-glass shadows in the lungs or MRI images showing space-occupying lesions in the liver. This text query is a direct reflection of the user's intention and a bridge connecting the user to the underlying imaging data. Although this original text query may have semantic ambiguity or non-standard expression problems, it is the basis for subsequent semantic enhancement and precise matching through the medical knowledge graph, providing the initial unstructured input for bridging the semantic gap, so that the retrieval process can start from the user's actual needs, rather than relying solely on preset labels or low-level visual features.

[0022] In particular, in one implementation of the present application, step S2 is processed as follows: the process is implemented by providing an intuitive and easy-to-use text input interface. The user can type or speak their medical image retrieval requirements in a specific area of ​​the retrieval interface, such as a search box or query bar, through various methods such as keyboard input and voice recognition text conversion. When the user completes the input in the text input box and triggers the query operation, for example, by clicking the search button or pressing the enter key, the method instantly captures the complete text content entered by the user. The captured text content is a raw string, such as lung nodules with chest pain. After capture, some basic text verification is performed, such as checking whether the character encoding is correct and whether the text length is within a preset range, for example, limiting the query length to between 5 and 200 characters to avoid invalid queries that are too long or too short, and removing potential malicious input or special characters. These preliminary verifications are intended to ensure the quality and security of the text data subsequently processed. After verification, the raw text string is output as the user's text query.

[0023] In step S3, based on the medical knowledge graph, the user text query is semantically enhanced and embedded to obtain a user text query graph-driven enhanced embedding vector and a query-related graph concept set. It should be understood that the original text query input by the user is often in the form of natural language, which may be ambiguous, non-standardized, or lack deep medical semantics. Simple text matching or retrieval based on low-level features cannot accurately capture the high-level medical concepts and intentions behind the user query. To this end, in this application, by introducing a medical knowledge graph, unstructured text queries can be converted into structured representations containing medical semantics, namely, graph-driven enhanced embedding vectors and query-related graph concept sets. This enables the system to go beyond literal matching, truly understand the medical meaning of the query, and utilize the rich conceptual relationships in the knowledge graph to enhance the expressive power of the query, thereby laying the foundation for subsequent precise retrieval and ensuring the high relevance of the retrieval results at the medical level.

[0024] In particular, in one possible implementation of the present application, Figure 3 FIG is a flow chart of step S3 in the graph-driven intelligent medical image retrieval method according to an embodiment of the present application. Figure 3 As shown, step S3, based on the medical knowledge graph, performs semantic enhancement embedding encoding on the user text query to obtain a user text query graph-driven enhanced embedding vector and a query-related graph concept set, including: S31, performing text preprocessing on the user text query and inputting it into a text editor to obtain a user text query semantic feature vector; S32, using a medical named entity recognition tool to identify medical terms from the user text query, and linking the medical terms to the corresponding entity IDs in the medical knowledge graph to obtain the query-related graph concept set; S33, inputting the query-related graph concept set and the user text query semantic feature vector into a graph-driven text attention fusion module to obtain the user text query graph-driven enhanced embedding vector.

[0025] It's worth noting that the original text queries entered by users are in natural language, containing noise, redundant information, and unstructured features that make them difficult for computers to directly understand and process. Text preprocessing is essential to converting this human-readable text into a machine-understandable and computable numerical representation, namely a semantic feature vector. It cleans and normalizes the text, removes irrelevant characters, and unifies the expression, thereby improving the accuracy and efficiency of subsequent semantic encoding. The role of a text editor is to map the preprocessed text into a high-dimensional vector space, so that semantically similar queries are close in distance in the vector space. This lays the foundation for subsequent vector-based similarity calculations and semantic matching, effectively bridging the semantic gap between natural language and machine understanding.

[0026] In particular, in one implementation of the present application, step S31 is processed as follows: First, the user text query is preprocessed, which includes text cleaning and normalization, such as removing special symbols and redundant spaces in the text, converting full-width characters into half-width characters, and unifying the punctuation format. Subsequently, word segmentation is performed. For Chinese text, it is necessary to segment the continuous sequence of Chinese characters into word units with independent semantics. For example, a word segmentation tool based on a dictionary and a statistical model is used to segment a CT image with ground-glass shadows in the lungs into lungs, have, ground-glass shadows, of, CT, image. Finally, according to the preset stop word list, common words in the text that do not contribute much to the semantics, such as of, is, and so on, are removed to reduce noise and highlight the core semantics.

[0027] A preprocessed text sequence, such as a ground-glass opacity CT image of the lung, is input into a text editor—a pretrained text encoding model—to generate a semantic feature vector. A pretrained language model based on the Transformer architecture can be used, such as the BERT model, which has been pretrained for Chinese medical applications. The specific architecture of this text encoding model consists of an input layer, a multi-layer Transformer encoder, and an output layer. The input layer converts the preprocessed word sequence into a numerical form that the model can process: each word is mapped to a corresponding word embedding vector; a positional embedding vector is added to capture the word's position in the sequence. The sum of these embedding vectors constitutes the model's input representation. This input representation is then fed into a multi-layer Transformer encoder. Each encoder layer consists of a multi-head self-attention mechanism and a feedforward neural network module. The multi-head self-attention mechanism allows the model to simultaneously attend to information at different positions in the input sequence and calculates the strength of each word's association with other words, thereby capturing the word's contextual semantics. The feedforward neural network performs nonlinear transformations on the output of the self-attention mechanism. By stacking these layers, the model can learn complex semantic patterns and contextual dependencies in the text.

[0028] After the model processes the entire sequence, it extracts the final hidden state corresponding to special tokens, such as the [CLS] token, as the semantic feature vector for the entire user text query. This vector is a high-dimensional, dense vector that encodes the overall semantic information of the query. Specifically, the model's weights and bias parameters are pre-trained on a large-scale general text corpus through self-supervised learning tasks, capturing rich linguistic knowledge and semantic associations. In practical applications, these pre-trained parameters can be used directly or fine-tuned on a small amount of domain-specific data to adapt to the characteristics of the medical field. The final output is the semantic feature vector of the user text query.

[0029] It should be understood that although the text query input by the user is in the form of natural language, its core is to express specific medical concepts. Traditional retrieval methods find it difficult to accurately extract these advanced medical concepts directly from unstructured text, resulting in a semantic gap between the retrieval results and the user's true intentions. Therefore, in this application, through medical named entity recognition, key medical terms such as disease names, symptoms, examination methods, etc. can be accurately extracted from the query. Subsequently, these identified terms are linked to the unique entity ID in the pre-built medical knowledge graph, so that the originally vague text expression is converted into a structured and unambiguous set of medical concepts. This concept set not only provides the core knowledge elements for subsequent query semantic enhancement, it also enables concept-level matching and secondary sorting based on medical expertise, thereby ensuring the high medical relevance and accuracy of the retrieval results, and effectively bridging the semantic gap between user queries and image content.

[0030] In particular, in one implementation of the present application, step S32 is processed as follows: First, the user text query is input into a medical named entity recognition tool. This tool is implemented using a deep learning model, such as a sequence labeling model based on the Transformer architecture, whose underlying layer can be a pre-trained language model fine-tuned for the medical field. The model's input is the sequence of tokens in the text query. Through a multi-layer self-attention mechanism and a feedforward network, the model learns the representation of each token in context. The model's output layer is connected to a linear classification layer and a conditional random field (CRF) layer to predict the medical entity label for each token, such as B-disease, I-disease, O, etc. The model's weights and bias parameters are obtained through supervised learning training on a large-scale, manually annotated medical text corpus. For example, for a query looking for CT images with ground-glass opacity in the lungs, the medical named entity recognition tool will identify lungs as a body part entity, ground-glass opacity as an image feature entity, and CT as an examination method entity.

[0031] Next, entity linking is performed on the identified medical terms, mapping them to the corresponding entity IDs in the pre-built medical knowledge graph. For each identified medical term, such as lung, the method queries the medical knowledge graph. The query strategy includes: first trying to perform an exact match to find out whether there is an entity name or alias in the knowledge graph that is exactly the same as the term. If it exists, the corresponding unique entity ID is directly obtained. For example, lung may be exactly matched to the lung concept with entity ID KG_ID_001 in the knowledge graph. If the exact match fails, fuzzy matching or semantic similarity matching is attempted. This can be achieved by calculating the string similarity between the identified term and the entity name or alias in the knowledge graph, such as Levenshtein distance or Jaccard similarity, or calculating the cosine similarity between their word embedding vectors. For example, a similarity threshold is set, such as 0.8, and if the similarity between a knowledge graph entity and the identified term exceeds this threshold, they are considered to be matched. For potentially ambiguous terms, such as nodule, if the query also includes the word "lung," the entity ID for lung nodule in the knowledge graph will be preferentially linked to the lung-related entity ID, rather than the thyroid nodule entity ID, demonstrating the power of contextual disambiguation. In this way, each identified medical term is successfully linked to one or more unique entity IDs in the medical knowledge graph. Ultimately, these linked entity IDs constitute the query-related graph concept set, such as {KG_ID_001 (lung), KG_ID_002 (ground glass density), KG_ID_003 (computed tomography)}.

[0032] Accordingly, after the initial processing of the original text query input by the user, although a general semantic feature vector and a set of clear medical concepts are obtained, these two parts of information need to be intelligently integrated to form a comprehensive query representation that contains both linguistic context and rich medical expertise. Moreover, the original text vector may not be able to fully capture the deep meaning of the medical concept, and the isolated medical concept lacks the overall context of the query. Based on this, in the technical solution of this application, a graph-driven attention fusion module is introduced, which can dynamically evaluate the importance of each medical concept for understanding the core intent of the user's query, and organically combine it with the overall semantics of the text, so that the final query representation can go beyond simple literal matching and truly understand the medical intent behind the user's query, thereby providing a more powerful and semantically insightful foundation for subsequent precise retrieval.

[0033] In particular, in one implementation method of the present application, step S33, inputting the query-related graph concept set and the user text query semantic feature vector into the graph-driven text attention fusion module to obtain the user text query graph-driven enhanced embedding vector, includes: S331, performing semantic embedding coding on each query-related graph concept in the query-related graph concept set to obtain a set of query-related graph concept semantic embedding coding vectors; S332, calculating the cross-attention weights of the user text query semantic feature vector relative to each query-related graph concept semantic embedding coding vector in the set of query-related graph concept semantic embedding coding vectors to obtain a set of cross-attention weights; S333, fusing the user text query semantic feature vector and the set of query-related graph concept semantic embedding coding vectors based on the set of cross-attention weights to obtain the user text query graph-driven enhanced embedding vector.

[0034] In particular, in one implementation of the present application, step S33 is processed as follows: First, sub-step S331 is performed. This step relies on a pre-trained knowledge graph embedding model. The model is a neural network model whose core goal is to map all entities and relationships in the medical knowledge graph into a low-dimensional, continuous vector space. For example, a knowledge graph embedding model based on an energy function, such as the TransE model, can be used. The network architecture of this model is relatively simple and mainly includes an embedding layer and a scoring function. The embedding layer stores the vector representation of each entity and relationship in the knowledge graph, and these vectors are learnable parameters of the model. When given a triple in a knowledge graph, a head entity, a relationship, and a tail entity, the model evaluates the rationality of the triple through a scoring function. For example, the TransE model evaluates by calculating the distance between the sum of the head entity vector and the relationship vector and the tail entity vector. The weights and bias parameters of the knowledge graph embedding model are obtained through supervised learning training on a large-scale medical knowledge graph. After training, each entity in the knowledge graph has a unique, fixed-dimensional semantic embedding encoding vector. When receiving the entity ID in the query-related graph concept set, this step directly searches and extracts the query-related graph concept semantic embedding encoding vector corresponding to the entity ID from the pre-trained knowledge graph embedding model.

[0035] Next, proceed to sub-step S332. This process uses an attention mechanism to measure the correlation between the user text query and each graph concept. In particular, in one implementation of the present application, step S332 calculates the cross-attention weights of the user text query semantic feature vector relative to each query-related graph concept semantic embedding encoding vector in the set of query-related graph concept semantic embedding encoding vectors to obtain a set of cross-attention weights, including: calculating the cross-attention weights using the following formula, which is: ;in, and is a learnable weight matrix, which is optimized by the back propagation algorithm during the model training process. Query semantic feature vectors for user text, For each query-related graph concept semantic embedding coding vector in the set of query-related graph concept semantic embedding coding vectors, is the scaling factor, i.e. The dimension of the transformed vector is, for example, 8, to prevent the dot product result from being too large and causing the Softmax function gradient to disappear. is the Softmax activation function, is the individual cross-attention weights in the set of cross-attention weights. This allows the model to dynamically assess the importance of each medical concept in understanding the core intent of the user's query. For example, if the query is "ground-glass opacity of the lung," the concept "ground-glass opacity" may receive a higher weight because it directly describes imaging features, while "lung," as an anatomical location, while important, may receive a slightly lower weight because the query focuses on pathological features. Through this weighting, the semantic representation of the query can more accurately focus on the medical concepts that the user truly cares about.

[0036] Finally, proceed to step S333. First, embed the semantics of each query-related graph concept into the encoding vector The corresponding cross attention weight Multiply them together to get the weighted concept vector. Then, sum all the weighted concept vectors to get a comprehensive graph concept representation vector, denoted as Finally, this comprehensive graph concept is represented by the vector Semantic feature vector of user text query The fusion method can be vector concatenation followed by a linear transformation layer, for example, The vectors are concatenated and then projected to a preset output dimension through a fully connected layer, for example, the same dimension as the image embedding vector, to obtain the user text query graph-driven enhanced embedding vector.

[0037] In step S4, the user text query graph-driven enhanced embedding vector is input into the vector retrieval engine, and the vector retrieval engine extracts the top-K images that are most similar to the user text query graph-driven enhanced embedding vector from the semantic index library as a preliminary candidate list. Accordingly, after the semantic enhancement in the previous step, the user query has been converted into a graph-driven enhanced embedding vector rich in medical knowledge, which accurately expresses the user's deep medical needs. The introduction of the vector retrieval engine can quickly and efficiently calculate the similarity between the query vector and all image embedding vectors in the pre-built semantic index library in this high-dimensional semantic space. This retrieval method based on vector similarity goes beyond traditional keyword matching and realizes true semantic matching, thereby being able to quickly screen out the top-K images that are most relevant to the user query semantics from a large image data set, forming a preliminary candidate list for subsequent refined processing, greatly improving the efficiency and relevance of the retrieval.

[0038] In particular, in one possible implementation of the present application, each data item in the semantic index library includes a medical image ID, a medical image multimodal semantic embedding vector, and an associated graph concept set. It is understandable that the medical image ID is a unique identifier, ensuring that the retrieval results can accurately point to the original image. The medical image multimodal semantic embedding vector is a numerical representation of the image content. It integrates the visual features of the image and the semantic information of the relevant text report, so that the vector retrieval engine can efficiently and preliminarily screen out images related to the user's query semantics from massive data based on similarity calculations in high-dimensional space, solving the problem of semantic understanding of image content. The associated graph concept set provides structured medical knowledge labels for image content, which associates images with concepts in the medical knowledge graph, and provides a clear semantic basis for subsequent knowledge-based refined filtering and secondary sorting, ensuring that the retrieval results are not only semantically similar, but also highly relevant and accurate at the medical professional level. Therefore, obtaining these data can make retrieval more accurate and efficient.

[0039] In particular, in one possible implementation of the present application, each data item in the semantic index library is obtained as follows: Generation of medical image ID. When a new piece of medical image data (for example, a CT image or a set of MRI sequences) is captured and stored, a globally unique identifier is assigned to it. This identifier is a string of letters and numbers, or a unique integer serial number, which ensures that each piece of image data is unique in the entire database, facilitating subsequent retrieval and management. For example, a UUID (universally unique identifier) ​​generation algorithm can be used to generate a 32-bit hexadecimal string for each image as its ID, or a database auto-increment primary key can be used.

[0040] Multimodal semantic embeddings for medical images are generated using a multimodal deep learning model. This model consists of two main branches: one for processing the visual modality and the other for processing the textual modality. The visual branch can be a pre-trained convolutional neural network (CNN), such as ResNet or EfficientNet. Its network architecture consists of multiple convolutional layers, pooling layers, and activation functions, extracting hierarchical visual features from raw pixel data. The image data is processed through these layers, ultimately outputting a fixed-dimensional visual embedding through a global average pooling layer or a fully connected layer. The weights and bias parameters of this CNN model are pre-trained on large-scale medical imaging datasets. The text branch can be a pre-trained language model, such as the medically fine-tuned BERT or ClinicalBERT models. Its network architecture is based on a Transformer encoder with a multi-head self-attention mechanism and a feed-forward network. The text content of the radiology report, after tokenization and encoding, is used as input. After processing, the model extracts the final hidden states corresponding to special tokens such as [CLS] as the text embeddings. The weights and bias parameters of this language model are pre-trained through self-supervised learning tasks on a large-scale medical text corpus. Finally, the visual and textual embeddings are input into a fusion module, which can be a simple concatenation layer followed by a fully connected layer to learn how to effectively combine information from the two modalities. The weights and bias parameters of the fusion module are obtained through contrastive learning or joint training on image-report pairs. Ultimately, the fusion module outputs a fixed-dimensional multimodal semantic embedding of the medical image.

[0041] Generation of the association graph concept set. This is done in two steps. The first step is to use a medical named entity recognition (NER) tool to identify key medical terms from the radiology report text, such as disease names like pulmonary nodules, anatomical sites like right upper lobe, and imaging features like ground-glass opacity. This NER tool is a deep learning-based sequence labeling model, such as a Bi-LSTM-CRF or Transformer-CRF model, whose weights and bias parameters are trained on a large number of manually annotated medical text datasets. The second step is entity linking, which maps the identified medical terms to the corresponding entity IDs in a pre-built medical knowledge graph. For each identified term, the method queries the knowledge graph and finds its unique identifier in the knowledge graph through exact matching, fuzzy matching, or a matching algorithm based on semantic similarity. For example, if a pulmonary nodule is identified, it is linked to the unique ID of the pulmonary nodule concept in the knowledge graph. Ultimately, all successfully linked entity IDs constitute the association graph concept set for the image.

[0042] Specifically, in one possible implementation of the present application, Figure 4FIG4 is a flow chart of step S4 in the graph-driven intelligent medical image retrieval method according to an embodiment of the present application. Figure 4 As shown, step S4, the user text query graph-driven enhanced embedding vector is input into a vector retrieval engine, and the vector retrieval engine extracts the Top-K images that are most similar to the user text query graph-driven enhanced embedding vector from the semantic index library as a preliminary candidate list, including: S41, the vector retrieval engine respectively calculates the semantic similarity between the user text query graph-driven enhanced embedding vector and the medical image multimodal semantic embedding vector of each data item in the semantic index library; S42, extracts the preliminary candidate list from the semantic index library based on the size sorting of the semantic similarity.

[0043] It is worth mentioning that the user's text query has been transformed into a high-dimensional, medical knowledge-rich vector representation after semantic enhancement in the previous steps, which accurately captures the user's search intention. At the same time, each medical image in the semantic index library is also encoded into a semantic embedding vector of the same high dimension through multimodal fusion technology, representing the visual and textual content of the image. Therefore, in order to be able to quantify the degree of semantic correlation between the user's query intention and the actual content of each image. This application calculates the similarity between the two, so that the retrieval can understand and match deep semantics, thereby efficiently and accurately identifying images that are highly relevant to user needs in terms of concept and content from massive image data, laying the foundation for subsequent refined screening.

[0044] In particular, in one possible implementation of the present application, step S41 is processed as follows: the vector retrieval engine receives the user text query graph driven enhanced embedding vector and traverses each data item in the semantic index library. For each medical image in the semantic index library, the retrieval engine extracts its corresponding medical image multimodal semantic embedding vector. Subsequently, the retrieval engine calculates the similarity between the user text query graph driven enhanced embedding vector and each medical image multimodal semantic embedding vector. In particular, in one possible implementation of the present application, the semantic similarity is the cosine similarity between the user text query graph driven enhanced embedding vector and the medical image multimodal semantic embedding vector. Those skilled in the art should know that cosine similarity is a measure of the cosine value of the angle between two non-zero vectors. It reflects the similarity in the direction of the two vectors. The numerical range is between -1 and 1. The closer the value is to 1, the more consistent the direction and the more similar the semantics. The calculation formula is: ;in, It is the user text query graph that drives the enhanced embedding vector Middle eigenvalues, is the multimodal semantic embedding vector of medical images Middle eigenvalues, yes and The number of eigenvalues ​​of is the semantic similarity. Through this calculation, each medical image in the semantic index receives a semantic similarity score relative to the user query. In other words, cosine similarity evaluates similarity by measuring vector direction rather than length. This effectively captures conceptual connections between high-dimensional semantic embedding vectors, avoids length bias, and achieves more accurate semantic matching.

[0045] It should be understood that here, when calculating the cosine similarity between the user text query graph-driven enhanced embedding vector and the medical image multimodal semantic embedding vector, considering that the query-related graph concept set and the user text query semantic feature vector are subjected to graph-driven text attention fusion, the leading vector inner product calculation for calculating the cross-attention weight will be forward propagated relative to the trailing vector inner product calculation in the cosine similarity, thereby causing precision attenuation due to the disappearance / explosion of the inner product numerical gradient during the overall calculation process, thereby affecting the calculation accuracy of the semantic similarity.

[0046] Therefore, the present application considers calculating the cosine similarity between the user text query graph driven enhanced embedding vector and the medical image multimodal semantic embedding vector, and realizes inner product gradient smoothing through the statistical quantile correction effect of the feature distribution of the user text query graph driven enhanced embedding vector and the medical image multimodal semantic embedding vector.

[0047] Based on this, preferably, in another implementable manner of the present application, in step S41, the vector retrieval engine respectively calculates the semantic similarity between the user text query graph-driven enhanced embedding vector and the medical image multimodal semantic embedding vector of each data item in the semantic index library, including: respectively calculating the vector quantile representation of the user text query graph-driven enhanced embedding vector and the medical image multimodal semantic embedding vector to obtain the user text vector quantile representation value and the medical image vector quantile representation value, that is: ;in, and They are user text query graph driven enhanced embedding vector The mean and standard deviation of the eigenvalues, and They are the multimodal semantic embedding vectors of medical images The mean and standard deviation of the eigenvalues, and They are the quantile representation values ​​of user text vectors and medical image vectors respectively.

[0048] Based on the quantile adjustment mechanism, the user text query graph-driven enhanced embedding vector and the medical image multimodal semantic embedding vector are changed by element-by-element shift operation through the quantile representation value of the user text vector and the quantile representation value of the medical image vector to obtain the user text displacement parameter matrix and the medical image displacement parameter matrix, namely: ;in, To round down to an integer value, Indicates right shift, that is, the eigenvalues ​​of the vector are circularly shifted right by the specified shift amount. For example, the last eigenvalue is moved to the head, and the remaining eigenvalues ​​are shifted right in sequence. is matrix multiplication, is the transpose operation, and They are the user text displacement parameter matrix and the medical image displacement parameter matrix, which are represented by vector quantiles based on the quantile adjustment mechanism to change the user text query graph through element-by-element displacement operations to drive the enhanced embedding vector. and multimodal semantic embedding vectors of medical images That is, according to the characteristic statistical quantile characteristics, the displacement amplitude of the vector eigenvalue is dynamically adjusted to construct a displacement parameter matrix with a quantile hierarchical structure, so that the user text query graph is driven to enhance the embedding vector and multimodal semantic embedding vectors of medical images Shift right respectively and After rounding down to integer values, construct the matrix and ,In this way, the overall semantic direction is guaranteed, and ,the bitwise gradient homogenization is introduced through the bitwise ,hierarchical structure.

[0049] Based on the user text displacement parameter matrix and the medical image displacement parameter matrix, the user text query graph-driven enhanced embedding vector and the medical image multimodal semantic embedding vector are alternately optimized to obtain a user text query graph-driven enhanced embedding optimization vector and a medical image multimodal semantic embedding optimization vector, namely: ;in, and They are user text query graph driven enhanced embedding optimization vector and medical image multimodal semantic embedding optimization vector, that is, user text query graph driven enhanced embedding vector and multimodal semantic embedding vectors of medical images Alternating optimization is performed based on the displacement parameter matrix.

[0050] The cosine similarity between the user text query graph-driven enhanced embedding optimization vector and the medical image multimodal semantic embedding optimization vector is calculated as the semantic similarity. In this way, the eigenvalue amplitude smoothing between the vectors is achieved through the quantile adjustment mechanism, thereby achieving inner product gradient smoothing based on the eigenvalue product calculation, thereby improving the calculation accuracy of the semantic similarity. In particular, the cosine similarity calculation is the same as the calculation method of the above embodiment.

[0051] It should be understood that in step S41, the semantic similarity between the user query vector and all image vectors in the semantic index library has been calculated. However, the semantic index library may contain millions or even hundreds of millions of images, and subsequent refined processing of all images is impractical and computationally expensive. Therefore, by sorting the similarities and extracting the Top-K images, the search scope can be quickly narrowed to the image set that is most likely to meet the user's needs. This not only significantly improves the retrieval efficiency and avoids unnecessary consumption of computing resources, but also ensures that subsequent more complex refined matching and sorting steps can focus on processing high-quality candidate images, thereby optimizing the performance and response speed of the overall retrieval process while ensuring retrieval relevance.

[0052] In particular, in one implementation of the present application, step S42 is processed as follows: after completing the semantic similarity calculation between all medical images and the user query vector, the retrieval engine will obtain a list containing all image IDs and their corresponding similarity scores. The retrieval engine sorts this list in descending order according to the similarity score from high to low. After the sorting is completed, the retrieval engine will select the top K medical images from the sorted list as a preliminary candidate list. Here, K is a preset integer value, for example, K can be set to 50 or 100. The setting of this value is based on experience, system performance requirements and the load capacity of subsequent refined processing. The purpose of selecting the Top-K images is to quickly screen out the most relevant parts in the massive data for subsequent more complex refined matching and sorting steps, thereby improving the overall retrieval efficiency. Ultimately, the preliminary candidate list will contain the IDs of these K images and their corresponding semantic similarity scores.

[0053] In step S5, based on the query-related atlas concept set, the preliminary candidate list is subjected to secondary atlas concept sorting to obtain a sorted list of related medical images. It should be understood that although preliminary semantic matching is achieved through vector similarity, pure vector space similarity may not be able to fully capture the precise correspondence at the medical concept level in some cases. For example, two images may be similar in vector space, but there may be slight differences in their core medical concepts. Therefore, in order to be able to perform knowledge-driven refined screening and sorting of the preliminary search results to further improve the medical professional relevance and accuracy of the search results, this application introduces the calculation of the overlap between the query-related atlas concept set and the image-related atlas concept set, which can perform secondary verification and sorting from the explicit medical knowledge level to ensure that the final presented images are not only semantically similar, but also highly consistent with the user query in key medical concepts, thereby effectively bridging the potential semantic gap and providing search results that are more in line with clinical needs.

[0054] In particular, in one possible implementation of the present application, step S5, based on the query-related atlas concept set, performs atlas concept secondary filtering on the preliminary candidate list to obtain a sorted list of related medical images, including: calculating the degree of conceptual overlap between the query-related atlas concept set and the associated atlas concept set of each data item in the preliminary candidate list; and performing atlas concept secondary filtering on the preliminary candidate list based on the size of the conceptual overlap sorting.

[0055] Step S5 is processed as follows: First, for each medical image in the preliminary candidate list, the processing process extracts its associated graph concept set and compares it with the query-related graph concept set of the user query. The concept overlap is calculated by first determining the number of medical concepts contained in both concept sets, and then dividing this number by the total number of all unique concepts in the two sets. This ratio reflects the degree of concept overlap, and the value range is between 0 and 1. The closer it is to 1, the higher the concept overlap, that is, the more the image content matches the query at the medical concept level. Through this calculation, each image in the preliminary candidate list will receive an additional concept overlap score.

[0056] After obtaining the initial semantic similarity score and the newly calculated concept overlap score for each image, these two scores need to be fused to form the final ranking criteria. A common fusion approach is to obtain a composite score by weighting the initial semantic similarity score with the concept overlap score. Specifically, each score is multiplied by a preset weight and then summed to obtain a composite ranking score. These weights are used to adjust the relative importance of the two similarities in the final ranking. For example, a weight of 0.7 for semantic similarity and 0.3 for concept overlap can be set to balance their influence. These weighting parameters can be optimized through offline experiments, cross-validation, or expert experience. After calculating the composite scores for all preliminary candidate images, the process sorts them in descending order based on these composite scores, resulting in a final ranked list of relevant medical images. This list is the same length as the initial candidate list, but its internal order has been adjusted to more precisely match the medical concepts, ensuring that the images presented to the user are highly relevant in terms of both semantics and medical expertise.

[0057] In step S6, the sorted list of relevant medical images is displayed. In other words, these carefully screened and sorted results are presented to the user in an intuitive and easy-to-understand manner, allowing the user to directly view, evaluate, and utilize these search results to transform abstract computational results into actionable clinical or research resources. This greatly improves the efficiency and accuracy of the user's acquisition of the required medical images, thereby supporting subsequent diagnosis, treatment decision-making, teaching, or scientific research.

[0058] In particular, in one implementation of the present application, step S6 is processed as follows: First, for each image identifier in the sorted list of related medical images, the processing process retrieves the corresponding original medical image data and its related metadata information from the back-end storage service based on the identifier. These back-end storage services are archiving communication systems (such as PACS) or dedicated image databases specifically used to store and manage medical images. The retrieved original image data may exist in a variety of formats, such as DICOM (Digital Imaging and Communications in Medicine) format, or common image formats such as JPEG, PNG, etc. At the same time, metadata associated with the image, such as the patient's anonymized basic information, examination date, examination type (such as CT, MRI, scan site, and original radiology report text, etc.), will also be obtained.

[0059] In order to present this information efficiently and user-friendly on the user interface, the retrieved data needs to be prepared and processed as necessary. For original medical images, if the file size is large or the format is not suitable for direct display on a web page or application, a smaller thumbnail or preview image will be generated. This process may involve an image processing module to convert the original high-resolution image into a size and format suitable for fast loading and previewing. At the same time, for lengthy radiology report texts, key information or summaries, such as diagnostic conclusions, main findings, etc., will be extracted so that they can be quickly presented to users in a list view. All of this prepared information, including image preview images, image identifiers, formatted key text information, and the comprehensive ranking score of the image, will be structured for easy rendering on the front-end interface.

[0060] Finally, the processing process renders and displays the prepared content through a user interface. This is implemented in a graphical user interface (GUI), which can be a web-based application built using HTML, CSS, and JavaScript, or a standalone desktop application. The display layout can be diverse, depending on the user experience design. For example, a grid view can be used, displaying thumbnails of multiple images in a matrix format, with a brief text description and ranking score below or next to each thumbnail. Another common layout is a list view, where each row displays detailed information about an image, including a small preview, image identifier, report summary, and score. The user interface may also provide interactive features. For example, users can click on an image thumbnail or link to open a separate image viewer to view the image's detailed sequence in full resolution or the complete radiology report text. Furthermore, the interface may provide paging functionality to load and display large results in batches, as well as basic sorting or filtering options to further meet user needs and ensure convenient browsing and access to retrieved high-quality medical images.

[0061] In summary, a graph-driven intelligent medical image retrieval method based on the embodiments of the present application is described, which aims to address the semantic gap problem existing in traditional retrieval, namely the semantic difference between low-level visual features and high-level medical concepts, as well as the shortcomings of existing methods in understanding the user's deep medical intent and ensuring the medical relevance of retrieval results. First, by constructing a medical knowledge graph, the text query entered by the user is deeply semantically enhanced and embedded. This not only identifies and links medical terms in the query to graph entities, but also generates a graph-driven enhanced embedding vector containing high-level medical concepts and a query-related graph concept set. This directly bridges the gap between the text query and medical concepts. Subsequently, the system uses this enhanced vector to perform a preliminary vector similarity search, quickly screening the top-K preliminary candidate images. Finally, based on the query-related graph concept set, these preliminary candidate images are secondary ranked by graph concepts, and the deep medical semantics of the images are accurately matched by calculating the concept overlap. This two-stage retrieval mechanism, which deeply integrates the graph, ensures that the retrieval results are not only visually similar but also highly relevant in terms of medical concepts, thereby significantly improving the accuracy and intelligence of medical image retrieval.

[0062] Figure 5 FIG is a block diagram of a graph-driven intelligent medical image retrieval system according to an embodiment of the present application. Figure 5 As shown, the graph-driven intelligent medical image retrieval system 100 according to an embodiment of the present application includes: a medical knowledge graph construction module 110, which is used to construct a medical knowledge graph; a user text query acquisition module 120, which is used to obtain a user text query; a user text query encoding module 130, which is used to perform semantic enhancement embedding encoding on the user text query based on the medical knowledge graph to obtain a user text query graph-driven enhanced embedding vector and a query-related graph concept set; a preliminary candidate list generation module 140, which is used to input the user text query graph-driven enhanced embedding vector into a vector retrieval engine, and the vector retrieval engine extracts the Top-K images most similar to the user text query graph-driven enhanced embedding vector from the semantic index library as a preliminary candidate list; a preliminary candidate list sorting module 150, which is used to perform graph concept secondary sorting on the preliminary candidate list based on the query-related graph concept set to obtain a sorted related medical image list; and a related medical image list display module 160, which is used to display the sorted related medical image list.

[0063] Here, those skilled in the art will appreciate that the specific operations of each step in the above-mentioned graph-driven intelligent medical image retrieval system have been described in the above reference. Figures 1 to 4 The description of the graph-driven intelligent medical image retrieval method has been introduced in detail, and therefore, its repeated description will be omitted.

Claims

1. A graph-driven intelligent medical image retrieval method, characterized in that: include: Construct medical knowledge graph; Get user text query; Based on the medical knowledge graph, the user text query is semantically enhanced and embedded to obtain a user text query graph-driven enhanced embedding vector and a query-related graph concept set; the user text query graph-driven enhanced embedding vector is input into a vector search engine, and the vector search engine extracts the top-K images most similar to the user text query graph-driven enhanced embedding vector from a semantic index library as a preliminary candidate list; Based on the query-related atlas concept set, performing atlas concept secondary sorting on the preliminary candidate list to obtain a sorted list of related medical images; and displaying the sorted list of related medical images; Wherein, based on the medical knowledge graph, the user text query is semantically enhanced and embedded in coding to obtain a user text query graph-driven enhanced embedding vector and a query-related graph concept set, including: performing text preprocessing on the user text query and inputting the query into a text editor to obtain a user text query semantic feature vector; using a medical named entity recognition tool to identify medical terms from the user text query, and linking the medical terms to corresponding entity IDs in the medical knowledge graph to obtain the query-related graph concept set; inputting the query-related graph concept set and the user text query semantic feature vector into a graph-driven text attention fusion module to obtain the user text query graph-driven enhanced embedding vector; Among them, based on the query-related atlas concept set, the preliminary candidate list is subjected to atlas concept secondary filtering to obtain a sorted list of related medical images, including: calculating the concept overlap between the query-related atlas concept set and the associated atlas concept set of each data item in the preliminary candidate list; and performing atlas concept secondary filtering on the preliminary candidate list based on the size of the concept overlap sorting.

2. The graph-driven intelligent medical image retrieval method according to claim 1, characterized in that: The query-related graph concept set and the user text query semantic feature vector are input into a graph-driven text attention fusion module to obtain the user text query graph-driven enhanced embedding vector, including: performing semantic embedding coding on each query-related graph concept in the query-related graph concept set to obtain a set of query-related graph concept semantic embedding coding vectors; calculating the cross-attention weights of the user text query semantic feature vector relative to each query-related graph concept semantic embedding coding vector in the set of query-related graph concept semantic embedding coding vectors to obtain a set of cross-attention weights; and fusing the user text query semantic feature vector and the set of query-related graph concept semantic embedding coding vectors based on the set of cross-attention weights to obtain the user text query graph-driven enhanced embedding vector.

3. The graph-driven intelligent medical image retrieval method according to claim 2, characterized in that: Calculating the cross-attention weights of the user text query semantic feature vector relative to each query-related graph concept semantic embedding encoding vector in the set of query-related graph concept semantic embedding encoding vectors to obtain a set of cross-attention weights, including: calculating the cross-attention weights using the following formula, wherein the formula is: ;in, and is the learnable weight matrix, Query semantic feature vectors for user text, For each query-related graph concept semantic embedding coding vector in the set of query-related graph concept semantic embedding coding vectors, is the scaling factor, is the Softmax activation function, are the individual cross-attention weights in the set of cross-attention weights.

4. The graph-driven intelligent medical image retrieval method according to claim 1, characterized in that: Each data item in the semantic index library includes a medical image ID, a medical image multimodal semantic embedding vector, and an associated graph concept set.

5. The graph-driven intelligent medical image retrieval method according to claim 4, characterized in that: The user text query graph-driven enhanced embedding vector is input into a vector retrieval engine, and the vector retrieval engine extracts the Top-K images most similar to the user text query graph-driven enhanced embedding vector from a semantic index library as a preliminary candidate list, including: the vector retrieval engine calculates the semantic similarity between the user text query graph-driven enhanced embedding vector and the medical image multimodal semantic embedding vector of each data item in the semantic index library; and extracts the preliminary candidate list from the semantic index library based on the order of the semantic similarity.

6. The graph-driven intelligent medical image retrieval method according to claim 5, characterized in that: The semantic similarity is the cosine similarity between the user text query graph-driven enhanced embedding vector and the medical image multimodal semantic embedding vector.

7. The graph-driven intelligent medical image retrieval method according to claim 5, characterized in that: The vector retrieval engine calculates the semantic similarity between the user text query graph-driven enhanced embedding vector and the medical image multimodal semantic embedding vector of each data item in the semantic index library, including: respectively calculating the vector quantile representation of the user text query graph-driven enhanced embedding vector and the medical image multimodal semantic embedding vector to obtain the user text vector quantile representation value and the medical image vector quantile representation value; based on the quantile adjustment mechanism, the user text query graph-driven enhanced embedding vector and the medical image vector quantile representation value are changed by element-by-element shift operation. The distribution form of the medical image multimodal semantic embedding vector is obtained to obtain a user text displacement parameter matrix and a medical image displacement parameter matrix; based on the user text displacement parameter matrix and the medical image displacement parameter matrix, the user text query graph driven enhanced embedding vector and the medical image multimodal semantic embedding vector are alternately optimized to obtain a user text query graph driven enhanced embedding optimization vector and a medical image multimodal semantic embedding optimization vector; the cosine similarity between the user text query graph driven enhanced embedding optimization vector and the medical image multimodal semantic embedding optimization vector is calculated as the semantic similarity.

8. A graph-driven intelligent medical image retrieval system for executing the graph-driven intelligent medical image retrieval method according to any one of claims 1 to 7, characterized in that: include: Medical knowledge graph construction module, used to construct medical knowledge graph; User text query acquisition module, used to obtain user text queries; A user text query encoding module is configured to perform semantic enhancement embedding encoding on the user text query based on the medical knowledge graph to obtain a user text query graph-driven enhanced embedding vector and a query-related graph concept set; a preliminary candidate list generation module is configured to input the user text query graph-driven enhanced embedding vector into a vector search engine, and the vector search engine extracts the top-K images most similar to the user text query graph-driven enhanced embedding vector from a semantic index library as a preliminary candidate list; A preliminary candidate list sorting module is used to perform secondary atlas concept sorting on the preliminary candidate list based on the query-related atlas concept set to obtain a sorted related medical image list; a related medical image list display module is used to display the sorted related medical image list.

Images