Atlas-driven intelligent medical image retrieval method and system

Abstract

The invention discloses a graph-driven intelligent medical image retrieval method and system, and relates to the field of image retrieval, and the method comprises the steps: firstly, coding a text query input by a user through constructing a medical knowledge graph, querying to generate a graph-driven enhanced embedded vector containing an advanced medical concept, and querying a related graph concept set; then, performing preliminary vector similarity retrieval by using the enhanced vector, and quickly screening out Top-K preliminary candidate images; and finally, on the basis of querying a related map concept set, carrying out map concept secondary sorting on the preliminary candidate images, and accurately matching deep medical semantics of the images by calculating a concept coincidence degree. The mechanism of two-stage retrieval and map deep fusion ensures that the retrieval result is not only similar in vision, but also highly related in medical concept, so that the accuracy and intelligent level of medical image retrieval are remarkably improved.

Description

Technical Field

[0001] This application relates to the field of image retrieval, and more specifically, to a graph-driven intelligent medical image retrieval method and system. Background Art

[0002] In the field of healthcare, with the rapid development of medical imaging technology, the explosion of various types of image data such as CT, MRI, and X-ray has provided a rich information source for clinical diagnosis and scientific research. However, how to efficiently and accurately retrieve images highly relevant to specific clinical needs from these massive amounts of image data has become a major challenge. This mainly stems from the semantic gap problem commonly existing in medical image retrieval. Specifically, the underlying visual features of medical images, such as pixel intensity, texture, shape, etc., are low-level information that is easy for computers to extract; while what doctors (users) rely on during diagnosis are high-level medical concepts such as nodules, infiltrations, space-occupying lesions, and thickened pulmonary markings. Mere visual feature similarity often cannot be equivalent to true similarity in medical diagnosis or pathology, resulting in traditional content-based image retrieval systems being difficult to accurately understand the deep medical intentions of users, with a significant semantic deviation between the retrieval results and the actual clinical needs, and it is difficult to achieve truly intelligent retrieval.

[0003] Some existing medical image retrieval methods either focus on using deep learning techniques to extract multi-modal features from images for vector matching or attempt to retrieve through text keywords in image reports. Although these methods have improved the retrieval efficiency to a certain extent, they generally fail to effectively bridge the above-mentioned semantic gap. For example, in visual feature-based retrieval, in the face of image variations under different patients, different devices, or different scanning parameters, the situations of the same disease with different images or different diseases with the same image are likely to occur, resulting in insufficient medical relevance of the retrieval results. And text keyword-based retrieval is limited by the ambiguity of natural language and the complexity of medical terms, and it is difficult to capture the deep semantic associations and logical relationships between medical concepts. Even if some systems attempt to introduce some structured knowledge, if these knowledge cannot be deeply integrated into the retrieval process to play a real driving role, the system may still degenerate into traditional low-level feature-based matching, unable to achieve precise semantic understanding of user queries and medical conceptual expression of image content, thus limiting its practical application value in auxiliary diagnosis, teaching, and scientific research.

[0004] In view of the above challenges, there is an urgent need for a more intelligent and accurate medical image retrieval method that can deeply understand user intentions and bridge the semantic gap. Summary of the Invention

[0005] In view of the above limitations of the existing methods, according to one aspect of the present application, there is provided a graph-driven intelligent medical image retrieval method, which includes: constructing a medical knowledge graph; obtaining a user text query; based on the medical knowledge graph, performing semantic enhanced embedding encoding on the user text query to obtain a graph-driven enhanced embedding vector of the user text query and a query-related graph concept set; inputting the graph-driven enhanced embedding vector of the user text query into a vector retrieval engine, and the vector retrieval engine extracts the top-K images most similar to the graph-driven enhanced embedding vector of the user text query from the semantic index library as a preliminary candidate list; based on the query-related graph concept set, performing secondary sorting of graph concepts on the preliminary candidate list to obtain a sorted list of relevant medical images; and displaying the sorted list of relevant medical images.

[0006] According to another aspect of the present application, there is provided a graph-driven intelligent medical image retrieval system, which includes: a medical knowledge graph construction module for constructing a medical knowledge graph; a user text query acquisition module for obtaining a user text query; a user text query encoding module for performing semantic enhanced embedding encoding on the user text query based on the medical knowledge graph to obtain a graph-driven enhanced embedding vector of the user text query and a query-related graph concept set; a preliminary candidate list generation module for inputting the graph-driven enhanced embedding vector of the user text query into a vector retrieval engine, and the vector retrieval engine extracts the top-K images most similar to the graph-driven enhanced embedding vector of the user text query from the semantic index library as a preliminary candidate list; a preliminary candidate list sorting module for performing secondary sorting of graph concepts on the preliminary candidate list based on the query-related graph concept set to obtain a sorted list of relevant medical images; and a relevant medical image list display module for displaying the sorted list of relevant medical images.

[0007] Compared with the prior art, the graph-driven intelligent medical image retrieval method and system provided by the present application first perform deep semantic enhanced embedding encoding on the text query input by the user by constructing a medical knowledge graph, which not only identifies and links medical terms in the query to graph entities, but also generates a graph-driven enhanced embedding vector containing advanced medical concepts and a query-related graph concept set. This directly bridges the gap between the text query and medical concepts. Subsequently, this enhanced vector is used for preliminary vector similarity retrieval to quickly screen out the top-K preliminary candidate images. Finally, based on the query-related graph concept set, secondary sorting of graph concepts is performed on these preliminary candidate images, and the deep medical semantics of the images are precisely matched by calculating the concept coincidence degree. This two-stage retrieval mechanism deeply integrated with the graph ensures that the retrieval results are not only visually similar, but also highly relevant in medical concepts, thus significantly improving the accuracy and intelligence level of medical image retrieval. Brief Description of the Drawings

[0008] The above and other objects, features, and advantages of the present application will become more apparent by describing the embodiments of the present application in more detail in conjunction with the accompanying drawings. The accompanying drawings are used to provide a further understanding of the embodiments of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the present application and do not constitute a limitation to the present application. In the accompanying drawings, the same reference numerals generally represent the same components or steps.

[0009] Figure 1 It is a flowchart of a graph-driven intelligent medical image retrieval method according to an embodiment of the present application.

[0010] Figure 2 It is a schematic diagram of data flow of a graph-driven intelligent medical image retrieval method according to an embodiment of the present application.

[0011] Figure 3 It is a flowchart of step S3 in a graph-driven intelligent medical image retrieval method according to an embodiment of the present application.

[0012] Figure 4 It is a flowchart of step S4 in a graph-driven intelligent medical image retrieval method according to an embodiment of the present application.

[0013] Figure 5 It is a block diagram of a graph-driven intelligent medical image retrieval system according to an embodiment of the present application. Detailed Description of the Embodiments

[0014] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although some embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. On the contrary, these embodiments are provided to more thoroughly and completely understand the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not used to limit the protection scope of the present disclosure.

[0015] In view of the problems in the above background art, the present application proposes a graph-driven intelligent medical image retrieval method. Figure 1 It is a flowchart of a graph-driven intelligent medical image retrieval method according to an embodiment of the present application. Figure 2 It is a schematic diagram of data flow of a graph-driven intelligent medical image retrieval method according to an embodiment of the present application. As Figure 1 and Figure 2As shown, the atlas-driven intelligent medical image retrieval method according to an embodiment of the present application includes: S1, constructing a medical knowledge graph; S2, obtaining a user text query; S3, based on the medical knowledge graph, performing semantic enhanced embedding encoding on the user text query to obtain a user text query atlas-driven enhanced embedding vector and a query-related atlas concept set; S4, inputting the user text query atlas-driven enhanced embedding vector into a vector retrieval engine, and the vector retrieval engine extracts the top-K images most similar to the user text query atlas-driven enhanced embedding vector from a semantic index library as a preliminary candidate list; S5, based on the query-related atlas concept set, performing secondary sorting of atlas concepts on the preliminary candidate list to obtain a sorted list of relevant medical images; S6, displaying the sorted list of relevant medical images.

[0016] In step S1, a medical knowledge graph is constructed. It should be understood that in the field of medical image retrieval, the core challenge faced by traditional methods is the semantic gap, that is, there is a huge difference between the low-level visual features recognized by a computer and the high-level medical concepts understood by a doctor. Simply relying on features such as pixels and textures for matching, or only retrieving through keywords, it is difficult to accurately capture the deep medical intention of a user query, resulting in a deviation between the retrieval results and the actual clinical needs and unable to achieve true intelligence. In order to bridge this gap and endow the retrieval process with the driving force of medical expertise, constructing a medical knowledge graph becomes an essential step. It is worth mentioning that a medical knowledge graph organizes and expresses medical entities and their relationships in a structured form, and can integrate scattered medical knowledge into a rigorous knowledge network. By pre-constructing this graph, subsequent retrieval processes can, based on rich medical semantic information, deeply understand and enhance user queries, thus realizing the transformation from surface feature matching to deep concept association, ensuring the medical relevance and accuracy of retrieval results, and is the cornerstone of realizing intelligent medical image retrieval.

[0017] Specifically, in an implementable manner of the present application, step S1 is processed as follows: First, diverse original medical data is collected, including but not limited to authoritative medical textbooks, clinical practice guidelines, electronic medical record texts, medical literature abstracts, and existing medical ontologies, such as the International Classification of Diseases (ICD) coding system, Medical Subject Headings (MeSH) thesaurus, and Systematized Nomenclature of Medicine (SNOMED) coding system.

[0018] Next, information extraction is carried out, which is the core part of knowledge graph construction, mainly including medical named entity recognition and relation extraction. For medical named entity recognition, a sequence labeling model based on deep learning is adopted, such as a fine-tuned model based on a pre-trained language model (such as BERT or RoBERTa). This model takes a text sequence as input and, through multiple layers of self-attention mechanisms and feed-forward networks, generates context-related embedding representations for each token. Subsequently, these embeddings are fed into a classification head, which is a linear layer, to predict the entity label of each token, for example, B-disease, I-disease, O. To ensure the validity of the label sequence, a conditional random field (CRF) layer is added after the classification head, which can learn the transition probabilities between labels and thus optimize the overall label sequence. The weights and bias parameters of this model are obtained through supervised learning training on a large number of labeled medical text corpora. For example, for the input text "The patient presents with pulmonary nodules and cough", the model will identify and label "pulmonary nodules" as a disease entity and "cough" as a symptom entity.

[0019] Based on entity recognition, relation extraction is carried out, aiming to identify the semantic relations between the recognized entities. This adopts a deep learning-based model, such as one that can also be based on a pre-trained language model. This model takes the sentence containing the recognized entities as input and obtains the context embedding of the sentence through the pre-trained language model. Subsequently, through specific pooling operations, such as the embedding of the start token of the entity or the average pooling of all token embeddings of the entity, the representation of the entity pair is extracted and concatenated with the overall representation of the sentence. This concatenated vector is fed into a multi-layer perceptron (MLP) classifier, which outputs the probability distribution of the possible relation types between the entity pairs, such as "causes", "is located in", "is treated by", etc., through the Softmax activation function. The weights and bias parameters of the model are also obtained through supervised learning training on a medical corpus containing entity relation annotations. For example, for the sentence "Pulmonary nodules cause cough", the model will identify that there is a "causes" relation between "pulmonary nodules" and "cough".

[0020] The extracted entities and relations are initially stored in the form of triples (entity 1, relation, entity 2). To ensure the accuracy and consistency of knowledge, knowledge fusion and alignment are also required. This includes merging synonymous entities from different data sources, resolving entity ambiguity problems, and adjudicating conflicting information. For example, if "lung mass" and "pulmonary space-occupying lesion" refer to the same concept in different sources, they are mapped to a unified entity identifier. Finally, these structured medical entities and relations are stored in a graph database to form a medical knowledge graph.

[0021] In step S2, a user text query is obtained. It can be understood that when faced with a vast amount of medical image data, users often describe the medical concepts, symptoms, or image features they are interested in through natural language. For example, they may search for CT images with ground-glass opacities in the lungs or MRI images showing liver space-occupying lesions. Such text queries are a direct manifestation of the user's intention and also a bridge connecting the user and the underlying image data. Although this original text query may have problems such as semantic ambiguity or non-standard expressions, it is the basis for subsequent semantic enhancement and precise matching through a medical knowledge graph, providing the initial unstructured input for bridging the semantic gap and enabling the retrieval process to start from the user's actual needs rather than relying solely on preset tags or low-level visual features.

[0022] Specifically, in an implementable manner of the present application, step S2 is processed as follows: This process is achieved by providing an intuitive and easy-to-operate text input interface. Users can input their medical image retrieval requirements by various means such as keyboard input or converting speech recognition into text in a specific area of the retrieval interface, such as a search box or query bar. When the user finishes inputting in the text input box and triggers a query operation, such as clicking the search button or pressing the Enter key, the method immediately captures the complete text content input by the user. The captured text content is an original string, such as "lung nodules, accompanied by chest pain". After capture, some basic text validations are performed, such as checking whether the character encoding is correct and whether the text length meets the preset range. For example, the query length is restricted to be between 5 and 200 characters to avoid invalid queries that are too long or too short, and to remove potential malicious input or special characters. These preliminary validations are aimed at ensuring the quality and security of the text data for subsequent processing. After validation, this original text string serves as the output of the user text query.

[0023] In step S3, based on the medical knowledge graph, semantic enhancement embedding encoding is performed on the user text query to obtain a user text query graph-driven enhanced embedding vector and a query-related graph concept set. It should be understood that the original text query input by the user is often in the form of natural language and may have problems such as ambiguity, non-standard expressions, or lack of deep medical semantics. Simple text matching or retrieval based on low-level features cannot accurately capture the high-level medical concepts and intentions behind the user's query. Therefore, in the present application, by introducing a medical knowledge graph, the unstructured text query can be transformed into a structured representation containing medical semantics, namely, a graph-driven enhanced embedding vector and a query-related graph concept set. This enables the system to go beyond literal matching, truly understand the medical meaning of the query, and utilize the rich concept relationships in the knowledge graph to enhance the expression ability of the query, thereby laying a foundation for subsequent precise retrieval and ensuring a high degree of relevance of the retrieval results at the medical level.

[0024] Specifically, in an implementable manner of the present application, Figure 3 is a flowchart of step S3 in the atlas-driven intelligent medical image retrieval method according to an embodiment of the present application. As Figure 3 shown, step S3, based on the medical knowledge atlas, performs semantic enhancement embedding encoding on the user text query to obtain a user text query atlas-driven enhanced embedding vector and a query-related atlas concept set, including: S31, performing text preprocessing on the user text query and then inputting it into a text editor to obtain a user text query semantic feature vector; S32, using a medical named entity recognition tool to recognize medical terms from the user text query and linking the medical terms to the corresponding entity IDs in the medical knowledge atlas to obtain the query-related atlas concept set; S33, inputting the query-related atlas concept set and the user text query semantic feature vector into an atlas-driven text attention fusion module to obtain the user text query atlas-driven enhanced embedding vector.

[0025] It is worth mentioning that the original text query input by the user is in natural language form, and its contained noise, redundant information, and unstructured characteristics make it difficult for the computer to directly understand and process. In order to convert this human-readable text into a machine-understandable and computable numerical representation, that is, a semantic feature vector, text preprocessing is an essential link. It can clean and standardize the text, remove irrelevant characters, and unify the expression form, thereby improving the accuracy and efficiency of subsequent semantic encoding. The role of the text editor is to map the preprocessed text into a high-dimensional vector space, so that queries with similar semantics are close in the vector space, laying a foundation for subsequent vector-based similarity calculation and semantic matching, and effectively bridging the semantic gap between natural language and machine understanding.

[0026] Specifically, in an implementable manner of the present application, step S31 is processed as follows: First, perform text preprocessing on the user text query, which includes text cleaning and standardization, such as removing special symbols and extra spaces in the text, converting full-width characters to half-width characters, and unifying the punctuation format. Subsequently, perform word segmentation. For Chinese text, it is necessary to segment continuous Chinese character sequences into word units with independent semantics. For example, use a word segmentation tool based on a dictionary and a statistical model to segment a CT image with ground-glass opacity in the lungs into lungs, have, ground-glass opacity, of, CT, image. Finally, according to a preset stop word list, remove common words in the text that contribute little to semantics, such as of, is, and has, etc., to reduce noise and highlight the core semantics.

[0027] The preprocessed text sequence, such as the CT image of pulmonary ground-glass opacity, is input into a text editor, namely a pre-trained text encoding model, to generate semantic feature vectors. Here, a pre-trained language model based on the Transformer architecture can be adopted, such as the BERT model pre-trained for the Chinese medical field. The specific architecture of this text encoding model includes an input layer, multiple layers of Transformer encoders, and an output layer. The input layer is responsible for converting the preprocessed word sequence into a numerical form that can be processed by the model: each word is mapped to a corresponding word embedding vector; to capture the position information of the word in the sequence, a position embedding vector is added. The sum of these embedding vectors constitutes the input representation of the model. These input representations are then fed into multiple layers of Transformer encoders, and each encoder layer consists of a multi-head self-attention mechanism module and a feed-forward neural network module. The multi-head self-attention mechanism allows the model to simultaneously focus on information at different positions in the input sequence and calculate the association strength between each word and other words, thereby capturing the context semantics of the words; the feed-forward neural network performs a non-linear transformation on the output of the self-attention mechanism. Through multiple layers of stacking, the model can learn complex semantic patterns and context dependencies in the text.

[0028] After the model processes the entire sequence, the final hidden state corresponding to special tokens such as the [CLS] token is extracted as the semantic feature vector of the entire user text query. This vector is a high-dimensional dense vector that encodes the overall semantic information of the query. In particular, the weights and bias parameters of this model are pre-trained on a large-scale general text corpus through self-supervised learning tasks, which capture rich language knowledge and semantic associations. In practical applications, these pre-trained parameters can be directly used or fine-tuned on a small amount of domain-specific data to adapt to the characteristics of the medical field. The final output is the semantic feature vector of the user text query.

[0029] It should be understood that although the text query input by the user is in the form of natural language, its core lies in expressing specific medical concepts. Traditional retrieval methods are difficult to directly extract these high-level medical concepts from unstructured text, resulting in a semantic gap between the retrieval results and the user's true intention. Therefore, in this application, through medical named entity recognition, key medical terms such as disease names, symptoms, and examination methods can be accurately extracted from the query. Subsequently, these identified terms are linked to the unique entity IDs in the pre-constructed medical knowledge graph, transforming the original vague text expression into a set of structured and unambiguous medical concepts. This concept set not only provides the core knowledge elements for subsequent query semantic enhancement, but also enables concept-level matching and secondary sorting based on medical expertise, thus ensuring a high degree of medical relevance and accuracy of the retrieval results and effectively bridging the semantic difference between the user's query and the image content.

[0030] In particular, in an implementable manner of this application, step S32 is processed as follows: First, the user text query is input into a medical named entity recognition tool. This tool is implemented based on a deep learning model, such as a sequence labeling model based on the Transformer architecture, and its underlying layer can be a pre-trained language model fine-tuned for the medical field. The input of this model is the token sequence in the text query. Through multiple layers of self-attention mechanisms and feed-forward networks, the model learns the representation of each token in the context. At the output layer of the model, a linear classification layer and a conditional random field (CRF) layer are connected to predict the medical entity labels of each token, such as B-disease, I-disease, O, etc. The weights and bias parameters of the model are obtained through supervised learning training on a large-scale, manually annotated medical text corpus. For example, for a query to find CT images with ground-glass opacities in the lungs, the medical named entity recognition tool will identify the lungs as a body part entity, ground-glass opacities as an imaging feature entity, and CT as an examination method entity.

[0031] Next, entity linking is performed on the identified medical terms to map them to the corresponding entity IDs in a pre - constructed medical knowledge graph. For each identified medical term, such as "lung", the method queries the medical knowledge graph. The query strategies include: First, attempt an exact match to find if there is an entity name or alias in the knowledge graph that is exactly the same as the term. If so, directly obtain its corresponding unique entity ID. For example, "lung" may exactly match the lung concept in the knowledge graph with entity ID KG_ID_001. If the exact match fails, then attempt a fuzzy match or semantic similarity match. This can be achieved by calculating the string similarity between the identified term and the entity name or alias in the knowledge graph, such as the Levenshtein distance or Jaccard similarity, or by calculating the cosine similarity between their word embedding vectors. For example, set a similarity threshold, such as 0.8. If the similarity between a knowledge graph entity and the identified term exceeds this threshold, they are considered to match. For terms that may be ambiguous, such as "nodule", if "lung" also appears in the query, it will be preferentially linked to the entity ID of the pulmonary nodule related to the lung in the knowledge graph, rather than the entity ID of the thyroid nodule, which reflects the ability of context disambiguation. In this way, each identified medical term is successfully linked to one or more unique entity IDs in the medical knowledge graph. Finally, these linked entity IDs form a query - related graph concept set, such as {KG_ID_001 (lung), KG_ID_002 (ground - glass opacity), KG_ID_003 (computed tomography)}.

[0032] Correspondingly, after the initial processing of the original text query input by the user, although a general semantic feature vector and a set of clear medical concepts are obtained, these two parts of information need to be intelligently integrated to form a comprehensive query representation that includes both language context and rich medical expertise. And the original text vector may not fully capture the deep meaning of medical concepts, while isolated medical concepts lack the overall context of the query. Based on this, in the technical solution of this application, a graph - driven attention fusion module is introduced, which can dynamically evaluate the importance of each medical concept for understanding the core intention of the user's query and organically combine it with the overall semantics of the text, so that the final query representation can go beyond simple literal matching, truly understand the medical intention behind the user's query, and thus provide a more powerful and semantically insightful basis for subsequent accurate retrieval.

[0033] In particular, in an implementable manner of the present application, step S33, inputting the query-related graph concept set and the user text query semantic feature vector into the graph-driven text attention fusion module to obtain the user text query graph-driven enhanced embedding vector, includes: S331, performing semantic embedding encoding on each query-related graph concept in the query-related graph concept set to obtain a set of query-related graph concept semantic embedding encoding vectors; S332, calculating the cross-attention weights of the user text query semantic feature vector with respect to each query-related graph concept semantic embedding encoding vector in the set of query-related graph concept semantic embedding encoding vectors to obtain a set of cross-attention weights; S333, fusing the user text query semantic feature vector and the set of query-related graph concept semantic embedding encoding vectors based on the set of cross-attention weights to obtain the user text query graph-driven enhanced embedding vector.

[0034] In particular, step S33 is processed as follows in an implementable manner of the present application: First, sub-step S331 is performed. This step relies on a pre-trained knowledge graph embedding model. This model is a neural network model, and its core goal is to map all entities and relationships in the medical knowledge graph into a low-dimensional and continuous vector space. For example, a knowledge graph embedding model based on an energy function, such as the TransE model, can be used. The network architecture of this model is relatively simple and mainly includes an embedding layer and a scoring function. The embedding layer stores the vector representations of each entity and relationship in the knowledge graph, and these vectors are the learnable parameters of the model. When a triple in the knowledge graph, head entity, relationship, tail entity, is given, the model evaluates the rationality of this triple through the scoring function. For example, the TransE model evaluates by calculating the distance between the sum of the head entity vector and the relationship vector and the tail entity vector. The weight and bias parameters of this knowledge graph embedding model are obtained through supervised learning training on a large-scale medical knowledge graph. After training is completed, each entity in the knowledge graph has a unique semantic embedding encoding vector with a fixed dimension. When receiving the entity ID in the query-related graph concept set, this step directly looks up and extracts the query-related graph concept semantic embedding encoding vector corresponding to the entity ID from the pre-trained knowledge graph embedding model.

[0035] Next, sub-step S332 is performed. This process uses an attention mechanism to measure the relevance between the user's text query and each graph concept. Specifically, in an implementable manner of this application, in step S332, calculating the cross-attention weights of the semantic feature vector of the user's text query with respect to each query-related graph concept semantic embedding coding vector in the set of query-related graph concept semantic embedding coding vectors to obtain a set of cross-attention weights, includes: calculating the cross-attention weights using the following formula: ; where and are learnable weight matrices, which are optimized through the backpropagation algorithm during model training, is the semantic feature vector of the user's text query, is each query-related graph concept semantic embedding coding vector in the set of query-related graph concept semantic embedding coding vectors, is the scaling factor, that is the transformed vector dimension, such as 8, to prevent the dot product result from being too large and causing the gradient of the Softmax function to disappear, is the Softmax activation function, is each cross-attention weight in the set of cross-attention weights. In this way, the model can dynamically evaluate the importance of each medical concept for understanding the core intention of the user's query. For example, if the query is ground-glass opacity in the lung, then the concept of ground-glass opacity may obtain a higher weight because it directly describes the imaging feature, while the lung as an anatomical location, although important, may have a slightly lower weight because the focus of the query is on the pathological feature. Through this weighting, the semantic representation of the query can more accurately focus on the medical concepts that the user really cares about.

[0036] Finally, step S333 is performed. First, multiply each query-related graph concept semantic embedding coding vector by its corresponding cross-attention weight to obtain a weighted concept vector. Then, sum all the weighted concept vectors to obtain a comprehensive graph concept representation vector, denoted as . Finally, fuse this comprehensive graph concept representation vector with the semantic feature vector of the user's text query . The fusion method can be to splice the vectors and then connect a linear transformation layer. For example, splice , and then project it to a preset output dimension through a fully connected layer, such as the same dimension as the image embedding vector, to obtain the user's text query graph-driven enhanced embedding vector.

[0037] In step S4, the graph-driven enhanced embedding vector of the user text query is input into the vector retrieval engine, and the vector retrieval engine extracts the top-K images most similar to the graph-driven enhanced embedding vector of the user text query from the semantic index library as the preliminary candidate list. Accordingly, through the semantic enhancement in the previous steps, the user query has been transformed into a graph-driven enhanced embedding vector rich in medical knowledge, which accurately expresses the user's deep medical needs. The introduction of the vector retrieval engine enables the rapid and efficient calculation of the similarity between this query vector and all image embedding vectors in the pre-constructed semantic index library in this high-dimensional semantic space. This retrieval method based on vector similarity transcends traditional keyword matching and achieves true semantic matching, enabling the rapid screening of the top-K images most relevant to the user query semantics from a large image dataset to form a preliminary candidate list for subsequent refined processing, greatly improving the efficiency and relevance of the retrieval.

[0038] Specifically, in an implementable manner of the present application, each data item in the semantic index library includes a medical image ID, a multi-modal semantic embedding vector of the medical image, and an associated graph concept set. It can be understood that the medical image ID is a unique identifier to ensure that the retrieval result can accurately point to the original image. The multi-modal semantic embedding vector of the medical image is a numerical representation of the image content, which integrates the visual features of the image and the semantic information of the relevant text report, enabling the vector retrieval engine to preliminarily screen out images relevant to the user query semantics from a large amount of data based on similarity calculation in a high-dimensional space, solving the problem of semantic understanding of image content. The associated graph concept set provides structured medical knowledge labels for the image content, which associates the image with concepts in the medical knowledge graph, providing a clear semantic basis for subsequent knowledge-based refined filtering and secondary sorting, ensuring that the retrieval results are not only semantically similar but also highly relevant and accurate at the medical professional level. Therefore, obtaining these data can make the retrieval more accurate and efficient.

[0039] Specifically, in an implementable manner of the present application, each data item in the semantic index library is obtained as follows: Generation of the medical image ID. When a new medical image data (for example, a CT image or a group of MRI sequences) is captured and stored, a globally unique identifier is assigned to it. This identifier is a string composed of letters and numbers, or a unique integer serial number, ensuring the uniqueness of each image data in the entire database for subsequent retrieval and management. For example, the UUID (Universally Unique Identifier) generation algorithm can be used to generate a 32-bit hexadecimal string as its ID for each image, or the database auto-increment primary key can be used.

[0040] Medical image multi-modal semantic embedding vector generation. This is achieved through a multi-modal deep learning model. The model consists of two main branches: one for processing the visual modality and the other for processing the text modality. The visual branch can be a pre-trained convolutional neural network (CNN), such as ResNet or EfficientNet, whose network architecture consists of multiple convolutional layers, pooling layers, and activation functions for extracting hierarchical visual features from the raw pixel data. The image data is used as input, and after passing through these layers, it finally outputs a visual embedding vector of a fixed dimension through a global average pooling layer or a fully connected layer. The weights and bias parameters of this CNN model are pre-trained on a large-scale medical image dataset. The text branch can be a pre-trained language model, such as BERT or ClinicalBERT models fine-tuned for the medical field, whose network architecture is based on the Transformer encoder and includes multi-head self-attention mechanisms and feed-forward networks. The text content of the radiology report is tokenized and encoded as input, and after the model processes it, the final hidden state corresponding to special tokens such as the [CLS] token is extracted as the text embedding vector. The weights and bias parameters of this language model are pre-trained through self-supervised learning tasks on a large-scale medical text corpus. Finally, the visual embedding vector and the text embedding vector are input into a fusion module, which can be a simple concatenation layer followed by a fully connected layer for learning how to effectively combine the information of the two modalities. The weights and bias parameters of the fusion module are obtained through contrastive learning or joint training on image-report pairs. Finally, the fusion module outputs a medical image multi-modal semantic embedding vector of a fixed dimension.

[0041] Generation of the associated graph concept set. It is carried out in two steps. The first step is to use a medical named entity recognition (NER) tool to identify key medical terms from the radiology report text, such as disease names like lung nodules, anatomical locations like the upper lobe of the right lung, and imaging features like ground-glass opacity, etc. The NER tool is a deep learning-based sequence labeling model, such as Bi-LSTM-CRF or Transformer-CRF model, whose weights and bias parameters are trained on a large number of manually annotated medical text datasets. The second step is entity linking, mapping the identified medical terms to the corresponding entity IDs in a pre-constructed medical knowledge graph. For each identified term, the method queries the knowledge graph and finds its unique identifier in the knowledge graph through exact matching, fuzzy matching, or semantic similarity-based matching algorithms. For example, if lung nodules are identified, they are linked to the unique ID of the lung nodule concept in the knowledge graph. Finally, all successfully linked entity IDs form the associated graph concept set of this image.

[0042] Specifically, in an implementable manner of this application, Figure 4It is a flowchart of step S4 in the atlas-driven intelligent medical image retrieval method according to an embodiment of the present application. As Figure 4 shown, in step S4, the user text query atlas-driven enhanced embedding vector is input into the vector retrieval engine, and the vector retrieval engine extracts the top-K images most similar to the user text query atlas-driven enhanced embedding vector from the semantic index library as a preliminary candidate list, including: S41, the vector retrieval engine calculates the semantic similarity between the user text query atlas-driven enhanced embedding vector and the medical image multi-modal semantic embedding vectors of each data item in the semantic index library; S42, the preliminary candidate list is extracted from the semantic index library based on the sorting of the semantic similarity magnitudes.

[0043] It is worth mentioning that the user text query has been semantically enhanced through the previous steps and has been transformed into a high-dimensional vector representation rich in medical knowledge, accurately capturing the user's retrieval intention. At the same time, each medical image in the semantic index library has also been encoded into a semantic embedding vector of the same high dimension through multi-modal fusion technology, representing the visual and text content of the image. Therefore, in order to be able to quantify the semantic correlation degree between the user query intention and the actual content of each image. In this application, by calculating the similarity between the two, the retrieval can understand and match deep semantics, so as to efficiently and accurately identify images highly relevant to the user's needs in terms of concept and content from a large amount of image data, laying a foundation for subsequent refined screening.

[0044] Specifically, in an implementable manner of the present application, step S41 is processed as follows: The vector retrieval engine receives the user text query atlas-driven enhanced embedding vector and traverses each data item in the semantic index library. For each medical image in the semantic index library, the retrieval engine extracts its corresponding medical image multi-modal semantic embedding vector. Subsequently, the retrieval engine calculates the similarity between the user text query atlas-driven enhanced embedding vector and each medical image multi-modal semantic embedding vector. Specifically, in an implementable manner of the present application, the semantic similarity is the cosine similarity between the user text query atlas-driven enhanced embedding vector and the medical image multi-modal semantic embedding vector. Those skilled in the art should know that the cosine similarity is a measure of the cosine value of the angle between two non-zero vectors, which reflects the similarity in the directions of the two vectors. The value range is between -1 and 1, and the closer the value is to 1, the more consistent the directions are and the more similar the semantics are. Its calculation formula is: ; where is the user text query atlas-driven enhanced embedding vector the th eigenvalue in is the medical image multi-modal semantic embedding vector the th eigenvalue in is and the number of eigenvalues of is the semantic similarity. Through this calculation, each medical image in each semantic index library will obtain a semantic similarity score relative to the user query. That is, the cosine similarity evaluates similarity by measuring the vector direction rather than the length, can effectively capture the conceptual associations between high-dimensional semantic embedding vectors, avoid length bias, and thus achieve more accurate semantic matching.

[0045] It should be understood that here, when calculating the cosine similarity between the graph-driven enhanced embedding vector of the user text query and the multi-modal semantic embedding vector of the medical image, considering that when performing graph-driven text attention fusion on the query-related graph concept set and the semantic feature vector of the user text query, the inner product calculation of the pre-vector for calculating the cross-attention weight will perform forward propagation relative to the inner product calculation of the post-vector in the cosine similarity, resulting in accuracy decay due to the disappearance / explosion of the inner product numerical gradient in the overall calculation process, affecting the calculation accuracy of the semantic similarity.

[0046] Therefore, this application considers calculating the cosine similarity between the graph-driven enhanced embedding vector of the user text query and the multi-modal semantic embedding vector of the medical image, and realizes the smoothing of the inner product gradient through the statistical quantile correction effect of the feature distributions of the graph-driven enhanced embedding vector of the user text query and the multi-modal semantic embedding vector of the medical image.

[0047] Based on this, preferably, in another implementable manner of this application, in step S41, the vector retrieval engine calculates the semantic similarity between the graph-driven enhanced embedding vector of the user text query and the multi-modal semantic embedding vectors of each data item in the semantic index library respectively, including: calculating the vector quantile representations of the graph-driven enhanced embedding vector of the user text query and the multi-modal semantic embedding vector of the medical image respectively to obtain the user text vector quantile representation value and the medical image vector quantile representation value, that is: ; where and are respectively the mean and standard deviation of the eigenvalues of the graph-driven enhanced embedding vector of the user text query of and are respectively the mean and standard deviation of the eigenvalues of the multi-modal semantic embedding vector of the medical image of and are respectively the user text vector quantile representation value and the medical image vector quantile representation value.

[0048] Based on the quantile representation values of the user text vector and the medical image vector, through a quantile adjustment mechanism, an element-wise displacement operation is used to change the distribution patterns of the user text query graph-driven enhanced embedding vector and the medical image multi-modal semantic embedding vector to obtain a user text displacement parameter matrix and a medical image displacement parameter matrix, that is: ; where is the floor value, represents a right shift, that is, the eigenvalues of the vector are circularly shifted to the right by a specified shift amount. For example, the last eigenvalue is moved to the head, and the other eigenvalues are shifted to the right in sequence. is matrix multiplication, is a transpose operation, and are the user text displacement parameter matrix and the medical image displacement parameter matrix respectively. Through the quantile adjustment mechanism based on the quantile representation of the vector, an element-wise displacement operation is used to change the distribution patterns of the user text query graph-driven enhanced embedding vector and the medical image multi-modal semantic embedding vector . That is, according to the quantile characteristics of feature statistics, the displacement amplitude of the eigenvalues of the vector is dynamically adjusted to construct a displacement parameter matrix with a quantile hierarchical structure. In this way, the user text query graph-driven enhanced embedding vector and the medical image multi-modal semantic embedding vector are respectively right-shifted by and floor values, and then matrices and are constructed. In this way, both the overall semantic direction is ensured, and the bit-by-bit gradient homogenization is introduced through the quantile hierarchical structure.

[0049] Based on the user text displacement parameter matrix and the medical image displacement parameter matrix, the user text query graph-driven enhanced embedding vector and the medical image multi-modal semantic embedding vector are alternately optimized respectively to obtain a user text query graph-driven enhanced embedding optimized vector and a medical image multi-modal semantic embedding optimized vector, that is: ; where and are the user text query graph-driven enhanced embedding optimized vector and the medical image multi-modal semantic embedding optimized vector respectively. That is, the user text query graph-driven enhanced embedding vector and the medical image multi-modal semantic embedding vector are alternately optimized based on the displacement parameter matrix.

[0050] Calculate the cosine similarity between the graph-driven enhanced embedding optimization vector of the user text query and the multi-modal semantic embedding optimization vector of the medical image as the semantic similarity. In this way, the amplitude smoothing of each eigenvalue between vectors is achieved through the quantile adjustment mechanism, thereby realizing the inner product gradient smoothing calculated based on the product of each eigenvalue, and thus improving the calculation accuracy of the semantic similarity. In particular, the calculation of this cosine similarity is the same as the calculation method in the above embodiment.

[0051] It should be understood that in step S41, the semantic similarity between the user query vector and all image vectors in the semantic index library has been calculated. However, the semantic index library may contain millions or even hundreds of millions of images. It is impractical and extremely costly in terms of calculation to perform subsequent refined processing on all images. Therefore, by sorting the similarities and extracting the top-K images, the search scope can be quickly narrowed down to the set of images that are most likely to meet the user's needs. This not only significantly improves the retrieval efficiency, avoids unnecessary consumption of computing resources, but also ensures that the subsequent more complex refined matching and sorting steps can focus on processing high-quality candidate images, thereby optimizing the performance and response speed of the overall retrieval process while ensuring the retrieval relevance.

[0052] In particular, in an implementable manner of the present application, step S42 is processed as follows: After calculating the semantic similarity between all medical images and the user query vector, the retrieval engine will obtain a list containing all image IDs and their corresponding similarity scores. The retrieval engine sorts this list in descending order according to the similarity scores. After sorting, the retrieval engine will select the top K medical images from the sorted list as the preliminary candidate list. Here, K is a preset integer value. For example, K can be set to 50 or 100. The setting of this value is based on experience, system performance requirements, and the load capacity of subsequent refined processing. The purpose of selecting the top-K images is to quickly screen out the most relevant parts from the massive data for subsequent more complex refined matching and sorting steps to process, thereby improving the overall retrieval efficiency. Finally, the preliminary candidate list will contain the IDs of these K images and their corresponding semantic similarity scores.

[0053] In step S5, based on the query-related atlas concept set, the preliminary candidate list is subjected to a second sorting of atlas concepts to obtain a sorted list of relevant medical images. It should be understood that although a preliminary semantic match is achieved through vector similarity, pure vector space similarity may not fully capture the exact correspondence at the medical concept level in some cases. For example, two images may be similar in the vector space, but there may be subtle differences in their core medical concepts. Therefore, in order to perform knowledge-driven refined screening and sorting on the preliminary retrieval results to further improve the medical professional relevance and accuracy of the retrieval results, this application introduces the calculation of the coincidence degree between the query-related atlas concept set and the image-associated atlas concept set, which can perform secondary verification and sorting from the explicit medical knowledge level, ensuring that the finally presented images are not only semantically similar, but also highly consistent with the user's query in terms of key medical concepts, thus effectively bridging the potential semantic gap and providing retrieval results that better meet clinical needs.

[0054] Specifically, in an implementable manner of this application, step S5, based on the query-related atlas concept set, performs a second filtering of atlas concepts on the preliminary candidate list to obtain a sorted list of relevant medical images, including: calculating the concept coincidence degree between the query-related atlas concept set and the associated atlas concept sets of each data item in the preliminary candidate list; and performing a second filtering of atlas concepts on the preliminary candidate list based on the sorting of the concept coincidence magnitudes.

[0055] The processing of step S5 is as follows: First, for each medical image in the preliminary candidate list, the processing process extracts its associated atlas concept set and compares it with the query-related atlas concept set of the user query. The calculation method of the concept coincidence degree is to first determine the number of medical concepts jointly contained in the two concept sets, and then divide this number by the total number of all unique concepts in these two sets. This ratio reflects the degree of concept coincidence, and the value range is between 0 and 1. The closer it is to 1, the higher the concept coincidence degree, that is, the more the image content matches the query at the medical concept level. Through this calculation, each image in the preliminary candidate list will obtain an additional concept coincidence degree score.

[0056] Subsequently, after obtaining the initial semantic similarity scores and the newly calculated concept overlap scores for each image, these two scores need to be fused to obtain the final sorting basis. A commonly used fusion method is to obtain a comprehensive score by performing a weighted combination of the preliminary semantic similarity score and the concept overlap score. That is, multiply the two scores by the preset weights respectively, and then add the results to obtain a comprehensive sorting score. These weights are used to adjust the relative importance of the two similarities in the final sorting. For example, the weight of the semantic similarity can be set to 0.7 and the weight of the concept overlap can be set to 0.3 to balance their effects. These weight parameters can be optimized and set through offline experiments, cross-validation, or expert experience. After calculating the comprehensive scores of all preliminary candidate images, the processing will sort them in descending order according to these comprehensive scores, so as to obtain the final sorted list of relevant medical images. The length of this list is the same as that of the preliminary candidate list, but its internal order has been adjusted according to the more accurate medical concept matching degree, ensuring that the images finally presented to the user are highly relevant both semantically and medically.

[0057] In step S6, display the sorted list of relevant medical images. That is, present these carefully screened and sorted results to the user in an intuitive and easy-to-understand way, enabling the user to directly view, evaluate, and utilize these retrieval results to transform the abstract calculation results into actionable clinical or research resources, greatly improving the efficiency and accuracy of the user in obtaining the required medical images, thereby supporting subsequent diagnosis, treatment decision-making, teaching, or research work.

[0058] Specifically, in an implementable manner of the present application, step S6 is processed as follows: First, for each image identifier in the sorted list of relevant medical images, the processing will retrieve the corresponding original medical image data and its related metadata information from the backend storage service according to this identifier. These backend storage services are archival communication systems (such as PACS) or dedicated image databases specifically used for storing and managing medical images. The retrieved original image data may exist in various formats, such as DICOM (Digital Imaging and Communications in Medicine) format, or common image formats such as JPEG, PNG, etc. At the same time, the metadata associated with the image, such as the anonymized basic information of the patient, the examination date, the examination type, such as CT, MRI, the scanned part, and the original radiology report text, etc., will also be obtained together.

[0059] To efficiently and user - friendly display this information on the user interface, the retrieved data needs to be prepared and processed as necessary. For the original medical images, if the file is large or the format is not suitable for direct display on a web page or application, smaller thumbnails or preview images will be generated. This process may involve an image - processing module to convert the original high - resolution image into a size and format suitable for quick loading and preview. At the same time, for the long radiology report text, key information or summaries, such as the diagnosis conclusion, main findings, etc., will be extracted for quick presentation to the user in a list view. All this prepared information, including the image preview, image identifier, formatted key text information, and the comprehensive ranking score of the image, will be structured for easy rendering by the front - end interface.

[0060] Finally, the processing process renders and displays these prepared contents through the user interface. This is achieved in a graphical user interface (GUI), which can be a web - based application, such as one built using HTML, CSS, and JavaScript, or a stand - alone desktop application. The display method can be diversified according to the user experience design. For example, a grid view can be adopted, arranging the thumbnails of multiple images in a matrix form, with brief text descriptions and ranking scores attached below or beside each thumbnail. Another common layout is the list view, where each row displays the detailed information of one image, including a small preview, image identifier, report summary, and score. The user interface also provides interactive functions. For example, the user can click on the thumbnail or link of an image to open an independent image viewer to view the detailed sequence of the image at full resolution or view the complete radiology report text. In addition, the interface may also provide paging functions to load and display in batches when the number of results is large, as well as basic sorting or filtering options to further meet the personalized needs of users, ensuring that users can conveniently browse and access the retrieved high - quality medical images.

[0061] In summary, the atlas-driven intelligent medical image retrieval method based on the embodiments of the present application is elucidated. It aims to solve the semantic gap problem in traditional retrieval, that is, the semantic difference between low-level visual features and high-level medical concepts, as well as the deficiencies of existing methods in understanding the deep medical intentions of users and ensuring the medical relevance of retrieval results. First, by constructing a medical knowledge graph, deep semantic enhanced embedding encoding is performed on the text query input by the user. This not only identifies and links medical terms in the query to graph entities but also generates an atlas-driven enhanced embedding vector containing high-level medical concepts and a query-related graph concept set. This directly bridges the gap between the text query and medical concepts. Subsequently, the system uses this enhanced vector for preliminary vector similarity retrieval to quickly screen out the Top-K preliminary candidate images. Finally, based on the query-related graph concept set, secondary sorting of graph concepts is performed on these preliminary candidate images, and the deep medical semantics of the images are accurately matched by calculating the concept coincidence degree. This two-stage retrieval mechanism deeply integrated with the graph ensures that the retrieval results are not only visually similar but also highly relevant in medical concepts, thus significantly improving the accuracy and intelligence level of medical image retrieval.

[0062] Figure 5 FIG. is a block diagram of an atlas-driven intelligent medical image retrieval system according to an embodiment of the present application. As Figure 5 shown, the atlas-driven intelligent medical image retrieval system 100 according to an embodiment of the present application includes: a medical knowledge graph construction module 110 for constructing a medical knowledge graph; a user text query acquisition module 120 for acquiring a user text query; a user text query encoding module 130 for performing semantic enhanced embedding encoding on the user text query based on the medical knowledge graph to obtain a user text query atlas-driven enhanced embedding vector and a query-related graph concept set; a preliminary candidate list generation module 140 for inputting the user text query atlas-driven enhanced embedding vector into a vector retrieval engine, and the vector retrieval engine extracting the Top-K images most similar to the user text query atlas-driven enhanced embedding vector from a semantic index library as a preliminary candidate list; a preliminary candidate list sorting module 150 for performing secondary sorting of graph concepts on the preliminary candidate list based on the query-related graph concept set to obtain a sorted relevant medical image list; and a relevant medical image list display module 160 for displaying the sorted relevant medical image list.

[0063] Here, those skilled in the art can understand that the specific operations of each step in the above atlas-driven intelligent medical image retrieval system have been introduced in detail in the description of the atlas-driven intelligent medical image retrieval method above with reference to Figures 1 to 4 and thus, the repeated description thereof will be omitted.

Claims

1. A method for atlas-driven intelligent medical image retrieval, characterized in that Including: Constructing a medical knowledge graph; Obtaining a user text query; Based on the medical knowledge graph, performing semantic enhanced embedding encoding on the user text query to obtain a graph-driven enhanced embedding vector of the user text query and a query-related graph concept set; inputting the graph-driven enhanced embedding vector of the user text query into a vector retrieval engine, and the vector retrieval engine extracts the top-K images most similar to the graph-driven enhanced embedding vector of the user text query from a semantic index library as a preliminary candidate list; Based on the query-related graph concept set, performing secondary sorting on the preliminary candidate list to obtain a sorted list of relevant medical images; displaying the sorted list of relevant medical images.

2. The atlas-driven intelligent medical image retrieval method according to claim 1, wherein Based on the medical knowledge graph, performing semantic enhanced embedding encoding on the user text query to obtain a graph-driven enhanced embedding vector of the user text query and a query-related graph concept set, including: performing text preprocessing on the user text query and then inputting it into a text editor to obtain a semantic feature vector of the user text query; using a medical named entity recognition tool to identify medical terms from the user text query, and linking the medical terms to corresponding entity IDs in the medical knowledge graph to obtain the query-related graph concept set; inputting the query-related graph concept set and the semantic feature vector of the user text query into a graph-driven text attention fusion module to obtain the graph-driven enhanced embedding vector of the user text query.

3. The atlas-driven intelligent medical image retrieval method according to claim 2, wherein Inputting the query-related graph concept set and the semantic feature vector of the user text query into a graph-driven text attention fusion module to obtain the graph-driven enhanced embedding vector of the user text query, including: performing semantic embedding encoding on each query-related graph concept in the query-related graph concept set to obtain a set of query-related graph concept semantic embedding encoding vectors; calculating cross-attention weights of the semantic feature vector of the user text query with respect to each query-related graph concept semantic embedding encoding vector in the set of query-related graph concept semantic embedding encoding vectors to obtain a set of cross-attention weights; fusing the semantic feature vector of the user text query and the set of query-related graph concept semantic embedding encoding vectors based on the set of cross-attention weights to obtain the graph-driven enhanced embedding vector of the user text query.

4. The atlas-driven intelligent medical image retrieval method according to claim 3, wherein Calculating the cross-attention weights of each query-related graph concept semantic embedding coding vector in the set of query-related graph concept semantic embedding coding vectors with respect to the semantic feature vector of the user text query to obtain a set of cross-attention weights, including: calculating the cross-attention weights using the following formula: ; where and are learnable weight matrices, is the semantic feature vector of the user text query, is each query-related graph concept semantic embedding coding vector in the set of query-related graph concept semantic embedding coding vectors, is the scaling factor, is the Softmax activation function, is each cross-attention weight in the set of cross-attention weights.

5. The atlas-driven intelligent medical image retrieval method according to claim 1, characterized in that, Each data item in the semantic index library includes a medical image ID, a medical image multi-modal semantic embedding vector, and an associated graph concept set.

6. The atlas-driven intelligent medical image retrieval method according to claim 5, wherein Inputting the graph-driven enhanced embedding vector of the user text query into a vector retrieval engine, and the vector retrieval engine extracts the top-K images most similar to the graph-driven enhanced embedding vector of the user text query from a semantic index library as a preliminary candidate list, including: the vector retrieval engine calculates the semantic similarity between the graph-driven enhanced embedding vector of the user text query and the medical image multi-modal semantic embedding vectors of each data item in the semantic index library respectively; extracting the preliminary candidate list from the semantic index library based on the sorting of the semantic similarity magnitudes.

7. The atlas-driven intelligent medical image retrieval method according to claim 6, characterized in that, The semantic similarity is the cosine similarity between the user text query graph-driven enhanced embedding vector and the medical image multimodal semantic embedding vector.

8. The atlas-driven intelligent medical image retrieval method according to claim 6, wherein The vector retrieval engine calculates the semantic similarity between the user text query graph-driven enhanced embedding vector and the medical image multimodal semantic embedding vectors of each data item in the semantic index library, including: calculating the vector quantile representations of the user text query graph-driven enhanced embedding vector and the medical image multimodal semantic embedding vector respectively to obtain the user text vector quantile representation value and the medical image vector quantile representation value; based on the user text vector quantile representation value and the medical image vector quantile representation value, through the quantile adjustment mechanism, changing the distribution forms of the user text query graph-driven enhanced embedding vector and the medical image multimodal semantic embedding vector by element-wise displacement operations to obtain the user text displacement parameter matrix and the medical image displacement parameter matrix; based on the user text displacement parameter matrix and the medical image displacement parameter matrix, alternately optimizing the user text query graph-driven enhanced embedding vector and the medical image multimodal semantic embedding vector respectively to obtain the user text query graph-driven enhanced embedding optimized vector and the medical image multimodal semantic embedding optimized vector; calculating the cosine similarity between the user text query graph-driven enhanced embedding optimized vector and the medical image multimodal semantic embedding optimized vector as the semantic similarity.

9. The atlas-driven intelligent medical image retrieval method according to claim 8, wherein Based on the query-related graph concept set, perform secondary graph concept filtering on the preliminary candidate list to obtain the sorted relevant medical image list, including: calculating the concept overlap degree between the query-related graph concept set and the associated graph concept sets of each data item in the preliminary candidate list; performing secondary graph concept filtering on the preliminary candidate list based on the sorting of the concept overlap sizes.

10. A spectrum-driven intelligent medical image retrieval system, characterized in that, Including: A medical knowledge graph construction module for constructing a medical knowledge graph; A user text query acquisition module for acquiring a user text query; A user text query encoding module for performing semantic enhanced embedding encoding on the user text query based on the medical knowledge graph to obtain a user text query graph-driven enhanced embedding vector and a query-related graph concept set; a preliminary candidate list generation module for inputting the user text query graph-driven enhanced embedding vector into a vector retrieval engine, and the vector retrieval engine extracts the top-K images most similar to the user text query graph-driven enhanced embedding vector from the semantic index library as the preliminary candidate list; A preliminary candidate list sorting module for performing secondary graph concept sorting on the preliminary candidate list based on the query-related graph concept set to obtain the sorted relevant medical image list; a relevant medical image list display module for displaying the sorted relevant medical image list.

Images