Medical image treatment method and system for multi-source heterogeneous data

Abstract

The invention discloses a medical image treatment method and system for multi-source heterogeneous data, and relates to the field of medical image treatment, and the method comprises the steps: firstly, respectively extracting an image embedding vector and a text embedding vector from original medical image data and a description text thereof through a deep learning model; then, the vectors of the two different modes are fused, and a unified semantic embedding vector is formed; based on the unified vector, through semantic similarity calculation with a standard term library, candidate mapping can be automatically generated, and dependence on rigid artificial rules is eliminated. More importantly, a closed-loop mechanism of manual auditing-feedback learning is introduced in the scheme, high-confidence mapping is adopted automatically, low-confidence mapping is audited by experts, and an auditing result is absorbed into a mapping knowledge base, so that the system has continuous learning and self-evolution capabilities, and a semantic gap of cross-mechanism data can be eliminated more intelligently and more accurately.

Description

Technical Field

[0001] This application relates to the field of medical image governance, and more specifically, to a method and system for medical image governance of multi-source heterogeneous data. Background Technology

[0002] With the rapid development of medical artificial intelligence and big data technologies, the cross-institutional aggregation and utilization of massive amounts of medical imaging data has become crucial for driving clinical research and AI model development. However, a serious challenge has emerged in the actual data fusion process: different medical institutions often have diverse naming conventions for the same examination item. For example, a standard chest CT scan may be recorded as chest CT, lung CT, or THORAX PLAIN in different systems. This semantic gap caused by multi-source heterogeneity seriously hinders the construction of large-scale, high-quality, standardized datasets and limits the development and generalization capabilities of downstream applications.

[0003] To address this challenge, existing technologies primarily rely on manually defined rules and mapping dictionaries for data governance. These methods typically hard-code non-standard terms to standard terms by establishing keyword matching or regular expression rules. However, this approach has significant limitations: First, it lacks scalability. Whenever a new data source is added or new, previously uncollected terms appear, substantial manpower is required to update and maintain the rule base, resulting in high costs and slow response times. More critically, these traditional methods simplify the complex semantic governance problem into a purely text matching problem, completely ignoring the most stable and richest contextual information inherent in medical images themselves. The image content, as the cornerstone of anchoring textual semantic descriptions, fails to effectively utilize its inherent visual features. Furthermore, existing solutions are mostly one-off, static cleaning processes, lacking feedback and self-learning mechanisms. They cannot continuously learn from expert corrections and new data to optimize system performance, resulting in a failure to evolve their intelligence over time.

[0004] In other words, existing technologies, when dealing with the semantic standardization of multi-source heterogeneous medical image data, have failed to effectively utilize images as a core semantic anchor and lack an adaptive framework capable of continuously learning from new data and expert feedback. Therefore, there is a pressing need for a technology that can deeply integrate multimodal information from images and text to truly achieve seamless and efficient fusion and governance of cross-institutional data. Summary of the Invention

[0005] To address the aforementioned problems in the existing technology, this application provides a medical image management method for multi-source heterogeneous data, which includes: acquiring raw medical image examination data to be processed.

[0006] Multimodal feature extraction and encoding are performed on the raw medical image examination data to be processed to obtain image embedding vectors and text embedding vectors.

[0007] Multimodal fusion of image embedding vectors and text embedding vectors is performed to obtain a unified semantic embedding vector.

[0008] Based on a standard terminology database, similarity-based candidate mappings are generated from the unified semantic embedding vectors to obtain candidate mappings, which include the original terms, recommended standard terms, and confidence scores.

[0009] If the confidence level of a candidate mapping is greater than or equal to the confidence threshold, the candidate mapping is stored in the mapping knowledge base.

[0010] In response to a candidate mapping having a confidence level less than a confidence threshold, the candidate mapping is pushed to the expert review interface, which includes options for confirmation, rejection, and correction.

[0011] Upon receiving confirmation from the expert review interface, the candidate mapping is stored in the mapping knowledge base.

[0012] This application also provides a medical image management system for multi-source heterogeneous data, which includes: a medical image examination data acquisition module for acquiring raw medical image examination data to be processed.

[0013] The medical image encoding module is used to extract and encode multimodal features from the raw medical image examination data to be processed, so as to obtain image embedding vectors and text embedding vectors.

[0014] The multimodal fusion module is used to perform multimodal fusion of image embedding vectors and text embedding vectors to obtain a unified semantic embedding vector.

[0015] The candidate mapping generation module is used to generate candidate mappings based on similarity from a standard terminology library and a unified semantic embedding vector. The candidate mappings include the original terms, recommended standard terms, and confidence scores.

[0016] The candidate mapping storage module is used to store candidate mappings into the mapping knowledge base when the confidence level of a candidate mapping is greater than or equal to the confidence threshold.

[0017] The candidate mapping push module is used to push the candidate mapping to the expert review interface in response to the confidence level of the candidate mapping being less than the confidence level threshold. The expert review interface includes confirmation, rejection and correction.

[0018] The confirmation module is used to store candidate mappings into the mapping knowledge base in response to receiving confirmation from the expert review interface.

[0019] Compared with existing technologies, this application provides a medical image governance method and system for multi-source heterogeneous data. To address the problems of existing solutions relying solely on text matching and lacking scalability and self-learning capabilities, this solution first extracts image embedding vectors and text embedding vectors from the original medical image data and its descriptive text using a deep learning model. Then, these two different modalities are fused to form a unified semantic embedding vector. This vector simultaneously carries the visual content of the image and the semantic information of the text, truly utilizing the value of the image as a semantic anchor and overcoming the limitations of pure text methods. Based on this unified vector, candidate mappings can be automatically generated through semantic similarity calculation with a standard terminology database, eliminating reliance on rigid manual rules. More importantly, the solution introduces a closed-loop mechanism of manual review and feedback learning: high-confidence mappings are automatically adopted, while low-confidence mappings are reviewed by experts. The review results are absorbed into the mapping knowledge base, enabling the system to continuously learn and evolve, thus more intelligently and accurately bridging the semantic gap between cross-institutional data. Attached Figure Description

[0020] The above and other objects, features and advantages of this application will become more apparent from the more detailed description of the embodiments of this application in conjunction with the accompanying drawings.

[0021] Figure 1 This is a flowchart of a medical image management method for multi-source heterogeneous data according to an embodiment of this application.

[0022] Figure 2 This is a schematic diagram of data flow in a medical image management method for multi-source heterogeneous data according to an embodiment of this application.

[0023] Figure 3 This is a flowchart of step 2 in the medical image management method for multi-source heterogeneous data according to an embodiment of this application.

[0024] Figure 4 This is a flowchart of step 4 in the medical image management method for multi-source heterogeneous data according to an embodiment of this application.

[0025] Figure 5 This is a block diagram of a medical image management system for multi-source heterogeneous data according to an embodiment of this application. Detailed Implementation

[0026] The embodiments of this application will now be described in more detail with reference to the accompanying drawings. It should be understood that the drawings and embodiments of this application are for illustrative purposes only and are not intended to limit the scope of protection of this application.

[0027] Based on the shortcomings in the aforementioned technical fields, this application proposes a medical image management method for multi-source heterogeneous data. Figure 1 This is a flowchart of a medical image management method for multi-source heterogeneous data according to an embodiment of this application. Figure 2 This is a schematic diagram of data flow in a medical image management method for multi-source heterogeneous data according to an embodiment of this application. Figure 1 and Figure 2 As shown, the medical image governance method for multi-source heterogeneous data according to an embodiment of this application includes: Step 1, acquiring the original medical image examination data to be processed; Step 2, performing multimodal feature extraction and encoding on the original medical image examination data to be processed to obtain image embedding vectors and text embedding vectors; Step 3, performing multimodal fusion on the image embedding vectors and text embedding vectors to obtain a unified semantic embedding vector; Step 4, generating candidate mappings based on similarity on the unified semantic embedding vectors based on a standard terminology database to obtain candidate mappings, the candidate mappings including original terms, recommended standard terms, and confidence levels; Step 5, in response to the confidence level in the candidate mapping being greater than or equal to a confidence threshold, storing the candidate mapping in a mapping knowledge base; Step 6, in response to the confidence level in the candidate mapping being less than a confidence threshold, pushing the candidate mapping to an expert review interface, the expert review interface including confirmation, rejection, and correction; Step 7, in response to receiving confirmation from the expert review interface, storing the candidate mapping in the mapping knowledge base.

[0028] In step 1, the raw medical imaging examination data to be processed is acquired. It should be understood that in a multi-source, heterogeneous medical data environment, medical imaging examination data generated by different institutions exhibit significant differences in naming and description, constituting an initial obstacle to data governance and fusion. To effectively bridge this semantic gap, these scattered and inconsistent raw data must first be used as the starting point for processing. Therefore, the step of acquiring the raw medical imaging examination data to be processed provides the basic data input containing complete image information and original text descriptions for subsequent multimodal feature extraction, encoding, and semantic fusion, which is the fundamental prerequisite for the implementation of the entire technical solution.

[0029] In one embodiment of step 1, the specific processing is as follows: This will be illustrated with a specific application scenario. In a scenario aiming to integrate imaging data from multiple hospitals for collaborative research, it is necessary to standardize the examination data uploaded by each hospital. The goal of this step is to securely and completely receive a set of raw medical imaging examination data from a specific hospital and structure it for subsequent processing.

[0030] The specific implementation process begins with establishing a secure data transmission link with the data source (e.g., the image archiving and communication server of a partner hospital). Through a pre-defined data interface protocol, a data retrieval command is invoked to retrieve a test record to be processed from the source database. This record is the raw medical imaging test data to be processed. Logically, this data is considered a single entity, but it contains two core components: the image volume data and the original descriptive text.

[0031] Taking a specific examination data point as an example, the acquired data is a head computed tomography (CT) scan. The image volume data is a DICOM file set conforming to digital imaging and communication standards. This dataset consists of a series of consecutive two-dimensional tomographic images, collectively forming a three-dimensional image volume. For example, this image volume might consist of 120 consecutive tomographic images with a matrix size of 512×512 pixels, each pixel recording the absorption attenuation value of the corresponding tissue location, with a slice thickness of 5 mm. This data completely preserves the anatomical information of the patient's examined area, forming the basis for subsequent image feature analysis.

[0032] Simultaneously, the original descriptive text is extracted from the metadata of the DICOM file. This text is stored in a specific data element label, such as the (0008,1030) inspection description field under the DICOM standard. In this example, the text content of this field is "Cranial CT". This text is the original record of the examination by the hospital technician or doctor, reflecting the institution's naming conventions. In other institutions, an examination of the same nature might be recorded as a plain head CT scan or a brain CT scan.

[0033] After acquisition, the raw medical imaging data is loaded into memory or intermediate storage, such as a cache or dedicated data processing queue, for efficient pipelined processing. The image volume data is organized into a three-dimensional numerical array. This process uses a dedicated DICOM parsing library to read the pixel matrix of each tomographic image layer by layer and stack them according to their spatial order, ultimately forming a tensor with dimensions of 120×512×512. This three-dimensional tensor then undergoes a series of preliminary preprocessing steps to make its numerical range suitable for subsequent deep learning model input. This preprocessing is particularly crucial for CT images, starting with window width and level adjustments. Given the extremely wide range of Henlein (HU) values ​​in raw CT images, and the different optimal display ranges for different human tissues (such as bone, soft tissue, and lung windows), specific window width and level values ​​need to be set to highlight the details of the target tissue. These specific window width and level parameters are precisely selected based on mature clinical application standards and the specific range of radioactivity in Henlein units specific to the target anatomical structure or pathological tissue being observed. For example, for cranial CT scans, a window level of 40 HU and a window width of 80 HU can be set to linearly map the original HU values ​​to a new grayscale range, thereby enhancing the contrast between the brain parenchyma and surrounding structures. Next, the image data after window level and width adjustment is normalized, scaling the pixel values ​​to a fixed range more suitable for deep learning model training, such as between [-1, 1]. Each pixel value V is transformed into V' by calculating the maximum and minimum values ​​of the current data batch, or using globally preset statistical values. This transformation can be expressed as V' = (V - Vmin) / (Vmax - Vmin). This normalization operation helps accelerate the model's convergence process and improve its stability. Simultaneously, the extracted original descriptive text, i.e., the string "Cranial CT," is saved as an independent metadata field and uniquely associated with the processed 3D image tensor, for example, by sharing a unique examination identifier. This association ensures that in the subsequent multimodal fusion stage, the text information can be accurately paired with the image content it describes and stored separately.

[0034] In step 2, multimodal feature extraction and encoding are performed on the raw medical image examination data to obtain image embedding vectors and text embedding vectors. Correspondingly, the raw medical image examination data consists of two distinct forms of information: one is the image volume data represented by a high-dimensional pixel / voxel matrix, which contains rich anatomical structures and pathological features; the other is the raw descriptive text existing as a string, which carries semantic information at the human language level. These two types of data belong to different information spaces in their original form and cannot be directly compared or fused at the semantic level. To enable the machine to understand the intrinsic relationship between the text of a cranial CT scan and the corresponding three-dimensional head scan image, they must first be transformed into the same unified, measurable mathematical space. Therefore, the step of performing multimodal feature extraction and encoding on the raw medical image examination data is to map these heterogeneous, unstructured raw data into low-dimensional, dense, and semantically rich numerical vectors, i.e., embedding vectors.

[0035] Figure 3 This is a flowchart of step 2 in the medical image management method for multi-source heterogeneous data according to an embodiment of this application. Figure 3 As shown, in one embodiment of step 2, multimodal feature extraction and encoding are performed on the raw medical image examination data to be processed to obtain image embedding vectors and text embedding vectors, including: step 21, extracting image volume data and original descriptive text from the raw medical image examination data to be processed; step 22, extracting image embedding vectors from image volume data; and step 23, extracting text embedding vectors from original descriptive text.

[0036] In the above implementation, step 2 is specifically processed as follows: First, step 21 is executed, which is a logical data distribution process. It formally designates the preprocessed 3D image tensor as the image volume data input for this stage from the data passed from the previous step, while designating the string "Cranial CT" as the original descriptive text input. This operation ensures that the subsequent two parallel processing branches (steps 22 and 23) can obtain clear and explicit data sources for their respective processing, preparing for specialized feature extraction.

[0037] Next, steps 22 and 23 are executed in parallel. In step 22, the operation of extracting image embedding vectors from the image volume data is performed. In one embodiment of step 22, extracting image embedding vectors from the image volume data includes: inputting the image volume data into a pre-trained 3D convolutional neural network to obtain image embedding vectors. The core of this process is to utilize a powerful deep learning model to understand and compress image information. Specifically, the image volume data prepared in the previous step, with dimensions of 120×512×512, is input into a pre-trained 3D convolutional neural network. Here, pre-training means that the parameters of the neural network (i.e., the connection weights and biases between neurons in each layer of the network) are not randomly initialized, but are pre-trained on a large-scale, diverse, general medical image dataset (e.g., a public or private 3D medical image dataset containing hundreds of thousands of cases of different parts and modalities). Through this pre-training process, the network has learned universal visual features from low to high levels, such as edges, textures, basic shapes, and even specific anatomical structure patterns. This ability to transfer knowledge allows the network to have an excellent starting point for feature extraction when processing new, specific image data. In this embodiment, a 3D convolutional neural network architecture based on the residual network concept, such as 3DResNet-50, can be selected. This network consists of a series of stacked residual blocks. Its input layer receives a 120×512×512 three-dimensional image tensor. The data first passes through an initial 3D convolutional layer, for example, using a 7×7×7 convolutional kernel and a 3D max pooling layer, for preliminary feature extraction and size reduction. Subsequently, the data flows through four stages of residual modules, each stage containing multiple residual blocks. A typical residual block consists of several 3D convolutional layers, batch normalization layers, and non-linear activation functions (such as ReLU). Its core design lies in introducing a "shortcut connection," which directly adds the block's input to the block's output. This structure greatly alleviates the gradient vanishing problem in deep network training, allowing the network to be built very deep, thereby learning more complex and abstract features. As data is processed deeper into the network, its spatial dimension gradually decreases through convolution or pooling operations with a stride greater than 1, while the number of feature channels increases accordingly (e.g., from 64 to 2048). This means the network gradually shifts its focus from local details to capturing global, more semantic macroscopic features. After the last residual block, a global average pooling layer is connected. This layer averages the values ​​at all spatial locations of each feature map, compressing a high-dimensional feature map tensor (e.g., 4×16×16×2048) into a one-dimensional feature vector with a dimension of 2048. To obtain the embedding vector of the desired dimension, a fully connected layer can be appended to this vector, mapping it to a preset dimension, such as 512, through linear and non-linear transformations.Therefore, when the three-dimensional image tensor corresponding to the cranial CT scan is forward-propagated through this pre-trained 3D ResNet-50 network, it ultimately outputs a 512-dimensional floating-point vector. This vector is the image embedding vector, which is a compact and precise mathematical representation of the original image volume data in a low-dimensional semantic space. Each dimension of the vector represents a high-order abstract feature learned from the image.

[0038] In step 23, the operation of extracting text embedding vectors from the original descriptive text is performed synchronously. This process aims to convert human language into a numerical form that machines can understand. In one embodiment of step 23, extracting text embedding vectors from the original descriptive text includes: inputting the original descriptive text into a pre-trained language model to obtain text embedding vectors. Similar to the image model, the language model here is also pre-trained. It is trained on a huge, general text corpus (such as Wikipedia, publicly available web page text) and possibly further on domain-specific text corpora (such as massive amounts of medical literature, clinical reports). Through self-supervised learning tasks such as masked language modeling or next-sentence prediction, the model learns to deeply understand the grammatical structure of the language, the semantic relationships between words, and contextual dependencies. In this embodiment, a pre-trained language model based on the Transformer architecture, such as the BERT model, can be selected. The input processing flow of this model is quite sophisticated. First, the input string "Cranial CT" is segmented into words. For Chinese, this is usually based on the granularity of characters, and the resulting word segmentation sequence is ["skull", "brain", "C", "T"]. Next, special control characters are added to the beginning and end of the sequence, such as adding [CLS] at the beginning and [SEP] at the end, forming the final input sequence: [[CLS], "skull", "brain", "C", "T", [SEP]]. The model then uses an embedding layer to map each character in the sequence (including the special control characters) to an initial numerical vector. To allow the model to perceive word order information, a unique positional encoding vector is superimposed on the vector at each position. The preprocessed vector sequence is then fed into the main body of the BERT model, a structure composed of multiple layers (e.g., the BERT-base model contains 12 layers) of transformer encoders. Each encoder layer contains a multi-head self-attention mechanism and a feedforward neural network. The multi-head self-attention mechanism is its core, allowing the model to dynamically calculate and pay attention to the importance of all other characters in the sequence (including itself) when processing each character, thereby capturing long-distance, bidirectional contextual dependencies. For example, when processing the word "brain," the model can simultaneously associate it with the preceding "skull" and the following "CT," thus understanding the overall meaning of the compound word "skull-brain CT." After complex transformations through multiple encoders, the model generates an output vector that deeply integrates contextual information for each position in the input sequence. To obtain a single semantic vector representing the entire input text, a fixed strategy is adopted: the final output vector corresponding to the [CLS] control character is used. This vector is designed to aggregate the semantic information of the entire sequence. For the BERT model, its output vector has a dimension of 768. Therefore, when the string "skull-brain CT" is processed by this pre-trained language model, the final output is a 768-dimensional floating-point vector.This vector is the text embedding vector, which numerically captures the precise medical semantics of the phrase "cranial CT".

[0039] In step 3, the image embedding vector and text embedding vector are fused using a multimodal method to obtain a unified semantic embedding vector. It is understandable that although the preceding steps have successfully transformed high-dimensional image data and discrete text symbols into structured image embedding vectors and text embedding vectors, respectively, these two types of vectors remain independent entities existing in different feature spaces and describing different modalities. They each carry rich semantic information, but this information is partial: image embedding vectors lack explicit linguistic labels, while text embedding vectors are detached from the anchoring of the specific visual content they refer to. To accurately determine whether a set of cranial CT scan data is equivalent to a plain head CT scan in a standard terminology database, it is not possible to compare images or text in isolation, but rather to comprehensively consider the deep semantic meaning they both point to. Therefore, performing a multimodal fusion of the image embedding vector and text embedding vector can construct a unified, shared semantic space, and generate a single, comprehensive, unified semantic embedding vector within this space that can simultaneously capture the visual features of the image and the descriptive connotations of the text, laying a solid foundation for subsequent accurate semantic matching and mapping.

[0040] In one embodiment of step 3, multimodal fusion of the image embedding vector and the text embedding vector is performed to obtain a unified semantic embedding vector, including: step 31, performing feature concatenation on the image embedding vector and the text embedding vector to obtain an image-text concatenated embedding vector; step 32, inputting the image-text concatenated embedding vector into one or more fully connected layers for dimensionality reduction and nonlinear transformation to obtain a unified semantic embedding vector.

[0041] In the above implementation, step 3 is specifically processed as follows: First, step 31, i.e., feature concatenation, is performed. This is a direct and efficient early fusion strategy. Specifically, the 512-dimensional image embedding vector and the 768-dimensional text embedding vector are concatenated end-to-end in a predetermined order to form a longer new vector containing all the original feature information of both modalities. For example, the image embedding vector can be placed first, and the text embedding vector can be placed last. Numerically, this operation is equivalent to creating a new vector whose first 512 elements come entirely from the image embedding vector, and the following 768 elements come entirely from the text embedding vector. After this operation, a vector with dimensions of 512 plus 768, i.e., 1280 dimensions, is obtained. This vector is called the image-text concatenated embedding vector.

[0042] While the image-text concatenated embedding vector formally completes the information aggregation, it merely juxtaposes the features of the two modalities; the deeper, non-linear interactions between them remain unexplored. Therefore, step 32, a crucial step in the multimodal fusion process, is executed. The 1280-dimensional image-text concatenated embedding vector generated in the previous step is input into a neural network module consisting of one or more fully connected layers for deep feature interaction, dimensionality reduction, and non-linear transformation. This fully connected layer, also known as a dense layer, is characterized by each neuron being connected to all neurons in the previous layer. Each connection corresponds to a learnable weight parameter, and each neuron also has a learnable bias parameter. In this embodiment, the module can be designed as a multilayer perceptron structure containing two fully connected layers. The first fully connected layer, acting as a hidden layer, learns the complex interaction patterns between the two modal features and projects the concatenated high-dimensional features into a lower-dimensional intermediate feature space. The second fully connected layer, acting as the output layer, further refines and integrates the intermediate features, ultimately outputting them to a unified semantic space of the target dimension. The specific calculation process is as follows: Let the input 1280-dimensional image-text concatenation embedding vector be X. When X is fed into the first fully connected layer, it first undergoes matrix multiplication with the weight matrix W1 of that layer. The dimension of the weight matrix W1 is set to 1024×1280, where 1024 is the number of neurons in the hidden layer. This is a hyperparameter that needs to be selected through experiments and validation, aiming to balance the model's expressive power and computational complexity. The result of the matrix multiplication is a 1024-dimensional vector. Subsequently, a 1024-dimensional bias vector b1 is added to this result vector. The mathematical expression for this process is Z1=W1*X+b1. Next, in order to introduce nonlinearity and enable the neural network module to learn and fit more complex dependencies between images and text, Z1 needs to be processed through a nonlinear activation function. Here, the Corrected Linear Unit (ReLU) function can be used, defined as a(z)=max(0,z). After processing by the activation function, the output H1 of the first hidden layer is obtained, with a dimension of 1024. Subsequently, the 1024-dimensional hidden layer output vector H1 is fed as input into the second fully connected layer (i.e., the output layer). This layer performs a similar operation. H1 is multiplied by the layer's weight matrix W2, whose dimension is set to 512 multiplied by 1024, where 512 is the pre-defined target dimension of the final unified semantic embedding vector. The result is then added to a 512-dimensional bias vector b2 to obtain the final output vector Y, mathematically expressed as Y = W2 * H1 + b2. In the output layer, no activation function or a linear activation function is used to allow the dimensions of the embedding vector to take values ​​freely in the real number domain, thus preserving a richer semantic expression space. The final 512-dimensional vector Y is the unified semantic embedding vector.It's worth noting that the weight matrices W1 and W2, and the bias vectors b1 and b2, are not fixed but learned through end-to-end training on a dataset containing a large amount of triplet-annotated data (images, text, and standard terminology). The training goal is to ensure that, after the entire process (from feature extraction to multimodal fusion), the unified semantic embedding vectors generated from semantically similar or identical (image, text) pairs are as close as possible in the vector space, while the unified semantic embedding vectors of semantically different pairs are as far apart as possible. By employing metric learning methods such as contrastive loss or triplet loss, and utilizing backpropagation and gradient descent optimizers (such as the Adam optimizer), these parameters can be iteratively adjusted until the model can generate unified embedding vectors with excellent semantic discriminative power. Therefore, when the 1280-dimensional stitched vector from cranial CT scans passes through this well-trained two-layer fully connected network module, the resulting 512-dimensional vector is no longer a simple stacking of image and text features. It is a highly condensed coordinate point in a unified semantic space, whose position reflects the visual commonality of head CT images and also encodes the specific semantics of the text description of cranial CT.

[0043] In step 4, based on a standard terminology database, candidate mappings are generated from the unified semantic embedding vector using similarity-based methods to obtain candidate mappings. These candidate mappings include the original terminology, recommended standard terms, and confidence levels. It should be understood that after the deep fusion of multimodal information, a unified embedding vector capable of comprehensively representing the core semantics of the original medical imaging examination data has been successfully obtained. However, this vector itself is still an abstract coordinate in a high-dimensional mathematical space and has not yet established a clear correspondence with any specific, universally accepted standard terminology. The ultimate goal of data governance is to converge various non-standard expressions into a standardized vocabulary system. To achieve this goal, a mechanism is needed that can use this unified semantic embedding vector to find the landmark in a pre-established standard terminology map that is semantically closest to it. Therefore, performing candidate mapping generation based on a standard terminology database transforms the abstract semantic vector into a concrete, interpretable mapping suggestion with confidence assessment, providing a decision-making basis for subsequent automated adoption or expert review processes. This is a crucial step connecting semantic understanding with actual governance actions.

[0044] Figure 4 This is a flowchart of step 4 in the medical image processing method for multi-source heterogeneous data according to an embodiment of this application. Figure 4As shown, in one embodiment of step 4, based on a standard terminology library, a similarity-based candidate mapping is generated for the unified semantic embedding vector to obtain a candidate mapping, including: step 41, calculating the semantic matching score between the unified semantic embedding vector and each standard term embedding vector in the standard terminology library; step 42, taking the standard term with the highest semantic matching score as the recommended standard term in the candidate mapping and taking the semantic matching score as the confidence level in the candidate mapping.

[0045] In the above implementation, step 4 is specifically processed as follows: Before executing step 41, a key external resource, namely a standard terminology library, needs to be constructed and prepared. This library is the benchmark and target set for semantic mapping in this application. It is not a simple list of strings, but a carefully constructed knowledge base containing standardized terms and their corresponding embedding vectors. The construction process of this library is as follows: First, an authoritative and comprehensive list of standard medical imaging examination terms is collected and organized. These terms are derived from international or domestic clinical guidelines, radiology terminology sets, or industry consensus. For example, it contains thousands of standardized terms such as head CT plain scan, chest CT enhanced scan, and abdominal MRI plain scan + enhanced scan. Subsequently, to ensure the consistency of the semantic space, the embedding vector of each standard term in the library is generated in a manner compatible with the aforementioned steps. Specifically, each standard term string, such as head CT plain scan, is input into the same pre-trained language model used in step 23, namely the BERT model, and after the same processing flow, its 768-dimensional text embedding vector is generated. To enable comparison with the 512-dimensional unified semantic embedding vectors in the same space, these 768-dimensional text embedding vectors need to be passed through a dedicated mapping network. Its architecture can be similar to some of the networks in step 32, such as a fully connected layer that maps from 768 dimensions to 512 dimensions, projecting it onto the final 512-dimensional unified semantic space. The parameters of this mapping network are optimized along with the parameters of the multimodal fusion network during end-to-end training. Finally, the standard terminology corpus is stored in key-value pairs, where the key is the string of the standard term and the value is its corresponding pre-computed 512-dimensional embedding vector. Step 41, the execution phase, quantifies the semantic similarity between the data to be processed and each standard term. The input to this process is a 512-dimensional unified semantic embedding vector representing a cranial CT scan, denoted as Vu. During execution, a traversal operation is initiated, sequentially reading each entry in the standard terminology corpus. For the i-th standard term in the corpus, denoted as Si, its pre-stored 512-dimensional embedding vector is retrieved, denoted as Vi. Next, the semantic matching score between Vu and Vi is calculated. In one embodiment of step 41, the semantic matching score between the unified semantic embedding vector and each standard term embedding vector in the standard term library is calculated, including: calculating the cosine similarity between the unified semantic embedding vector and each standard term embedding vector in the standard term library as the semantic matching score. Cosine similarity is an index that measures the similarity between two vectors in a direction. Its value range is [-1, 1]. The closer the value is to 1, the more consistent the directions of the two vectors are, that is, the more similar the semantics they represent. Its calculation formula is as follows: Semantic matching score (Vu, Vi) = Cosine similarity = (Vu·Vi) / (||Vu||*||Vi||), where (Vu·Vi) represents the dot product of the two vectors, that is, the elements in the corresponding dimensions are multiplied one by one and then summed.||Vu|| and ||Vi|| represent the L2 norm (or Euclidean length) of two vectors, respectively, which is the square root of the sum of the squares of each element in the vector. Taking a specific numerical example, let Vu be a 512-dimensional vector representing a cranial CT scan. When traversing to the head CT scan S1 in the standard terminology database, its corresponding vector V1 is extracted, and the cosine similarity between the two is calculated. Due to the high semantic correlation, the score might be 0.98. When traversing to the chest CT scan S2, its vector V2 differs significantly from Vu semantically, and the calculated score might only be 0.25. When traversing to the cervical spine CT scan S3, due to the proximity of the anatomical location and the same modality, its vector V3 might have some similarity to Vu, and the score might be 0.68. This calculation process will be performed on all thousands of terms in the standard terminology database one by one, ultimately generating a score list of the same length as the standard terminology database.

[0046] Next, step 42 is executed. In practice, the score list is first searched to find the maximum value. In this example, the maximum score is 0.98. Then, the standard term corresponding to this maximum score is determined, namely, plain head CT scan. Finally, a candidate mapping data object is generated according to a preset structure. This object contains the following three fields: 1. Original term: Directly refers to the original input text of this processing flow, i.e., cranial CT. 2. Recommended standard term: Filled with the standard term found with the highest matching score, i.e., plain head CT scan. 3. Confidence: Filled with the corresponding highest semantic matching score value, i.e., 0.98. The final output is a structured candidate mapping, such as a data object containing {Original term: "cranial CT", Recommended standard term: "plain head CT scan", Confidence: 0.98}. This output clearly indicates that for the original term "cranial CT", through deep multimodal semantic analysis, the most likely standardized result is plain head CT scan, and the confidence in this judgment is very high.

[0047] In step 5, in response to a confidence level greater than or equal to a confidence threshold in a candidate mapping, the candidate mapping is stored in the mapping knowledge base. Accordingly, the preceding steps have successfully generated a candidate mapping containing recommended standard terminology and quantified confidence levels for each medical imaging examination data to be processed. However, this candidate mapping is essentially a suggestion based on model calculations, and its reliability varies. When building a large-scale, high-precision mapping knowledge base, both efficiency and quality must be balanced. Submitting all suggestions, regardless of quality, to medical experts for review would incur enormous human costs, violating the initial intention of automated governance; conversely, accepting all suggestions wholesale might introduce erroneous mappings, contaminating the knowledge base. Therefore, the step of storing candidate mappings in the knowledge base when their confidence level is greater than or equal to a specific threshold aims to efficiently and reliably identify and adopt those mappings that the model is highly confident in, achieving a high degree of automation in the governance process, while simultaneously diverting mappings with uncertainties, paving the way for subsequent manual intervention.

[0048] In one embodiment of step 5, the specific processing is as follows: Before implementation, a crucial parameter needs to be pre-set, namely the confidence threshold. This threshold is a floating-point value ranging from 0 to 1, playing the role of a watershed between high-confidence and low-confidence mappings. The setting of this threshold is scientifically determined through rigorous experiments and evaluation processes. Specifically, an independent validation dataset with precise human annotations is prepared. This dataset contains a large number of original terms and their unique corresponding standard terms. The first four steps of this invention are run through the data in this validation set to calculate a confidence score for each data point. Subsequently, by analyzing the relationship between these scores and the true labels (i.e., whether the human-annotated mapping is correct), a precision-recall curve can be plotted, for example. Precision refers to the proportion of truly correct mappings among all mappings judged as correct by the model; recall is the proportion of mappings successfully retrieved by the model among all actually correct mappings. The selection of the confidence threshold is essentially a trade-off between precision and recall. A very high threshold (e.g., 0.99) ensures extremely high precision, meaning almost all mappings automatically stored in the knowledge base are correct. However, this can lead to a decrease in recall, where many equally correct mappings with slightly lower model confidence are mistakenly rejected and sent to manual review. Conversely, a lower threshold improves recall but may sacrifice precision. Therefore, the threshold is selected based on specific business needs. For example, it might be the confidence score corresponding to the highest recall point when precision reaches 99.5%, or the point that maximizes the harmonic mean (F1 score) of precision and recall. In this embodiment, after the above evaluation process, the confidence threshold is set to 0.95.

[0049] Once a candidate mapping is received, the execution logic for this step officially begins. First, the confidence field value, 0.98, is extracted from the candidate mapping. Then, this value is compared with a preset confidence threshold of 0.95. The condition for judgment is that the confidence level is greater than or equal to the confidence threshold. In this example, 0.98 is greater than 0.95, therefore the condition is met.

[0050] With the conditions met, the next core execution action is triggered: storing the candidate mapping in the mapping knowledge base. This knowledge base is a structured, persistent data storage entity, such as a specific data table in a relational database or a collection in a document database. It is not only the final result of this governance task, but also a dynamically evolving and continuously accumulating living knowledge base with dual functions: first, as an efficient query cache, when the same original term "craniocerebral CT" is encountered again in the future, the verified mapping results can be retrieved directly from the database, avoiding repetitive and computationally intensive feature extraction and fusion processes; second, as a valuable training data source, after accumulating a certain number, these high-quality mapping relationships can be used for incremental training or periodic retraining of the model, thereby continuously improving the model's performance and generalization ability.

[0051] The process of storing data in the knowledge base involves creating a new record or document. This record is carefully structured to store complete mapping information. For example, a table named MappingKnowledge might contain the following fields: OriginalTerm, StandardTerm, ConfidenceScore, Source, and CreationTimestamp. Specifically, an insert command is executed, filling the corresponding fields with information from the candidate mapping: the OriginalTerm field is assigned the value "Cranial CT". This field is indexed to support fast lookups. The StandardTerm field is assigned the value "Cranial CT Scan". The ConfidenceScore field is assigned a value of 0.98. The Source field is assigned a predefined identifier, such as "Auto-Generated - High Confidence", to clearly distinguish this record as being generated through this automated process, differentiating it from records that may be subsequently confirmed or corrected by experts. The CreationTimestamp field records the timestamp of the current operation.

[0052] Once this record is successfully written to and submitted to the database, the semantic governance process for this original brain CT scan medical imaging data is complete. The successful change of state in the mapping knowledge base means a new mapping rule has been added and validated with high confidence. Afterward, the governance process will wait for and receive the next piece of raw medical imaging data to be processed, repeating the entire process.

[0053] In step 6, in response to a candidate mapping having a confidence level lower than a confidence threshold, the candidate mapping is pushed to the expert review interface, which includes options for confirmation, rejection, and correction. It is understandable that in automated data governance processes, although a large number of highly reliable semantic mappings are automatically adopted through the confidence threshold, there will inevitably be some candidate mappings whose corresponding confidence levels fail to meet the preset standard representing certainty. These low-confidence mappings often originate from more complex or ambiguous scenarios, such as rare abbreviations, ambiguous expressions, or uncertainty exhibited by the model in newly encountered data patterns. Simply discarding these mappings would not only result in information loss but also miss opportunities to improve the model's cognitive boundaries and ability to handle complex scenarios. Therefore, pushing candidate mappings to the expert review interface in response to a confidence level lower than the threshold establishes a crucial human-machine collaboration link. This allows medical experts with deep domain knowledge to make the final decision on judgments that the machine cannot yet be certain of. This serves as the last line of defense for ensuring the accuracy of the entire mapping knowledge base and is also a necessary pathway to building an adaptive governance framework that can continuously learn and evolve from expert knowledge.

[0054] In one embodiment of step 6, the specific processing is as follows: The execution logic of this step complements that of step 5. It specifically handles candidate maps that failed the high-confidence automated screening. Its input is also the candidate map data object generated in step four. For illustration, a new processing example is introduced here. For instance, a raw medical imaging examination data to be processed, whose original descriptive text is "head MR scan," after processing in the preceding steps, generates the following candidate map: {Original terminology: "head MR scan," recommended standard terminology: "plain head MRI scan," confidence level: 0.85}.

[0055] This step begins with a conditional check. First, the confidence score field value, 0.85, is extracted from the candidate mapping. Then, this value is compared to a pre-defined confidence threshold of 0.95. The condition is that the confidence score is less than the threshold. In this example, 0.85 is less than 0.95, and the condition is met. Therefore, the governance process proceeds to the expert review path defined in this step.

[0056] Once the conditions are triggered, the core action is to push the candidate mappings to the expert review interface. This push is not simply data display; it's a process of structuring the tasks to be reviewed and storing them in a specific task queue. Specifically, a task record to be reviewed is generated and written to a dedicated database table or message queue for consumption by the expert review interface application. This task record contains all the contextual information needed to make an accurate judgment, such as: a unique task identifier, the complete content of the candidate mapping to be reviewed (original terminology, recommended standard terminology, confidence level), a unique identifier for the original medical imaging examination data for backtracking the original images and text, a timestamp of task generation, and an initial state such as pending review. In this example, a new task containing information such as {original terminology: "head MR scan", recommended standard terminology: "plain head MRI", confidence level: 0.85} is created and placed at the head of the review queue.

[0057] The expert review interface is an interactive, visual software application designed for qualified medical experts (such as radiologists or clinical data scientists), presented in the form of a web application. When an expert logs in, the interface pulls and displays a list of tasks awaiting review from the task queue. After selecting a task, the expert is taken to a carefully designed review details page. This page is divided into several functional areas. One area is dedicated to displaying raw data information, clearly presenting the raw terminology of a head MRI scan, and embedding a professional medical image browser, allowing experts to interactively view the raw image volume data associated with the term. This includes zooming, panning, adjusting window width and level, or switching between different sequences to fully understand the image content. The core area of ​​the page presents the candidate mappings generated by the model. It clearly displays the standard terminology recommended by the model, i.e., a head MRI plain scan, along with its confidence score of 0.85, allowing experts to intuitively understand the model's confidence level in this recommendation. To provide richer decision support, this area may also list several other candidate standard terms with the next highest scores and their scores, revealing the specific objects of the model's hesitation. The most crucial part is the interactive area, which offers three clear options for experts to make a final decision based on their expertise: Confirm, Reject, and Correct. The Confirm option corresponds to a button. If, after reviewing the images and text, the expert determines that the model-recommended head MRI plain scan is the most accurate and appropriate standard terminology for the current scenario, they will click this button. The Reject option also corresponds to a button. In some cases, such as when the raw data quality is extremely poor, or the original terminology is so vague that no meaningful mapping can be made, the expert may choose to reject it. This action will mark the task as rejected or unmappable and remove it from the active review queue. The Correct option provides a more flexible and valuable feature for knowledge contribution. It consists of an input box that supports search and auto-completion, along with a submit button. If the expert believes the model-recommended terminology is incorrect, but a more suitable standard term exists, this function can be activated. For example, after reviewing the head MR scan images, the expert might find that it is a specific set of T1-weighted sequence images, while the standard terminology library contains a more precise term: head MRI T1 sequence plain scan. At this point, the expert can enter keywords in the correction input box, select the head MRI T1 sequence plain scan as the correct mapping target from the pop-up standard terminology library list, and then submit the correction.

[0058] In step 7, in response to receiving confirmation from the expert review interface, the candidate mapping is stored in the mapping knowledge base. That is, in the human-machine collaborative review process, when a medical expert reviews a low-confidence candidate mapping and ultimately presses the confirmation button, this action represents the highest level of authority in the field recognizing the model's initial recommendation. This confirmation signal from a human expert elevates an initially uncertain, algorithm-generated hypothesis into a verified, high-quality fact. If this valuable, verified knowledge is not solidified, the expert's intellectual input will only be effective for a single data governance task, unable to be reused or used to improve subsequent automated processes. Therefore, storing the mapping in the knowledge base upon receiving confirmation from the expert review interface captures and permanently stores the expert's tacit knowledge, ensuring that each manually confirmed mapping relationship enriches and optimizes the core knowledge base in real time, thereby achieving effective knowledge accumulation and continuous learning and self-improvement of the entire governance framework.

[0059] In one embodiment of step 7, the specific processing is as follows: For detailed explanation, this embodiment will continue the scenario of step 6. In the previous step, the candidate mapping {original terminology: "head MR scan", recommended standard terminology: "plain head MRI", confidence level: 0.85} was pushed to the expert review interface because its confidence level was lower than the threshold of 0.95, and a task to be reviewed was generated. If a radiology expert reviews the detailed information of the task, including viewing the corresponding head MR scan image sequence, and determines that the head MRI plain scan recommended by the model is indeed the most appropriate standardized description of the examination data, then the expert clicks the confirmation button on the task review interface.

[0060] This click action marks the starting point of this step. Technically, the front-end application of the expert review interface captures this button click event. This event triggers a predefined function that constructs and sends a request to the back-end application server. This request must contain two key pieces of information: first, a unique identifier for the review task, used to precisely locate which item to be reviewed has been processed; and second, the type of operation performed, i.e., confirmation. Sending this request constitutes the process of receiving confirmation from the expert review interface.

[0061] On the backend application server, a dedicated listening service or interface (API endpoint) continuously waits for such requests. Upon receiving the aforementioned confirmation request, the backend logic for this step is activated. First, based on the task identifier carried in the request, the corresponding task record is queried and locked in the pending task queue or database. From this record, the program will completely read the candidate mapping information associated with the task, i.e., {original terminology: "head MR scan", recommended standard terminology: "cranial MRI scan"}.

[0062] After obtaining the expert-confirmed mapping relationship, the next core action is the same as the storage operation in step 5 in terms of goal: storing the candidate mappings into the mapping knowledge base. However, there are differences in details to reflect the authority of its source. The program will construct a new record, ready to insert it into the knowledge base data table named MappingKnowledge. The fields of this record are assigned the following values: OriginalTerm field: assigned to head MR scan. StandardTerm field: assigned to head MRI plain scan. ConfidenceScore field: The original confidence score of 0.85 given by the model is no longer used here because the mapping has been confirmed by experts, and its reliability has reached the highest level. Therefore, this field can be assigned a specific value representing complete confidence, such as 1.0. Source field: This is the key difference from step 5. This field will be assigned a clear identifier, such as expert review-confirmation. This label is crucial; it records the lineage of the knowledge, indicating that it is high-quality data endorsed by human experts. In future model iterations and optimizations, when high-quality training samples need to be selected, this data with expert review labels will be the preferred and most valuable training corpus. The CreationTimestamp field records the precise time the current operation was executed. After the record is constructed, a database insertion operation is performed to persist this new, highly reliable mapping relationship to the mapping knowledge base.

[0063] After successfully storing the data in the knowledge base, a final step is required to maintain the integrity and consistency of the entire process. Specifically, the task's status in the review queue is updated again based on the task identifier, changing it from "pending review" to "confirmed." This status change ensures that the task is not repeatedly pushed to other experts for review.

[0064] Furthermore, when a medical expert rejects a candidate mapping after reviewing it, this conveys a crucial negative feedback: the model not only failed to find the correct mapping, but its recommended terminology is also considered completely incorrect by the expert, and there may not be a clear alternative. This decision occurs when the original data itself is severely ambiguous or of poor quality, making meaningful standardized mapping impossible. Simply ignoring or discarding this rejection signal deprives the entire governance approach of a valuable learning opportunity—learning the boundaries of its own capabilities and identifying which types of inputs are unknowable or unreliable. Therefore, the core purpose of a process for responding to rejection signals is to capture and record these failure cases, building a negative sample knowledge base to improve the robustness and accuracy of future model iterations, teaching the model not only what it is, but also what it "is not."

[0065] Specifically, when an expert clicks the "Reject" button on a review task (e.g., for a head scan with a specific sequence of original terminology), the front-end application of the expert review interface sends a request to the back-end server containing the task's unique identifier and the operation type "Reject." Upon receiving this request, the back-end initiates the following process: First, it locates the review task based on the task identifier and reads its complete context information, including the original term "head scan with a specific sequence of original terminology," the model's previously recommended standard terminology such as a plain head MRI scan, and the corresponding original image data identifier. Next, the core action is not to store the information in the main mapping knowledge base, but rather in a specially established, structured negative sample library or anomaly log. This library is used to systematically record all mapping attempts rejected by experts. A new record is created in this library, containing the following fields: original terminology assignment "head scan with a specific sequence of original terminology," model-recommended terminology assignment "plain head MRI scan," original image identifier, expert action "Reject," and a timestamp. This record provides the model with an explicit negative constraint. This data from the negative sample library will be utilized in subsequent model retraining or incremental learning phases. For example, when training using contrastive learning or a triplet loss function, the unified semantic embedding vector generated from a specific sequence of head scans and their images can be paired with the embedding vector from a plain head MRI scan as a negative pair. The penalty term in the loss function forces the model to widen the distance between these two pairs in the vector space. By learning from a large number of such negative samples, the model can more accurately characterize the boundaries between different semantic concepts, thereby reducing the probability of making similar errors in future predictions. Finally, similar to handling confirmation signals, the status of the review task in the queue will be updated to rejected.

[0066] The correction signal is the most information-rich feedback in the human-machine collaboration process. It occurs when the model's recommendation is inaccurate, but the expert can provide a clear and correct answer. This signal directly provides a high-quality, expert-annotated (input data, correct output) data pair. This is invaluable nourishment for an intelligent governance method aimed at continuous learning and evolution. Without utilizing it, the system's cognitive level will stagnate. Therefore, the fundamental purpose of a process that responds to correction signals is to establish a dynamic, self-learning closed-loop feedback mechanism, instantly transforming each expert correction into a direct enhancement to the core knowledge base, and using it as gold-standard training data for future model optimization, driving the entire governance method to continuously approach and even surpass the average level of human experts. Continuing the example in step 6, the expert believes that for the original term "head MR scan," the correct standard term is not the model-recommended "head MRI plain scan," but rather "head MRI T1 sequence plain scan." The expert inputs the latter through the correction function and submits it. The front-end interface will capture this correction action and send a more information-rich request to the back-end. In addition to the task identifier and operation type correction, the request must also carry the new standard terminology provided by experts, namely the string "Head MRI T1 sequence plain scan". Upon receiving this request, the backend performs the following key steps: First, it updates the mapping knowledge base. This is the process of instantly solidifying expert knowledge. The program constructs a new mapping record and stores it in the MappingKnowledge knowledge base. This record is constructed as follows: the original terminology is assigned "Head MR scan". The standard terminology is assigned the correct terminology provided by the expert, "Head MRI T1 sequence plain scan", rather than the model's original incorrect recommendation. The confidence level is also assigned a value representing complete trust, such as 1.0. The source is assigned "Expert review-correction" to identify it as a knowledge source of the highest quality level. The creation time is recorded with the current timestamp. Second, it generates incremental learning samples. To allow the model to learn from this error, this correction needs to be transformed into a sample that can be used for training. The program stores this expert-annotated correct mapping pair (original terminology: "Head MR scan", correct standard terminology: "Head MRI T1 sequence plain scan") and its corresponding original image data identifier in a dedicated incremental training dataset. This dataset is a core asset driving model evolution. In future model iterations, this incremental training dataset will be used to fine-tune existing models or participate in the training of entirely new models. During training, when the model processes head MR scans and their image data again, the loss function calculates the distance between its generated unified semantic embedding vector and the embedding vector of the head MRI T1 sequence plain scan, and drives the backpropagation algorithm to adjust the network weights to minimize this distance. Simultaneously, it can also be used to create negative sample pairs with the head MRI plain scans that the model previously incorrectly recommended, further optimizing the model's discriminative ability. Third, the review task is disabled.Finally, the status of the review task will be updated to corrected, and it will be removed from the review queue.

[0067] In summary, the medical image governance method for multi-source heterogeneous data based on the embodiments of this application has been clarified. To address the problems of existing solutions in the background art that rely solely on text matching and lack scalability and self-learning capabilities, this solution first extracts image embedding vectors and text embedding vectors from the original medical image data and its descriptive text using a deep learning model. Then, these two different modal vectors are fused to form a unified semantic embedding vector. This vector simultaneously carries the visual content of the image and the semantic information of the text, truly utilizing the value of the image as a semantic anchor and overcoming the limitations of pure text methods. Based on this unified vector, candidate mappings can be automatically generated through semantic similarity calculation with a standard terminology database, eliminating the reliance on rigid manual rules. More importantly, the solution introduces a closed-loop mechanism of manual review and feedback learning: high-confidence mappings are automatically adopted, low-confidence mappings are submitted to experts for review, and the review results are absorbed into the mapping knowledge base, enabling the system to continuously learn and self-evolve, thereby more intelligently and accurately bridging the semantic gap between cross-institutional data.

[0068] Figure 5 This is a block diagram of a medical image processing system for multi-source heterogeneous data according to an embodiment of this application. Figure 5 As shown, the medical image governance system 100 for multi-source heterogeneous data according to an embodiment of this application includes: a medical image examination data acquisition module 110, used to acquire raw medical image examination data to be processed; a medical image encoding module 120, used to extract and encode multimodal features from the raw medical image examination data to be processed to obtain image embedding vectors and text embedding vectors; a multimodal fusion module 130, used to perform multimodal fusion of the image embedding vectors and text embedding vectors to obtain a unified semantic embedding vector; and a candidate mapping generation module 140, used to generate a unified semantic embedding vector based on a standard terminology library. The input vector is used to generate candidate maps based on similarity. The candidate maps include the original terms, recommended standard terms, and confidence scores. The candidate map storage module 150 stores the candidate map in the mapping knowledge base in response to the confidence score in the candidate map being greater than or equal to the confidence score threshold. The candidate map push module 160 pushes the candidate map to the expert review interface in response to the confidence score in the candidate map being less than the confidence score threshold. The expert review interface includes confirmation, rejection, and correction. The confirmation module 170 stores the candidate map in the mapping knowledge base in response to receiving confirmation from the expert review interface.

[0069] Here, those skilled in the art will understand that the specific operations of each step in the above-described medical image management system for multi-source heterogeneous data have been referenced above. Figures 1 to 4The description of the medical image management method for multi-source heterogeneous data is detailed here, and therefore, its repeated description will be omitted.

[0070] Preferably, a medical image management method for multi-source heterogeneous data according to this application includes: S1, acquiring raw medical image examination data to be processed; S2, extracting and encoding multimodal features from the raw medical image examination data to be processed to obtain image embedding vectors and text embedding vectors; S3, generating candidate mappings based on similarity using a standard terminology database, the candidate mappings include original terms, recommended standard terms, and confidence levels; S4, storing the candidate mappings in a mapping knowledge base in response to a confidence level greater than or equal to a confidence level threshold; S5, pushing the candidate mappings to an expert review interface in response to a confidence level less than the confidence level threshold, the expert review interface includes confirmation, rejection, and correction options; S6, storing the candidate mappings in a mapping knowledge base in response to receiving confirmation from the expert review interface. Specifically, the implementation processes of S1, S2, and S4-S6 in this preferred embodiment are the same as those of steps 1, 2, and 4-7, and therefore will not be described further. The implementation of S3 will be detailed here.

[0071] In one embodiment of S3, based on a standard terminology library, candidate mappings based on similarity are generated for image embedding vectors and text embedding vectors to obtain candidate mappings. The candidate mappings include original terms, recommended standard terms, and confidence scores, including: S31, based on the standard terminology library, decoupled dynamic weighting for multimodal semantic matching is performed on image embedding vectors and text embedding vectors to obtain a set of semantic matching scores; S32, the highest standard term in the set of semantic matching scores is taken as the recommended standard term in the candidate mappings, and the highest score in the set of semantic matching scores is taken as the confidence score in the candidate mappings.

[0072] It is understandable that directly comparing a unified semantic embedding vector that integrates heterogeneous information with a standard term embedding vector consisting only of textual information presents significant challenges in the field of multimodal data semantic matching. Specifically, traditionally, a unified semantic embedding vector is expected to simultaneously carry high-dimensional image features and semantic textual information, while a standard term embedding vector purely represents the textual semantic space defined by the language model. A rough cosine similarity calculation between the two may lead to several drawbacks. First, when the original descriptive text and the standard term are highly semantically correlated, if the unified vector is generated by simply concatenating the original image embedding vector and the text embedding vector, the strong image features may dilute the proportion of textual features in the final vector, thus interfering with pure text matching and potentially artificially lowering the final similarity score. Second, the image component in the unified semantic embedding vector cannot find a direct corresponding comparison object in the single-modal standard term embedding vector, making the physical interpretation of the similarity calculation unclear and difficult to determine which modality contributes to the similarity. Furthermore, when the original descriptive text is very short, ambiguous, or contains noise and ambiguity, if the unified semantic embedding vector relies excessively on its image information, but the image itself may correspond to multiple different types of standard checks on a macroscopic level—for example, scans of the head, chest, or abdomen may have macroscopic similarities in visual features—this can lead to insufficient robustness to the original text when the text information is uncertain, resulting in generally low similarity calculation results for all standard terms. In view of the above challenges, this application adopts a decoupled dynamic weighting processing method based on a standard terminology library to perform multimodal semantic matching on image embedding vectors and text embedding vectors. This aims to overcome the shortcomings of direct comparison by decoupling the comparison process of the two modalities, evaluating them separately on their respective independent dimensions, and dynamically adjusting the weights of each modality's contribution according to the characteristics of the input data itself, thereby achieving more accurate and robust semantic matching degree calculation.

[0073] More specifically, in one embodiment of S31, based on a standard terminology database, the image embedding vector and the text embedding vector are subjected to decoupled dynamic weighting for multimodal semantic matching to obtain a set of semantic matching scores, including: The cosine similarity between the text embedding vector and the embedding vectors of each standard term in the standard terminology library is calculated to obtain a text-to-text similarity set. The core purpose of this step is to independently evaluate the pure text semantic closeness between the original descriptive text of the examination to be processed and each standardized term in the standard terminology library. By eliminating the influence of image information, the system can focus on precise matching at the text level. When the original descriptive text is clear and accurate, standard terms that are highly related in text semantics can be directly and efficiently identified, thus providing strong text support signals for subsequent decision-making. Specifically: First, a 768-dimensional text embedding vector is received as input, which corresponds to the medical imaging examination to be processed, such as a cranial CT scan. At the same time, the system accesses a pre-built standard terminology library. Each standard term in this library, such as cranial CT plain scan, chest CT plain scan, etc., is pre-encoded using the same pre-trained language model as the generated text embedding vector, and its corresponding 768-dimensional standard term embedding vector set is stored. The processing traverses each standard term in the standard terminology library. For each standard term, the corresponding standard term embedding vector is extracted, and then the cosine similarity between the text embedding vector and the embedding vectors of each standard term in the standard terminology library is calculated. For example, for the original descriptive text "brain CT," its text embedding vector has already been generated. When this vector is compared with the text embedding vectors of different standard terms in the standard terminology library using cosine similarity calculation, the following results may be obtained: With the text embedding vector of a plain head CT scan, the text-to-text similarity is 0.96; with the text embedding vector of a plain chest CT scan, the text-to-text similarity is 0.30; and with the text embedding vector of an enhanced abdominal CT scan, the text-to-text similarity is 0.25. By performing this operation on all standard terms in the library, a set containing the text-to-text similarity scores for each standard term is obtained—that is, the text-to-text similarity set.

[0074] The cosine similarity between the image embedding vector and the centroid embedding vectors of each standard term in the standard terminology corpus is calculated to obtain the image-concept similarity set. This step aims to evaluate whether the image content to be examined visually conforms to the typical visual concept represented by the standard term. It is independent of textual information and focuses on image modality matching. Even if the original text is unclear or misleading, the system can still effectively identify and match based on the visual features of the image, demonstrating the method's stronger robustness to noisy text. The input to this step is a 512-dimensional image embedding vector, which corresponds to the medical image examination to be processed. This processing also relies on the standard terminology corpus, but it stores not the text embeddings of the standard terms, but their centroid embedding vectors. The centroid embedding vector represents the typical visual concept of a standard term in the image feature space. During model training or data preprocessing, for each standard term in the standard terminology corpus, such as a head CT scan, the system collects a large amount of raw image data that has been rigorously annotated manually and reliably mapped to the standard term. Image embedding vectors are extracted from these image data, and then the arithmetic mean of these sample image embedding vectors is calculated to obtain the 512-dimensional visual centroid embedding vector of the head CT plain scan. The processing traverses each standard term in the standard terminology library. For each standard term, the program extracts its pre-calculated and stored 512-dimensional visual centroid embedding vector. Then, the cosine similarity between the image embedding vector and the centroid embedding vectors of each standard term in the standard terminology library is calculated. For example, the image embedding vector for the image corresponding to the original descriptive text head CT scan has already been generated. When the vector is used to calculate cosine similarity with the centroid embedding vectors of different standard terms in the standard terminology library, the following results may be obtained: With the centroid embedding vector of a head CT scan, the image-concept similarity is 0.90; with the centroid embedding vector of a chest CT scan, the image-concept similarity is 0.15 (because head CT images and chest CT images are visually very different); with the centroid embedding vector of a cervical spine CT scan, the image-concept similarity is 0.50 (all are CT scans, but the anatomical locations are different, so there are some common features but they are not completely identical). Through this operation, a set containing the image-concept similarity scores for each standard term is obtained, i.e., the image-concept similarity set.

[0075] The maximum value in the text-to-text similarity set is extracted, and a sigmoid function is applied to this maximum value to obtain the text weighting value. This step aims to quantify the credibility or clarity of the original descriptive text being processed, providing a dynamic and intelligent basis for subsequent cross-modal weighted combinations. This effectively avoids misleading the final matching results when the original text quality is poor, instead relying more on more stable image features for judgment. Highly clear and accurate text information is assigned a high weight; conversely, if the text is ambiguous or meaningless, its weight will be reduced. Specifically, first, the program iterates through this set, identifying and extracting the maximum value among all scores, denoted as Simtext max. This maximum value represents the peak text similarity between the current input text and a most relevant term in the standard terminology database, indirectly reflecting the quality of the input text information itself. Subsequently, this Simtext max is input into a sigmoid function for calculation, thus obtaining the final text weighting value α. The calculation formula is as follows: α = sigmoid(β × (Sim text max - δ)), where: α is the calculated text weight value, and its value range is between (0, 1). sigmoid(x) = 1 / (1 + ex) is a non-linear activation function that can smoothly map any real number to between 0 and 1. β is the gain coefficient, used to adjust the steepness of the sigmoid function curve, affecting the sensitivity of the transition of α value from low to high. It is determined through hyperparameter optimization experiments on the validation dataset, aiming to optimize the weight switching effect of the model when distinguishing clear text from ambiguous text. In this embodiment, the value of β is preset to 10. Sim text max is the maximum value in the text-text similarity set. δ is the center threshold, which defines the turning point where the weight changes sharply under the action of the sigmoid function. In this embodiment, the value of δ is preset to 0.7. This value is determined by optimizing the model performance on the validation set. For example, when the text similarity reaches 0.7 or higher, the text signal is considered strong enough, and its weight should be significantly increased. Here are two scenarios: 1. When the input text is clear: For example, the maximum value of the text-to-text similarity set, Sim text max, is 0.96, corresponding to the similarity between a cranial CT scan and a plain head CT scan. Then the text weighting value α = sigmoid(10 × (0.96 - 0.7)) ≈ 0.93. In this example, the α value is close to 1, indicating that the input text signal is very clear and reliable. 2. When the input text is ambiguous: For example, the maximum value of the text-to-text similarity set, Sim text max, is only 0.60, corresponding to the similarity between a head scan and a head image, because the description of a head scan is relatively general.The text weighting value α = sigmoid(10 × (0.60 - 0.7)) ≈ 0.27. In this example, the value of α is significantly lower than 0.5, indicating that the input text signal is relatively weak or ambiguous.

[0076] Based on text-weighted values, the text-to-text similarity and image-to-concept similarity scores for each pair in the text-to-text similarity set and the image-to-concept similarity set are weighted and summed to obtain a semantic matching score set. This step is the final integration stage of the entire decoupled dynamic weighting scheme and is also the key to its intelligent adaptation. Its purpose is to comprehensively utilize the matching information of both text and image modalities, and dynamically adjust the contribution ratio of both in the final confidence decision based on the evaluation of the original text quality in the previous step. This effectively overcomes the limitations of a single modality, allowing the system to rely more on text matching results when the text is clear, and to rely more on the more stable image visual features for judgment when the text is ambiguous or unclear, thereby improving the overall accuracy and robustness of semantic matching. The processing traverses each standard term in the standard terminology library. For each standard term, the program extracts its corresponding text-to-text similarity and image-to-concept similarity. Then, the text weighting value α and its complementary value (1-α) are used to weight and sum these two similarities to calculate the final semantic matching score of the standard term. The calculation formula is as follows: Scorefinal{i}=α×Simtext{i}+(1-α)×Simimage{i}, where Scorefinal{i} is the final semantic matching score for the i-th standard term, α is the text weighting value, Simtext{i} is the text-text similarity for the i-th standard term, and Simimage{i} is the image-concept similarity for the i-th standard term. Continuing with the previous example, for the examination data corresponding to the original descriptive text cranial CT scan, the following similarity sets have been obtained: Text-to-text similarity set: {Simtext{head CT plain scan}=0.96,Simtext{chest CT plain scan}=0.30,...}; Image-to-concept similarity set: {Simimage{head CT plain scan}=0.90,Simimage{chest CT plain scan}=0.15,...} Now, consider two different cases, based on the different α values ​​caused by the different original text quality: When the original text cranial CT scan is relatively clear, α≈0.93: Calculate the final score of the head CT plain scan: Scorefinal{head CT plain scan}=0.93×0.96+(1-0.93)×0.90≈0.956; Calculate the final score of the chest CT plain scan: Scorefinal{chest CT plain scan}=0.93×0.30+(1-0.93)×0.15≈0.2895. In this case, due to the high α value, text similarity contributes the vast majority of the final score, resulting in a high score with head CT scans and a low score with chest CT scans, which accurately reflects the semantics of the clear text.When the original head scan is relatively blurry, α≈0.27. In this case, if Simtextmax is 0.60 and Simtext{Head CT Scan} is still 0.60, the blurry text results in a text-to-text similarity score of only 0.60 for Simtext{Head CT Scan} compared to the head CT scan. The image-concept similarity set remains unchanged, with Simimage{Head CT Scan} at 0.90. The final score for the head CT scan is calculated as: Scorefinal{Head CT Scan}=0.27×0.60+(1-0.27)×0.90≈0.819; the final score for the chest CT scan is calculated as: Scorefinal{Chest CT Scan}=0.27×0.20+(1-0.27)×0.15≈0.1635. In this situation, due to the lower α value, image similarity has a greater weight in the final score. Although the text similarity between the original head scan and the head CT scan is not high (0.60), the image similarity is very high (0.90). With a high image weight (1-α=0.73), the final score (0.819) can still reasonably support mapping the examination to the head CT scan, which significantly demonstrates the robustness of this method to fuzzy text.

[0077] By performing this operation on all standard terms in the library, a final set of semantic matching scores is obtained. Each score in this set comprehensively and dynamically reflects the degree of multimodal semantic matching between the data to be processed and the corresponding standard terms, providing an accurate quantitative basis for the subsequent generation of candidate maps.

[0078] Next, in S32, a final semantic matching score is calculated for each standard term. The program iterates through this set and identifies the standard term with the highest score. For example, if the semantic matching score for "head CT scan" is 0.956, which is higher than the scores of all other standard terms in the set, it is designated as the recommended standard term for the original term to be processed. At the same time, this highest score (0.956) is used as the confidence score for this mapping. Finally, a candidate mapping containing the original term, the recommended standard term, and the confidence score is generated.

[0079] The various embodiments of this disclosure have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments.

Claims

1. A method for managing medical images based on multi-source heterogeneous data, characterized in that, include: The process involves: acquiring raw medical image examination data to be processed; extracting and encoding multimodal features from the raw medical image examination data to obtain image embedding vectors and text embedding vectors; and fusing the image embedding vectors and text embedding vectors in a multimodal manner to obtain a unified semantic embedding vector. Based on a standard terminology database, similarity-based candidate mappings are generated from the unified semantic embedding vectors to obtain candidate mappings, which include the original terms, recommended standard terms, and confidence scores. If the confidence level of a candidate mapping is greater than or equal to the confidence threshold, the candidate mapping is stored in the mapping knowledge base. In response to a candidate mapping having a confidence level less than a confidence threshold, the candidate mapping is pushed to the expert review interface, which includes confirmation, rejection, and correction options. In response to receiving confirmation from the expert review interface, the candidate mapping is stored in the mapping knowledge base.

2. The medical image management method for multi-source heterogeneous data according to claim 1, characterized in that, Multimodal feature extraction and encoding are performed on the raw medical image examination data to be processed to obtain image embedding vectors and text embedding vectors, including: extracting image volume data and original descriptive text from the raw medical image examination data to be processed; extracting image embedding vectors from image volume data; and extracting text embedding vectors from original descriptive text.

3. The medical image processing method for multi-source heterogeneous data according to claim 2, characterized in that, in, Extracting image embedding vectors from image volume data includes: inputting the image volume data into a pre-trained 3D convolutional neural network to obtain image embedding vectors; wherein, extracting text embedding vectors from original descriptive text includes: inputting the original descriptive text into a pre-trained language model to obtain text embedding vectors.

4. The medical image management method for multi-source heterogeneous data according to claim 1, characterized in that, Multimodal fusion of image embedding vectors and text embedding vectors to obtain a unified semantic embedding vector includes: concatenating features of image embedding vectors and text embedding vectors to obtain an image-text concatenated embedding vector; and inputting the image-text concatenated embedding vector into one or more fully connected layers for dimensionality reduction and nonlinear transformation to obtain a unified semantic embedding vector.

5. The medical image management method for multi-source heterogeneous data according to claim 1, characterized in that, Based on a standard terminology corpus, candidate mappings are generated from the unified semantic embedding vectors using similarity-based methods. This includes: calculating the semantic matching score between the unified semantic embedding vectors and each standard term embedding vector in the standard terminology corpus; selecting the standard term with the highest semantic matching score as the recommended standard term in the candidate mapping and using the semantic matching score as the confidence level in the candidate mapping.

6. The medical image management method for multi-source heterogeneous data according to claim 5, characterized in that, Calculating the semantic matching score between the unified semantic embedding vector and each standard term embedding vector in the standard term library includes: calculating the cosine similarity between the unified semantic embedding vector and each standard term embedding vector in the standard term library as the semantic matching score.

7. A medical image processing system for multi-source heterogeneous data, characterized in that, include: The medical imaging examination data acquisition module is used to acquire the raw medical imaging examination data to be processed. The medical image encoding module is used to extract and encode multimodal features from the raw medical image examination data to be processed to obtain image embedding vectors and text embedding vectors; the multimodal fusion module is used to fuse the image embedding vectors and text embedding vectors in a multimodal manner to obtain a unified semantic embedding vector. The candidate mapping generation module is used to generate candidate mappings based on similarity from the unified semantic embedding vector using a standard terminology library. The candidate mappings include the original terms, recommended standard terms, and confidence scores. The candidate mapping storage module is used to store candidate mappings into the mapping knowledge base when the confidence level of a candidate mapping is greater than or equal to a confidence threshold. The candidate mapping push module is used to push the candidate mapping to the expert review interface in response to the confidence level of the candidate mapping being less than the confidence level threshold. The expert review interface includes confirmation, rejection, and correction. The confirmation module is used to store the candidate mapping in the mapping knowledge base in response to receiving confirmation from the expert review interface.

Images