Cross-modal medical data hash retrieval method based on partial label disambiguation
Through partial label disambiguation technology and quantitative perceived disambiguation method, combined with the confidence perception comparison hash, the problem of difficulty and inefficiency in cross-modal medical data retrieval is solved, and efficient and low-cost cross-modal medical data retrieval is achieved, which improves the search accuracy and efficiency.
Patent Information
- Application Number
- CN202510560123.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-04-30
- Publication Date
- 2025-08-15
AI Technical Summary
The existing cross-modal medical data relies on a large number of accurate medical data annotations, which is time-consuming and labor-intensive and affected by subjective factors of doctors. The search efficiency is inefficient and cannot effectively support precise medical and clinical decision-making.
Partial label disambiguation technology is used, and the comparison hashing method of quantized perceived disambiguation and confidence perception is combined. Cross-modal medical data is retrieved through roughly annotated medical data, and sample weight is calculated using hash functions and pseudo-label confidence to reduce label ambiguity and quantization error.
Efficient and low-cost cross-modal medical data retrieval is realized, effectively reducing label ambiguity and quantitative errors in Hamming space, and improving retrieval accuracy and efficiency.
Smart Images
Figure CN120492676A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the technical field of medical data analysis, and in particular to a cross-modal medical data hash retrieval method based on partial label disambiguation. Background Art
[0002] Cross-modal hash retrieval projects data from different modalities into a shared Hamming space and retrieves the most similar cross-modal data by calculating the Hamming distance. With the exponential growth of multimodal medical data (CT images, MRI images, diagnostic reports, etc.), cross-modal hash retrieval has become a key technology for improving diagnosis and treatment efficiency. Previously, manual retrieval of pathology reports for similar cases based on medical images was not only time-consuming and labor-intensive, but also susceptible to subjective factors, resulting in low retrieval efficiency and inaccurate results. Therefore, cross-modal hash retrieval of medical data has important research significance and practical application value for promoting precision medicine and clinical decision-making.
[0003] Traditional cross-modal medical data retrieval requires a large amount of accurate medical data annotation, relying on the personal expertise and clinical experience of clinicians, as well as extensive labeling work. However, in real-world scenarios, the massive amount of medical data makes large-scale labeling time-consuming and labor-intensive. Furthermore, labeling is influenced by the subjective factors of doctors and their personal limitations, making it impossible to accurately diagnose disease categories. Currently, cross-modal medical data retrieval methods are primarily real-valued cross-modal retrieval methods based on accurate supervisory information. However, such methods require a large number of accurate and unique annotations as supervisory information during training, which greatly complicates manual labeling. Furthermore, real-valued cross-modal medical data retrieval suffers from low retrieval efficiency and consumes a large amount of storage resources, seriously hindering the practical application of cross-modal medical data retrieval technology. Summary of the Invention
[0004] The technical problem to be solved by the present invention is to provide a cross-modal medical data hash retrieval method based on partial label disambiguation. The retrieval method realizes efficient and low-cost cross-modal medical data retrieval by adopting partial label disambiguation technology.
[0005] In order to solve the above technical problems, the present invention is implemented in the following ways:
[0006] The cross-modal medical data hash retrieval method based on partial label disambiguation specifically includes the following steps:
[0007] S1. Collect medical data and preprocess the medical data;
[0008] S2, extracting medical image and pathology text data features from medical data;
[0009] S3, quantized perception disambiguation;
[0010] S4, confidence-aware contrastive hashing;
[0011] S5. Model parameter update.
[0012] Furthermore, the specific method of step S1 is as follows:
[0013] The collected raw medical data is converted into numerical form, the input sample space is set to X, and the label space is set to K represents the number of disease categories; the training data is in and y j They represent the input j-th medical data sample pair and the corresponding candidate label set, N represents the number of medical data sample pairs, i=1 and i=2 represent medical image modality data and pathology text modality data, respectively;
[0014] Candidate label set y j Contains multiple disease category labels, if the candidate label set y j If the element at the kth position of is 1 (or 0), it means that the label is (or is not) the candidate label for the corresponding input medical data sample pair.
[0015] Furthermore, the specific method of step S2 is as follows:
[0016] The input preprocessed data is converted into a hash representation in the Hamming space. The specific expression is as follows:
[0017]
[0018] Where f(·) represents the hash mapping function, tanh represents the tanh activation function, represents the jth medical data sample pair input, and Represent medical image modality data and pathological text modality data respectively, The hash representation of medical data and the specific expression of the hash code are as follows:
[0019]
[0020] in, and Represent the hash codes of medical images and pathology texts in the binary Hamming space respectively;
[0021] After the above calculation, we get the hash code set of medical data sample pairs. The specific expression is as follows:
[0022]
[0023] Where L represents the length of the hash code, which is usually a value (32, 64, 128, etc.).
[0024] Furthermore, the specific method of step S3 is as follows:
[0025] In partial label learning, a classification confidence is maintained for each candidate label, and the prediction probability distribution expression of the calculation model is as follows:
[0026]
[0027] in, Represents a medical data sample pair The predicted probability distribution of and Represent medical image modality data and pathology text modality data respectively;
[0028] Model for medical data samples The label confidence expression is as follows:
[0029]
[0030] in, Represents the candidate label set y j The label confidence of the j-th sample pair after disambiguation, the cross-modal disambiguation loss function expression is as follows:
[0031]
[0032] Where n represents the size of a training batch;
[0033] In order to solve the impact of quantization error from real-valued features to hash codes in Hamming space on model accuracy, the quantization loss function used in this application is expressed as follows:
[0034]
[0035] in, The element at the lth position of the hash code of the jth medical data sample of the i-th modality;
[0036] The cross-modal quantization-aware disambiguation loss function is expressed as follows:
[0037]
[0038] in, and They represent the cross-modal disambiguation loss function and the cross-modal quantization-aware disambiguation loss function respectively, and λ represents a hyperparameter.
[0039] Furthermore, the specific method of step S4 is as follows:
[0040] The probability expression of the hash medical data sample belonging to the jth category is as follows:
[0041]
[0042] in, and denote the pseudo labels of the vth and jth sample pairs, respectively, τ denotes the temperature hyperparameter, which takes a value of 1.0, represents the sample hash representation of the vth modality in the tth sample pair, represents the sample hash representation of the i-th modality in the j-th sample pair, and sim(·) represents the similarity calculation operation;
[0043] The pseudo label expression of the j-th sample pair is as follows:
[0044]
[0045] For pairs of samples with partial labels, the training weights between different pairs of samples are calculated as follows:
[0046]
[0047] Among them, w j represents the jth sample pair The training weights of the credibility scores, Represents sample pairs The confidence level of belonging to the kth category, K represents the total number of disease categories in the medical data, ∈ represents a hyperparameter that determines the lower bound of the weight, and N represents the size of the training set;
[0048] The expression of the comparative hash loss function is as follows:
[0049]
[0050] Furthermore, the specific method of step S5 is as follows:
[0051] The quantization-aware disambiguation loss function is Comparison with confidence-aware hash loss function The model calculates the total loss function, which is expressed as follows:
[0052]
[0053] Among them, α represents the hyperparameter of the balance loss function;
[0054] By calculating the total loss of the model, completing the model training through back propagation and parameter update, an effective cross-modal medical data retrieval model is obtained.
[0055] Compared with the prior art, the present invention has the following beneficial effects:
[0056] The present invention adopts partial label disambiguation technology to achieve efficient and low-cost cross-modal medical data retrieval based on roughly annotated medical data; uses quantization-aware disambiguation technology to balance the disambiguation loss of partial labels and the quantization loss of hash functions, effectively reducing label ambiguity and quantization error in Hamming space; uses confidence-aware comparative hashing technology to formulate sample weights based on the credibility scores calculated based on pseudo-labels and label confidence, thereby effectively eliminating modal differences. BRIEF DESCRIPTION OF THE DRAWINGS
[0057] Figure 1 Schematic diagram of the flow of the hash retrieval method of the present invention. DETAILED DESCRIPTION
[0058] The specific implementation of the present invention will be further described in detail below with reference to the accompanying drawings and specific examples.
[0059] like Figure 1 As shown in FIG, the cross-modal medical data hash retrieval method based on partial label disambiguation specifically includes the following steps:
[0060] S1. Collect medical data and preprocess the medical data. The specific methods are as follows:
[0061] In cross-modal medical data retrieval, the original data are medical images and pathology reports. The collected original medical data are converted into numerical form for subsequent deep learning calculations. The input sample space is set to X and the label space is set to K represents the number of disease categories; the training data is in and y j They represent the input j-th medical data sample pair and the corresponding candidate label set, N represents the number of medical data sample pairs, i=1 and i=2 represent medical image modality data and pathology text modality data, respectively;
[0062] Candidate label set y j Contains multiple disease category labels, only one of which is the correct label, and the rest are label ambiguities; if the candidate label set y j If the element at the kth position of is 1 (or 0), it means that the label is (or is not) the candidate label for the corresponding input medical data sample pair.
[0063] S2. Extract medical image and pathology text data features from medical data. The specific method is as follows:
[0064] In order to perform cross-modal hash retrieval between the input medical image and text feature data, the input preprocessed data is converted into a hash representation in the Hamming space. The specific expression is as follows:
[0065]
[0066] Where f(·) represents the hash mapping function, tanh represents the tanh activation function, represents the jth medical data sample pair input, and Represent medical image modality data and pathological text modality data respectively, The hash representation of medical data and the specific expression of the hash code are as follows:
[0067]
[0068] in, and Represent the hash codes of medical images and pathology texts in the binary Hamming space respectively;
[0069] After the above calculation, we get the hash code set of medical data sample pairs. The specific expression is as follows:
[0070]
[0071] Where L represents the length of the hash code, which is usually a value (32, 64, 128, etc.).
[0072] S3. Quantization-aware disambiguation. The specific method is as follows:
[0073] In partial label learning, a classification confidence is maintained for each candidate label. The candidate label with the highest confidence is the result after disambiguation. In order to obtain the confidence of each sample pair, the prediction probability distribution expression of the calculation model is as follows:
[0074]
[0075] in, Represents a medical data sample pair The predicted probability distribution of and Represent medical image modality data and pathology text modality data respectively;
[0076] According to the setting of partial label learning, the confidence on non-candidate labels is always 0, and the model is not very accurate for medical data samples. The label confidence expression is as follows:
[0077]
[0078] in, Represents the candidate label set y j The label confidence of the j-th sample pair after disambiguation is an important basis for subsequent model algorithms; the cross-modal disambiguation loss function is expressed as follows:
[0079]
[0080] Where n represents the size of a training batch;
[0081] In order to solve the impact of quantization error from real-valued features to hash codes in Hamming space on model accuracy, the quantization loss function used in this application is expressed as follows:
[0082]
[0083] in, The element at the lth position of the hash code of the jth medical data sample of the i-th modality;
[0084] The cross-modal quantization-aware disambiguation loss function is expressed as follows:
[0085]
[0086] in, and They represent the cross-modal disambiguation loss function and the cross-modal quantization-aware disambiguation loss function respectively, and λ represents a hyperparameter.
[0087] S4. Confidence-aware comparative hashing. The specific method is as follows:
[0088] In cross-modal hash retrieval, contrastive hashing is an efficient method to reduce modality differences. The probability expression of a contrastive hash medical data sample belonging to the jth category is as follows:
[0089]
[0090] in, and denote the pseudo labels of the vth and jth sample pairs, respectively, τ denotes the temperature hyperparameter, which takes a value of 1.0, represents the sample hash representation of the vth modality in the tth sample pair, represents the sample hash representation of the i-th modality in the j-th sample pair, and sim(·) represents the similarity calculation operation;
[0091] The pseudo label expression of the j-th sample pair is as follows:
[0092]
[0093] In the sample pairs with partial labels, the training weights between different sample pairs are the confidence of their labels. Related, such as label confidence The more uniform the distribution of , the less information the model has about the sample pair, meaning the sample pair's weight in the comparative hash loss should be smaller. In machine learning, normalized information entropy is an effective way to measure the uniformity of probability distribution. The training weights between different sample pairs are calculated as follows:
[0094]
[0095] Among them, w j represents the jth sample pair The training weights of the credibility scores, Represents sample pairs The confidence level of belonging to the kth category, K represents the total number of disease categories in the medical data, ∈ represents a hyperparameter that determines the lower bound of the weight, and N represents the size of the training set;
[0096] The expression of the comparative hash loss function is as follows:
[0097]
[0098] S5. Model parameter update. The specific method is as follows:
[0099] The quantization-aware disambiguation loss function is Comparison with confidence-aware hash loss function The model calculates the total loss function, which is expressed as follows:
[0100]
[0101] Among them, α represents the hyperparameter of the balance loss function;
[0102] By calculating the total loss of the model, completing the model training through back propagation and parameter update, an effective cross-modal medical data retrieval model is obtained.
[0103] The above description is merely an embodiment of the present invention. It is stated again that, for a person skilled in the art, several improvements can be made to the present invention without departing from the principles of the present invention, and these improvements are also included in the scope of protection of the claims of the present invention.
Claims
1. A cross-modal medical data hash retrieval method based on partial label disambiguation, characterized by: The specific steps include: S1. Collect medical data and preprocess the medical data; S2, extracting medical image and pathology text data features from medical data; S3, quantized perception disambiguation; S4, confidence-aware contrastive hashing; S5. Model parameter update.
2. The cross-modal medical data hash retrieval method based on partial label disambiguation according to claim 1, characterized in that: The specific method of step S1 is as follows: The collected raw medical data is converted into numerical form, the input sample space is set to X, and the label space is set to K represents the number of disease categories; the training data is in and y j They represent the input j-th medical data sample pair and the corresponding candidate label set, N represents the number of medical data sample pairs, i=1 and i=2 represent medical image modality data and pathology text modality data, respectively; Candidate label set y j Contains multiple disease category labels. If the element at the kth position of the candidate label set y is 1 or 0, it means that the label is or is not a candidate label for the corresponding input medical data sample pair.
3. The cross-modal medical data hash retrieval method based on partial label disambiguation according to claim 1, characterized in that: The specific method of step S2 is as follows: The input preprocessed data is converted into a hash representation in the Hamming space. The specific expression is as follows: Where f(·) represents the hash mapping function, tanh represents the tanh activation function, represents the jth medical data sample pair input, and Represent medical image modality data and pathological text modality data respectively, The hash representation of medical data and the specific expression of the hash code are as follows: in, and Represent the hash codes of medical images and pathology texts in the binary Hamming space respectively; After the above calculation, we get the hash code set of medical data sample pairs. The specific expression is as follows: Where L represents the length of the hash code.
4. The cross-modal medical data hash retrieval method based on partial label disambiguation according to claim 1, characterized in that: The specific method of step S3 is as follows: In partial label learning, a classification confidence is maintained for each candidate label, and the prediction probability distribution expression of the calculation model is as follows: in, Represents a medical data sample pair The predicted probability distribution of and Represent medical image modality data and pathology text modality data respectively; Model for medical data samples The label confidence expression is as follows: in, Represents the candidate label set y j The label confidence of the j-th sample pair after disambiguation, the cross-modal disambiguation loss function expression is as follows: Where n represents the size of a training batch; The quantization loss function expression used is as follows: in, The element at the lth position of the hash code of the jth medical data sample of the i-th modality; The cross-modal quantization-aware disambiguation loss function is expressed as follows: in, and They represent the cross-modal disambiguation loss function and the cross-modal quantization-aware disambiguation loss function respectively, and λ represents a hyperparameter.
5. The cross-modal medical data hash retrieval method based on partial label disambiguation according to claim 1, characterized in that: The specific method of step S4 is as follows: The probability expression of the hash medical data sample belonging to the jth category is as follows: in, and denote the pseudo labels of the vth and jth sample pairs, respectively, τ denotes the temperature hyperparameter, which takes a value of 1.0, represents the sample hash representation of the vth modality in the tth sample pair, represents the sample hash representation of the i-th modality in the j-th sample pair, and sim(·) represents the similarity calculation operation; The pseudo label expression of the j-th sample pair is as follows: For pairs of samples with partial labels, the training weights between different pairs of samples are calculated as follows: Among them, w j represents the jth sample pair The training weights of the credibility scores, Represents sample pairs The confidence level of belonging to the kth category, K represents the total number of disease categories in the medical data, ∈ represents a hyperparameter that determines the lower bound of the weight, and N represents the size of the training set; The expression of the comparative hash loss function is as follows:
6. The cross-modal medical data hash retrieval method based on partial label disambiguation according to claim 1, characterized in that: The specific method of step S5 is as follows: The quantization-aware disambiguation loss function is Comparison with confidence-aware hash loss function The model calculates the total loss function, which is expressed as follows: Among them, α represents the hyperparameter of the balance loss function; By calculating the total loss of the model, completing the model training through back propagation and parameter update, an effective cross-modal medical data retrieval model is obtained.