Medical image partial multi-label classification method and related device

Abstract

The invention discloses a medical image partial multi-label classification method and a related device, and the method comprises the steps: obtaining a training set, and determining the credible label distribution of each training sample in the training set based on a K-nearest neighbor label attention mechanism; training a preset multi-label classifier through the training set and the credible label distribution to obtain a target multi-label classifier; obtaining a to-be-predicted medical image, and inputting the to-be-predicted medical image into the target multi-label classifier; performing label semantic extraction according to the to-be-predicted medical image through an encoder to obtain a label semantic matrix; performing label interaction enhancement operation according to the label semantic matrix through a decoder to obtain label semantic embedding; and linear prediction is carried out through a linear prediction layer according to label semantic embedding, and prediction label distribution of the to-be-predicted medical image is generated. According to the method, the relationship between the medical image features and the label semantics and the label interaction can be captured, so that the label classification precision in the multi-label classification task of the medical image is effectively improved.

Description

Technical Field

[0001] This invention relates to the field of medical image processing technology, and in particular to a method and apparatus for multi-label classification of medical images. Background Technology

[0002] With the widespread application of smart terminal devices and the rapid development of the mobile internet, multimodal data such as images, text, voice, and video are experiencing exponential growth in various practical application scenarios, including social media, smart security, medical diagnosis, and recommendation systems. Faced with such a massive information flow, accurately and automatically assigning multiple semantic labels to each sample has become a key task, thus driving the rapid development of Multi-Label Learning (MLL) technology. In MLL tasks, a sample often corresponds to multiple semantic labels. Traditional methods typically assume that each training sample is completely and accurately labeled with all relevant labels, using this as a supervisory signal to train the model. However, obtaining fully labeled and high-quality multi-label data faces the dual challenges of high manual labeling costs and label uncertainty caused by subjectivity. To address these issues, Partial Multi-Label Learning (PML) has been proposed as a weakly supervised learning framework that can effectively model even when label information is incomplete or noisy. In the PML task, each training sample is assigned only one set of candidate labels. Only a portion of the labels in this set are real and relevant labels, while the rest are noise labels. The model does not know which labels are real and which are mislabeled noise labels.

[0003] Existing multi-label learning methods rely on heuristic rules or similarity-based metrics (such as label propagation and K-nearest neighbor reconstruction) to first generate reliable labels, and then train them using traditional multi-label classifiers. While these methods are simple in structure and easy to implement, they have several drawbacks: they use one-hot encoding to represent labels, ignoring potential semantic relationships between labels, and their processing flow is prone to error propagation and accumulation. Especially in complex scenarios like medical image analysis, image data often suffers from missing labels, inconsistent annotations, or semantic complexity, further amplifying these shortcomings and severely impacting the accuracy of medical image label classification. Summary of the Invention

[0004] This invention provides a method and apparatus for partial multi-label classification of medical images, which addresses the technical problem of low label classification accuracy in existing partial multi-label learning methods for multi-label classification tasks of medical images.

[0005] This invention provides a method for multi-label classification of medical images, the method comprising:

[0006] Obtain the training set and determine the reliable label distribution of each training sample in the training set based on the K-nearest neighbor label attention mechanism;

[0007] A preset multi-label classifier is trained using the training set and the reliable label distribution of each training sample to obtain the target multi-label classifier;

[0008] A medical image to be predicted is acquired and input into the target multi-label classifier. In the target multi-label classifier, the encoder extracts the label semantics based on the medical image to be predicted to obtain a label semantic matrix. The decoder performs label interaction enhancement operations based on the label semantic matrix to obtain the label semantic embedding. The linear prediction layer performs linear prediction based on the label semantic embedding to generate the predicted label distribution of the medical image to be predicted.

[0009] Optionally, the training set includes medical image training samples with real labels and noisy labels; the step of determining the reliable label distribution of each training sample in the training set based on the K-nearest neighbor label attention mechanism includes:

[0010] Based on the K-nearest neighbor label attention mechanism, the neighbor set and the candidate label set of each training sample in the training set are determined;

[0011] Calculate the similarity between each training sample and each neighbor sample in its neighbor set, and determine the corresponding weighting weight based on the similarity of the neighbor samples.

[0012] The reliable label distribution of each training sample is calculated based on the weighted weights of all the neighboring samples corresponding to each training sample and their candidate label sets.

[0013] Optionally, the step of extracting label semantics from the medical image to be predicted using an encoder to obtain a label semantic matrix includes:

[0014] In the encoder, the medical image to be predicted is convolved by a convolutional layer to obtain multiple convolutional sequences;

[0015] Max pooling is performed on multiple convolutional sequences to obtain multiple pooled features;

[0016] The multiple pooling features are mapped to the label semantic space using a linear mapping to obtain multiple label semantic vectors;

[0017] By concatenating all the aforementioned label semantic vectors, the label semantic matrix of the medical image to be predicted is obtained.

[0018] Optionally, the decoder includes multiple decoding layers connected in sequence, and each decoding layer includes a single-head attention interaction module, a label semantic enhancement module, a feedforward neural network, and a connection module connected in sequence.

[0019] In each of the decoding layers, the single-head attention interaction module performs attention interaction operations based on the input features to obtain the label attention output; based on the label summary representation of the label semantic matrix, the label semantic enhancement module performs label semantic enhancement operations based on the label attention output to obtain the label semantic enhancement output; the feedforward neural network performs a nonlinear transformation on the label semantic enhancement output to obtain the feedforward output; and the connection module performs a layer normalization operation on the label semantic enhancement output and the feedforward output to obtain the decoding output.

[0020] Optionally, the step of obtaining the tag semantic enhancement output by performing tag semantic enhancement operations on the tag semantic matrix-based tag summary representation through the tag semantic enhancement module based on the tag attention output includes:

[0021] In the tag semantic enhancement module, the tag summary representation of the tag semantic matrix is ​​projected as a modulation matrix;

[0022] By fusing the label attention output and the modulation matrix, the modulation weights in the label dimension are obtained;

[0023] Based on the modulation weights, the label attention output and the modulation matrix are fused to obtain label fusion;

[0024] A layer normalization operation is performed on the label attention output and the label fusion to obtain the label semantic enhancement output.

[0025] Optionally, the step of training a preset multi-label classifier using the training set and the reliable label distribution of each training sample to obtain the target multi-label classifier includes:

[0026] A preset multi-label classifier is trained using training samples from the training set, and the predicted label distribution of the training samples is output. The model parameters of the preset multi-label classifier are adjusted according to the predicted label distribution and the reliable label distribution of the training samples until the loss function of the preset multi-label classifier meets a preset threshold, thereby generating a target multi-label classifier.

[0027] The loss function is expressed as:

[0028]

[0029] In the formula: This represents the loss function of a predefined multi-label classifier; This represents the total number of training samples in the training set. This represents the prediction confidence weighting function. Indicates the first The distribution of predicted labels for each training sample; Indicates the first The distribution of reliable labels for each training sample.

[0030] The present invention also provides a medical image partial multi-label classification system, the system comprising:

[0031] The training set and trusted label distribution acquisition unit is used to acquire the training set and determine the trusted label distribution of each training sample in the training set based on the K-nearest neighbor label attention mechanism.

[0032] The target multi-label classifier generation unit is used to train a preset multi-label classifier through the training set and the reliable label distribution of each training sample to obtain the target multi-label classifier.

[0033] The predicted label distribution calculation unit is used to acquire the medical image to be predicted and input the medical image to be predicted into the target multi-label classifier. In the target multi-label classifier, the encoder extracts the label semantics based on the medical image to be predicted to obtain the label semantic matrix. The decoder performs label interaction enhancement operation based on the label semantic matrix to obtain the label semantic embedding. The linear prediction layer performs linear prediction based on the label semantic embedding to generate the predicted label distribution of the medical image to be predicted.

[0034] The present invention also provides a computer device, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the medical image partial multi-label classification method as described above.

[0035] The present invention also provides a computer-readable storage medium having a computer program / instructions stored thereon, which, when executed by a processor, implements the steps of the medical image partial multi-label classification method as described above.

[0036] The present invention also provides a computer program product, including a computer program / instructions, characterized in that, when the computer program / instructions are executed by a processor, they implement the steps of the medical image partial multi-label classification method as described above.

[0037] As can be seen from the above technical solutions, the present invention has the following advantages:

[0038] This invention provides a method and related apparatus for partial multi-label classification of medical images. The method includes: acquiring a training set and determining the reliable label distribution of each training sample in the training set based on a K-nearest neighbor label attention mechanism; training a preset multi-label classifier using the training set and the reliable label distribution of each training sample to obtain a target multi-label classifier; acquiring a medical image to be predicted and inputting the medical image to be predicted into the target multi-label classifier; in the target multi-label classifier, extracting label semantics from the medical image to be predicted using an encoder to obtain a label semantic matrix; performing label interaction enhancement operations based on the label semantic matrix using a decoder to obtain label semantic embeddings; and generating a predicted label distribution of the medical image to be predicted by performing linear prediction based on the label semantic embeddings using a linear prediction layer.

[0039] In this invention, a reliable label distribution can be extracted from noisy labels based on the K-nearest neighbor label attention mechanism, which can effectively alleviate the interference of noisy labels on the training of the preset multi-label classifier. The target multi-label classifier based on the encoder-decoder architecture can achieve unified optimization of medical image features, label semantics, label interaction relationships, and label prediction. The encoder can establish a high-order representation of medical image features in the label semantic space, and the decoder can further capture the interaction relationships between labels, thereby improving the classification accuracy of the predicted label distribution. This solves the technical problem of low label classification accuracy in the multi-label classification task of medical images by existing biased multi-label learning methods. Attached Figure Description

[0040] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0041] Figure 1 A flowchart illustrating the steps of a medical image multi-label classification method provided in this embodiment of the invention;

[0042] Figure 2 This is a training architecture diagram of a preset multi-label classifier provided in an embodiment of the present invention;

[0043] Figure 3 This is a structural block diagram of a medical image multi-label classification system provided in an embodiment of the present invention. Detailed Implementation

[0044] This invention provides a medical image partial multi-label classification method and related apparatus to solve the technical problem of low label classification accuracy in existing partial multi-label learning methods for medical image multi-label classification tasks.

[0045] To make the objectives, features, and advantages of this invention more apparent and understandable, the technical solutions of the embodiments of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the embodiments described below are only some embodiments of this invention, and not all embodiments. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0046] Please see Figure 1 This invention provides a method for multi-label classification of medical images, the method comprising:

[0047] Step 101: Obtain the training set and determine the reliable label distribution of each training sample in the training set based on the K-nearest neighbor label attention mechanism.

[0048] The training set includes medical image training samples with real labels and noisy labels. The medical images can be X-rays, CT scans, and MRI (magnetic resonance imaging), and the corresponding labels can include pneumonia, emphysema, and cerebral hemorrhage.

[0049] For ease of understanding, the data symbols for the multi-label task involved in this implementation are defined below:

[0050] set up Let be the feature space of the samples, where n represents the total number of samples and d represents the dimension of the sample features. The label space is defined as follows: , which contains There are 10 different labels. Therefore, the training set can be denoted as: ,in, Indicates the first Feature vectors of each sample; Indicates the first The candidate label set corresponding to each sample is a set of length . The binary vector, where each bit corresponds to a specific label: when a bit is 1, it means that the label is a candidate label for the sample; if it is 0, it means that it is not a candidate label.

[0051] In multi-label classification tasks, it is assumed that the candidate label set must contain all true labels, but it may also contain irrelevant noise labels. Therefore, the following set inclusion relationship exists: That is, the set of real labels All positive tags must also be in the candidate tag set. It appeared in, but The dataset may also contain some additional incorrect labels. The goal of partial multi-label classification is to base the training dataset on a subset of observations. Learn a multi-label classifier function: This classifier is able to classify any new test sample. Output its predicted set of labels This enables label prediction.

[0052] For example, an X-ray showing bilateral lung infection might have true labels of pneumonia and pulmonary edema, but the candidate label set (including noisy labels) includes pneumonia, pulmonary edema, and tuberculosis. In a partial multi-label classification task, the multi-label classifier needs to correctly predict pneumonia and pulmonary edema while suppressing interference from tuberculosis.

[0053] This invention proposes a reliable label distribution extraction method based on a label attention mechanism, applicable to some label learning scenarios. It aims to extract a more reliable label distribution from observed noisy labels through feature similarity measurement. Unlike traditional Transformer models where the attention mechanism is only applied to the feature space, this invention introduces the attention mechanism into the label space and combines it with feature similarity information between samples to guide the allocation of label weights, thereby effectively mitigating the interference of noisy labels on model training.

[0054] For further details, please refer to Figure 2 The steps for determining the reliable label distribution of each training sample in the training set based on the K-nearest neighbor label attention mechanism specifically include the following sub-steps S11~S13:

[0055] S11. Based on the K-nearest neighbor label attention mechanism, determine the neighbor set and candidate label set of each training sample in the training set.

[0056] From the training set Selecting from and Most similar Each sample constitutes its neighborhood set. Let these neighbor samples be... Its corresponding feature matrix and candidate label matrix They are defined as follows:

[0057]

[0058] S12. Calculate the similarity between each training sample and each neighbor sample in its neighbor set, and determine the corresponding weighting weight based on the similarity between the neighbor samples.

[0059] The similarity between each training sample and each neighbor sample in its neighbor set is calculated as follows:

[0060]

[0061] In the formula: represents the similarity between the i-th training sample and the t-th neighbor sample; · represents the vector inner product. Representing vectors Norm.

[0062] right Similarity calculated from all neighbor samples Perform softmax normalization to obtain the weighted weights of the neighbor labels. ,Right now:

[0063]

[0064] Among them, weighted weight Describes neighbor samples Its importance in the generation of trusted label distributions, satisfying .

[0065] S13. Calculate the reliable label distribution of each training sample based on the weighted weights of all neighboring samples and their candidate label sets.

[0066] Specifically, the candidate label set of neighbor samples With corresponding weights Multiply and sum by weight to obtain the training samples. Trusted label distribution , means as follows:

[0067]

[0068] In summary, the distribution of trustworthy labels The overall calculation formula can be expressed as:

[0069]

[0070] This embodiment establishes a reliable label distribution for training samples based on the K-nearest neighbor label attention mechanism, which can adaptively capture the semantic relationships between labels.

[0071] Step 102: Train a preset multi-label classifier using the training set and the reliable label distribution of each training sample to obtain the target multi-label classifier.

[0072] Specifically, a pre-defined multi-label classifier is trained using training samples from the training set, and the predicted label distribution of the training samples is output. The model parameters of the pre-defined multi-label classifier are adjusted based on the predicted label distribution and the reliable label distribution of the training samples until the loss function of the pre-defined multi-label classifier meets the preset threshold, thereby generating the target multi-label classifier.

[0073] The preset multi-label classifier is based on an encoder-decoder architecture; please refer to [link / reference]. Figure 2 In the pre-defined multi-label classifier, the encoder extracts the label semantics of multiple samples. Specifically, the encoder receives the input sample features, extracts high-level semantic features through convolutional layers and max pooling, and maps them to the label semantic space; further, it generates the semantic representation vector of each label through linear projection. This ultimately forms the tag semantic matrix. The decoder provides structural semantic information. In the pre-defined multi-label classifier, the decoder explores higher-order relationships between labels. Specifically, the decoder uses a recursive structure to model multi-layer label interactions. Each layer includes a single-head attention interaction module, a label semantic enhancement module, and a feedforward neural network. The single-head attention interaction module is used to mine the contextual relevance between labels, the label semantic enhancement module modulates the current label representation using the label summary vector, and the feedforward neural network enhances nonlinear modeling capabilities. The output of each layer is passed to the next layer through residual connections and LayerNorm normalization, ultimately obtaining the label-level semantic embedding. Then, the model is optimized using an adaptive weighted loss function. By applying the sigmoid function to each label vector through a shared linear predictor, the probability distribution of the label's existence is obtained. This probability distribution is then subjected to an adaptive weighted binary cross-entropy calculation with the distribution of trustworthy labels, thereby minimizing the loss function to optimize the various model parameters.

[0074] In this embodiment, the preset multi-label classifier is based on the collaborative design of the encoder-decoder architecture. Under the end-to-end framework, it completes the unified closed-loop optimization of medical image sample features, label semantics, label interaction relationships, and label prediction. It can effectively avoid error propagation, reduce dependence on strong priors, and exhibit stronger robustness and generalization ability in high-noise and weakly labeled medical image environments. This further improves the label prediction accuracy of medical images and provides interpretable decision-making basis for clinical diagnosis.

[0075] The loss function mentioned above takes into account the consistency between the model's predicted probability and the credible labels inferred by the K nearest neighbors, and dynamically weights them according to the prediction confidence. This effectively improves the model's learning ability in the case of weak supervision or label uncertainty, and enhances the training ability of a robust model with uncertainty awareness.

[0076] The loss function is expressed as:

[0077]

[0078] In the formula: This represents the loss function of a predefined multi-label classifier; This represents the total number of training samples in the training set. This represents the prediction confidence weighting function. Its value approaches 0 when the prediction is close to 0 or 1 (i.e. the model is relatively confident), and takes the maximum value of 1 when the prediction is close to 0.5 (i.e. the model is uncertain); Indicates the first The distribution of predicted labels for each training sample; Indicates the first The distribution of reliable labels for each training sample.

[0079] When the pre-defined multi-label classifier predicts a certain label as "almost certain" (i.e.) or The corresponding loss weight is lower if the model is uncertain about predicting a certain label (i.e., ...). ), and the distribution of neighboring labels If a deviation exists, the prediction contributes more to the total loss, guiding the model to prioritize correcting erroneous predictions in such uncertain areas.

[0080] Step 103: Obtain the medical image to be predicted and input it into the target multi-label classifier. In the target multi-label classifier, the encoder extracts the label semantics based on the medical image to be predicted to obtain the label semantic matrix. The decoder performs label interaction enhancement operation based on the label semantic matrix to obtain the label semantic embedding. The linear prediction layer performs linear prediction based on the label semantic embedding to generate the predicted label distribution of the medical image to be predicted.

[0081] In this embodiment, the target multi-label classifier uses an encoder to extract the semantic labels of the medical images to be predicted, thereby establishing a high-order feature representation of the medical images to be predicted in the label semantic space. This achieves feature alignment in the label semantic space, which can better distinguish visually similar but essentially different diseases, reduce prediction errors, and improve the accuracy of subsequent label prediction classification. The decoder further captures the interaction relationships between labels based on the label semantic matrix extracted by the encoder, establishes the display relationships between labels, achieves collaborative prediction, avoids contradictory prediction results, and improves the robustness of label prediction.

[0082] It should be noted that the medical image to be predicted needs to undergo feature extraction processing before being input into the target multi-label classifier. A pre-trained convolutional neural network (such as ResNet50 or DenseNet121) can be used to obtain the high-dimensional semantic feature vector of the medical image to be predicted.

[0083] Furthermore, the encoder extracts the semantic labels from the medical image to be predicted, resulting in a semantic label matrix. This process includes the following sub-steps S21-S24:

[0084] S21. In the encoder, the medical image to be predicted is convolved through a convolutional layer to obtain multiple convolutional sequences.

[0085] Suppose the feature vector of the i-th input sample (the medical image to be predicted) is To extract local features, the convolution operation is performed on the input vector in the form of a sliding window, and the specific calculation is as follows:

[0086]

[0087] in, For the first The convolution result at each position; This indicates the th feature vector from the input sample. Starting from position [number] and with a window width of [value]... Continuous local regions; The first convolutional layer represents the... Each convolutional kernel (or convolutional window) j (1 ≤ j ≤ q) represents the view number; The bias term is ReLU; ReLU is the activation function, defined as ReLU. .

[0088] By combining the convolution results from all positions, we obtain the output sequence of the entire convolution operation under this kernel:

[0089]

[0090] in, The convolution sequence output by the j-th convolution kernel.

[0091] S22. Perform max pooling on multiple convolutional sequences to obtain multiple pooled features.

[0092] To select the most significant feature information from the convolution results, a max pooling operation is introduced. Dimensionality reduction processing:

[0093]

[0094] in, The pooling feature corresponding to the j-th convolutional kernel; Max pooling is an operation that takes the maximum value for each feature dimension in the sequence. The vector is the local high-level feature representation corresponding to the convolution kernel.

[0095] S23. A linear mapping is used to map multiple pooling features to the label semantic space, resulting in multiple label semantic vectors.

[0096] eigenvectors The semantic vector representation used to construct the label is pooled using a linear mapping. Mapping to the label semantic space is defined as follows:

[0097]

[0098] in, The weight matrix is ​​a learnable linear mapping. For bias terms, For the first The semantic vector representation of each label, This indicates the dimension of the tag embedding space.

[0099] S24. Concatenate all the label semantic vectors to obtain the label semantic matrix of the medical image to be predicted.

[0100] Concatenate the semantic vectors of all tags into a tag semantic matrix. ,in The number of tags is represented as follows:

[0101]

[0102] This semantic matrix represents the input sample. The semantic structure in the label space provides a semantic foundation for subsequent label prediction.

[0103] In this embodiment, the decoder aims to perform high-order modeling of the label semantic representation extracted by the encoder, further capturing the contextual dependencies between labels. The overall structure of the decoder adopts a recursive hierarchical update, and the decoder includes multiple sequentially connected decoding layers. Each decoding layer includes a single-head attention interaction module, a label semantic enhancement module, a feedforward neural network, and a connection module, all connected in sequence.

[0104] Assume the decoder has a total of To facilitate the representation of intermediate states in each layer, a recursive variable is introduced into the layer decoding layer. ,in , This represents the output of the l-th decoding layer of the decoder. This variable will be recursively passed to the next layer to progressively construct the final predicted label distribution. The label semantic matrix represents the output of the encoder. ,Right now: .

[0105] The following provides a further explanation of the implementation process of each module in the decoding layer, specifically including the following sub-steps S31~S34:

[0106] S31. In each decoding layer, the single-head attention interaction module performs attention interaction operations based on the input features to obtain the label attention output.

[0107] It is understandable that the input features mentioned above can be the label semantic matrix output by the encoder. It can also be the decoded output of the previous decoding layer. The single-head attention interaction module is mainly used to enhance the semantic interaction capabilities between tags, thereby establishing dependencies between tags.

[0108] No. The input features of the single-head attention interaction module of the layer decoding layer are represented as follows: First, the input features are processed through three learnable linear mapping parameters to construct the query, key, and value vectors in the attention mechanism:

[0109]

[0110] in, All of these are learnable parameter matrices.

[0111] Next, the attention weights between labels and their aggregated output are calculated using a scaled dot product attention mechanism. The specific calculation formula is as follows:

[0112]

[0113] in, This represents the similarity score matrix between tags. As a scaling factor, to avoid excessively large values, the softmax function is used for normalization to obtain attention weights, which are then combined with... The attention output is obtained after multiplication. .

[0114] Then, in order to improve training stability and preserve the original feature information, a residual connection and layer normalization mechanism is adopted, as shown below:

[0115]

[0116] in, This represents the label attention output of the l-th layer single-head attention interaction module; LayerNorm is a layer normalization operation that performs independent normalization on each label vector to improve convergence efficiency.

[0117] In summary, the overall calculation formula for the attention output of the single-head attention interaction module is as follows:

[0118]

[0119] in, As input for subsequent tag semantic enhancement modules.

[0120] S32. Tag summarization representation based on the tag semantic matrix: the tag semantic enhancement module performs tag semantic enhancement operation based on the tag attention output to obtain the tag semantic enhancement output.

[0121] The tag semantic enhancement module further enhances the semantic interaction between tags by using the global tag summary vector of the tag semantic matrix. Modulate the label representation of the current layer to guide the model to perform more specific semantic fusion between different labels.

[0122] It should be noted that this can be achieved by analyzing the tag semantic matrix. Max pooling is performed to obtain the global tag summary vector of the tag semantic matrix. Global tag summary vector Global tag summary vector is used to represent the overall semantic information of a tag set. The calculation formula is expressed as:

[0123]

[0124] Furthermore, step S32 specifically includes the following sub-steps S321 to S324:

[0125] S321. In the tag semantic enhancement module, the tag summary representation of the tag semantic matrix is ​​projected into a modulation matrix.

[0126] Tag summary vector Projection as modulation matrix modulation matrix Specifically, it involves a nonlinear modulation representation of the tag summary semantics, used to guide the subsequent semantic fusion process, where the projection process is represented as:

[0127]

[0128] in, These are learnable parameters.

[0129] S322. Fuse the label attention output and modulation matrix to obtain the modulation weights in the label dimension.

[0130] Use a gating mechanism to output label attention and tag modulation matrix The modulation weights on the label dimension are calculated through fusion. ,Right now:

[0131]

[0132] in, This represents the sigmoid function. All of these are learnable parameters.

[0133] S323. Based on the modulation weight fusion of the label attention output and the modulation matrix, the label fusion is obtained.

[0134] The tag attention output and tag summary representation are fused by modulating weights, and the fused tag representation is calculated. :

[0135]

[0136] in, This indicates element-wise multiplication.

[0137] S324. Perform layer normalization on the tag attention output and tag fusion to obtain the tag semantic enhancement output.

[0138] By introducing residual connections and performing layer normalization, we obtain the final output representation of this layer:

[0139]

[0140] in, This represents the tag semantic enhancement output of the l-th layer tag semantic enhancement module.

[0141] In summary, the overall calculation formula for the tag semantic enhancement output of the tag semantic enhancement module is expressed as follows:

[0142] (twenty one)

[0143] S33. The semantic enhancement output of the label is transformed nonlinearly through a feedforward neural network to obtain the feedforward output.

[0144] Although the attention interaction and semantic enhancement in the previous stage have significantly improved the expressive power of the label representations, these operations mainly focus on interaction modeling at the label dimension. To further enhance the non-linear expressive power and feature transformation capabilities of each label representation, the decoding layer also introduces a feedforward neural network to perform bit-by-bit transformations on each label representation, enabling the model to have stronger representational power and the ability to handle complex semantic patterns.

[0145] The feedforward neural network in this embodiment adopts a standard two-layer feedforward structure to enhance the semantic output of the labels. A nonlinear transformation is performed to obtain the feedforward output of the feedforward neural network:

[0146]

[0147] in, This represents the feedforward output of the l-th layer feedforward neural network. , For learnable parameters, This is a bias term.

[0148] S34. The layer normalization operation is performed on the tag semantic enhancement output and feedforward output through the connection module to obtain the decoding output.

[0149] To stabilize model recursion, prevent gradient vanishing, and preserve original information, residual connections and layer normalization operations are introduced, ultimately updating... The decoded output of the layer is represented as ,Right now:

[0150]

[0151] So, after After the recursion, the final output label of the decoder is denoted as: As the final semantic embedding of the tag, that is:

[0152]

[0153] To achieve label-level prediction, this invention sets up a shared linear prediction layer for each label location, maps it to the category space, and calculates the prediction probability.

[0154] For the i-th label, the predicted output is obtained through linear transformation and the Sigmoid activation function:

[0155]

[0156] in, , These are learnable parameters.

[0157] The medical image multi-label classification method provided by this invention can be widely applied to intelligent medical image-assisted diagnostic systems, especially suitable for automatic label recognition and report generation scenarios of medical images such as CT and MRI. Taking lung CT screening as an example, doctors usually make a preliminary diagnosis of the images through the image reading system and manually select several prominent diagnostic labels, such as "pulmonary nodules," "calcifications," and "emphysema," and enter them into the electronic medical record system. However, in clinical practice, these manual labels are often incomplete or selectively omitted, resulting in some potential lesions not being covered, affecting subsequent data modeling and auxiliary analysis. This invention introduces a K-nearest neighbor attention mechanism to construct a reliable label distribution, which can adaptively capture the semantic associations between labels. At the same time, based on an encoder-decoder architecture, the target multi-label classifier achieves unified optimization of semantic modeling and label prediction in an end-to-end framework, thereby effectively avoiding error propagation and reducing dependence on strong priors, exhibiting stronger robustness and generalization ability in high-noise, weakly labeled medical image environments.

[0158] Please see Figure 3The present invention also provides a medical image multi-label classification system, the system comprising:

[0159] The training set and trusted label distribution acquisition unit 201 is used to acquire the training set and determine the trusted label distribution of each training sample in the training set based on the K-nearest neighbor label attention mechanism.

[0160] The target multi-label classifier generation unit 202 is used to train a preset multi-label classifier through the training set and the reliable label distribution of each training sample to obtain the target multi-label classifier.

[0161] The prediction label distribution calculation unit 203 is used to acquire the medical image to be predicted and input the medical image to be predicted into the target multi-label classifier. In the target multi-label classifier, the encoder extracts the label semantics based on the medical image to be predicted to obtain the label semantic matrix. The decoder performs label interaction enhancement operation based on the label semantic matrix to obtain the label semantic embedding. The linear prediction layer performs linear prediction based on the label semantic embedding to generate the prediction label distribution of the medical image to be predicted.

[0162] This invention provides a medical image multi-label classification system that can be deployed as an edge AI module or an access-based cloud server. It receives raw CT images and their initial labels from a hospital's Picture Archiving and Communication System (PACS) as input. First, features are extracted from the input images, and high-dimensional semantic vectors are obtained using a pre-trained convolutional neural network (such as ResNet50 or DenseNet121). In the label modeling stage, an encoder module maps image semantics to a label space, and a multi-layer attention mechanism in the decoder, along with a label semantic enhancement module, establishes association patterns between labels. The final output is a set of label suggestions based on a confidence ranking of the predicted label distribution. This suggestion can automatically complete potentially missing but relevant labels, such as "bronchial wall thickening," "fibrosis," or "mediastinal lymph node enlargement," significantly improving label comprehensiveness and accuracy. Doctors can then view the system's suggested label completion results on the image workstation interface, and after manual confirmation, can adopt them with a single click to generate a structured report. Furthermore, when combined with an electronic medical record system, the model output can be used to assist in symptom and diagnostic coding (such as ICD-10 mapping) for business processes such as medical insurance review, medical record archiving, or preoperative preparation.

[0163] In practical applications, this system can be deployed on NVIDIA Jetson series edge devices or run on dedicated hospital servers (such as compute nodes equipped with Tesla T4 or RTX 6000 processors), supporting real-time processing and LAN access to meet the data privacy and low latency requirements of medical scenarios. The entire system can be implemented using Python + PyTorch, and the front-end interface can be integrated with existing hospital systems via WebSocket or DICOM interfaces.

[0164] By deploying the provided medical image labeling-assisted diagnostic tools, we can effectively solve problems such as missing labels, ignored semantic relationships, and high prediction uncertainty in medical image annotation, improve the quality of structured reports and doctors' work efficiency, and provide a more robust labeling foundation for medical data mining and decision support.

[0165] The present invention also provides a computer device, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of any of the above-described medical image multi-label classification methods.

[0166] The present invention also provides a computer-readable storage medium having a computer program / instructions stored thereon, which, when executed by a processor, implements the steps of any of the above-described medical image multi-label classification methods.

[0167] The present invention also provides a computer program product, including a computer program / instructions that, when executed by a processor, implement the steps of any of the above-described medical image multi-label classification methods.

[0168] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0169] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection between apparatuses or units through some interfaces, and may be electrical, mechanical, or other forms.

[0170] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0171] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0172] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0173] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A medical image multi-label classification method, characterized in that, The method includes: Obtain the training set and determine the reliable label distribution of each training sample in the training set based on the K-nearest neighbor label attention mechanism; A preset multi-label classifier is trained using the training set and the reliable label distribution of each training sample to obtain the target multi-label classifier; A medical image to be predicted is acquired and input into the target multi-label classifier. In the target multi-label classifier, the encoder extracts the label semantics based on the medical image to be predicted to obtain a label semantic matrix. The decoder performs label interaction enhancement operations based on the label semantic matrix to obtain the label semantic embedding. The linear prediction layer performs linear prediction based on the label semantic embedding to generate the predicted label distribution of the medical image to be predicted.

2. The medical image multi-label classification method according to claim 1, characterized in that, The training set includes medical image training samples with real labels and noisy labels; the step of determining the reliable label distribution of each training sample in the training set based on the K-nearest neighbor label attention mechanism includes: Based on the K-nearest neighbor label attention mechanism, the neighbor set and the candidate label set of each training sample in the training set are determined; Calculate the similarity between each training sample and each neighbor sample in its neighbor set, and determine the corresponding weighting weight based on the similarity of the neighbor samples. The reliable label distribution of each training sample is calculated based on the weighted weights of all the neighboring samples corresponding to each training sample and their candidate label sets.

3. The medical image multi-label classification method according to claim 1, characterized in that, The step of extracting label semantics from the medical image to be predicted using an encoder to obtain a label semantic matrix includes: In the encoder, the medical image to be predicted is convolved by a convolutional layer to obtain multiple convolutional sequences; Max pooling is performed on multiple convolutional sequences to obtain multiple pooled features; The multiple pooling features are mapped to the label semantic space using a linear mapping to obtain multiple label semantic vectors; By concatenating all the aforementioned label semantic vectors, the label semantic matrix of the medical image to be predicted is obtained.

4. The medical image multi-label classification method according to claim 1, characterized in that, The decoder includes multiple decoding layers connected in sequence, and each decoding layer includes a single-head attention interaction module, a label semantic enhancement module, a feedforward neural network, and a connection module connected in sequence. In each of the decoding layers, the single-head attention interaction module performs attention interaction operations based on the input features to obtain the label attention output; based on the label summary representation of the label semantic matrix, the label semantic enhancement module performs label semantic enhancement operations based on the label attention output to obtain the label semantic enhancement output; and the feedforward neural network performs a nonlinear transformation on the label semantic enhancement output to obtain the feedforward output. The connection module performs layer normalization on the tag semantic enhancement output and the feedforward output to obtain the decoded output.

5. The medical image multi-label classification method according to claim 4, characterized in that, The tag summary representation based on the tag semantic matrix, and the tag semantic enhancement module performing tag semantic enhancement operations based on the tag attention output to obtain the tag semantic enhancement output, include the following steps: In the tag semantic enhancement module, the tag summary representation of the tag semantic matrix is ​​projected as a modulation matrix; By fusing the label attention output and the modulation matrix, the modulation weights in the label dimension are obtained; Based on the modulation weights, the label attention output and the modulation matrix are fused to obtain label fusion; A layer normalization operation is performed on the label attention output and the label fusion to obtain the label semantic enhancement output.

6. The medical image multi-label classification method according to claim 1, characterized in that, The step of training a preset multi-label classifier using the training set and the reliable label distribution of each training sample to obtain the target multi-label classifier includes: A preset multi-label classifier is trained using training samples from the training set, and the predicted label distribution of the training samples is output. The model parameters of the preset multi-label classifier are adjusted according to the predicted label distribution and the reliable label distribution of the training samples until the loss function of the preset multi-label classifier meets a preset threshold, thereby generating a target multi-label classifier. The loss function is expressed as: In the formula: This represents the loss function of a predefined multi-label classifier; This represents the total number of training samples in the training set. This represents the prediction confidence weighting function. Indicates the first The distribution of predicted labels for each training sample; Indicates the first The distribution of reliable labels for each training sample.

7. A medical image multi-label classification system, characterized in that, The system includes: The training set and trusted label distribution acquisition unit is used to acquire the training set and determine the trusted label distribution of each training sample in the training set based on the K-nearest neighbor label attention mechanism. The target multi-label classifier generation unit is used to train a preset multi-label classifier through the training set and the reliable label distribution of each training sample to obtain the target multi-label classifier. The predicted label distribution calculation unit is used to acquire the medical image to be predicted and input the medical image to be predicted into the target multi-label classifier. In the target multi-label classifier, the encoder extracts the label semantics based on the medical image to be predicted to obtain the label semantic matrix. The decoder performs label interaction enhancement operation based on the label semantic matrix to obtain the label semantic embedding. The linear prediction layer performs linear prediction based on the label semantic embedding to generate the predicted label distribution of the medical image to be predicted.

8. A computer device comprising a memory, a processor, and a computer program stored in the memory, characterized in that, The processor executes the computer program to implement the steps of the medical image partial multi-label classification method as described in any one of claims 1-6.

9. A computer-readable storage medium having a computer program / instructions stored thereon, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of the medical image partial multi-label classification method as described in any one of claims 1-6.

10. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of the medical image partial multi-label classification method as described in any one of claims 1-6.

Images