Image recognition model generation device and method

Abstract

The application provides a kind of generation device and method of image recognition model, it is related to image recognition model technical field, the application first constructs initial model, including sequentially connected anatomic perception module, sign discovery module and evidence reasoning module;Integrate credibility prediction sub-network in evidence reasoning module to calculate the credibility score vector of sign feature vector;Sample image is associated with three types of annotations of pixel level, area level and image level, and a training dataset is constructed;Adopt multi-task joint training mode, and total loss function includes anatomical segmentation loss, sign recognition loss, disease classification loss and credibility prediction auxiliary loss;Save model parameters after training, and obtain medical image recognition model.The application can output accurate disease classification results, and provide clear diagnostic basis, while improving the robustness and explainability of model decision through credibility evaluation, suitable for medical image assisted diagnosis and other scenarios.

Description

Technical Field

[0001] This invention relates to the field of image recognition model technology, specifically to an image recognition model generation device and method. Background Technology

[0002] In the field of medical image analysis, especially in computer-aided diagnostic systems, how to construct an intelligent model that can provide both accurate disease diagnosis and clear, reliable diagnostic evidence has always been a focus of clinical practice and scientific research. With the rapid development of deep learning technology, end-to-end classification models based on convolutional neural networks have achieved significant results in disease screening and classification tasks for various images such as chest X-rays, fundus ultrasound, and brain MRI. These models typically take the entire image as input and directly output the disease probability. However, these black-box models have significant limitations: firstly, their decision-making process lacks interpretability; doctors cannot know which specific regions or pathological features in the image the model bases its judgment on, which is unacceptable in serious medical decision-making; secondly, the model lacks robustness in assessing noise, artifacts, or annotation uncertainties in the image, failing to inform doctors which evidence in their diagnostic conclusions is reliable and which may be questionable, thus affecting the trustworthiness of clinical applications.

[0003] To enhance model interpretability, existing technologies have developed a class of methods based on a discovery and inference paradigm. These methods typically begin by using a segmentation or detection network to locate potentially disease-related anatomical regions or visual concepts (i.e., signs), and then classify the disease based on the features of these discovered regions. For example, some methods first segment pulmonary nodules, retinal hemorrhage areas, etc., and then extract features from these regions to determine benignity / malignancy or disease grading. However, these methods still face the following technical challenges: First, most methods focus only on image-level classification accuracy, often treating sign identification as an intermediate step. They lack fine-grained modeling and supervision of the existence of signs themselves and their quantitative attributes (such as size, density, and morphological scores), leading to a semantic gap between the discovered evidence and the clinical signs that radiologists are concerned with. Second, existing methods typically assume that all sign features discovered by the model have equal reliability. However, in reality, due to differences in image quality, anatomical variations, or segmentation errors, the model's confidence in extracting features from different regions varies. Ignoring this difference can affect the robustness of the final decision. Finally, model training heavily relies on high-quality labeled data, while the labeling cost of medical images is extremely high and often involves inter-observer variability. How to efficiently perform joint training using multi-level annotations (pixel-level segmentation, region-level feature description, and image-level diagnosis) that may contain uncertainties, and enable the model to learn to assess the credibility of the evidence it extracts, is a problem that current technologies have not yet systematically solved. Therefore, there is an urgent need for a new model generation method that can achieve robust reasoning based on the credibility of evidence within an integrated framework, and output diagnostic reports that are both highly interpretable and self-aware of reliability.

[0004] The information disclosed in the background section is only intended to enhance the understanding of the background of this disclosure, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0005] The purpose of this invention is to provide an image recognition model generation apparatus and method to solve the problems mentioned in the background art.

[0006] To achieve the above objectives, the present invention provides the following technical solution:

[0007] An image recognition model generation apparatus, comprising:

[0008] The initial model building module is used to build an initial model, which includes three modules connected in sequence: an anatomical perception module, a sign detection module, and an evidence reasoning module. The anatomical perception module is used to output an anatomical structure segmentation probability map of the sample image. The sign detection module generates sign feature vectors for a predefined target anatomical region based on the probability map. The evidence reasoning module is used to integrate all sign feature vectors to output the probability distribution of the target disease classification.

[0009] The credibility integration module is used to integrate the credibility prediction subnetwork into the evidence reasoning module to calculate the credibility score vector of the sign feature vector of each target anatomical region to characterize the reliability of the sign feature vector in its sample images.

[0010] The multi-class annotation association module is used to associate each sample image with three types of annotations: pixel-level annotations, region-level annotations, and image-level annotations. The sample image is used as input, the three types of annotations are used as supervision annotations, and the outputs of the three modules are used as labels to form the training dataset.

[0011] The multi-task joint training module is used to train the initial model using the training dataset. The total loss function during the training process includes anatomical segmentation loss, sign recognition loss, disease classification loss, and confidence prediction auxiliary loss.

[0012] The model application and diagnosis module is used to save the parameters of the initial model after training to obtain a medical image recognition model; input the image to be recognized into the medical image recognition model, diagnose the disease category of the image to be recognized according to the probability distribution of the target disease classification, and output the relevant diagnostic basis simultaneously.

[0013] Furthermore, the anatomical perception module is configured as a medical image semantic segmentation network, which classifies each pixel in the input sample image into a predefined anatomical structure category, thereby outputting an anatomical structure segmentation probability map of the same size as the input image; each pixel in the anatomical structure segmentation probability map corresponds to a vector, representing the probability that the pixel belongs to each anatomical structure category;

[0014] The medical image semantic segmentation network adopts an encoder and decoder structure. The encoder consists of multiple downsampling layers and convolutional layers, which are used to progressively extract multi-scale feature maps of the input sample image and reduce their spatial size. The decoder consists of multiple upsampling layers and convolutional layers, and is fused with the multi-scale feature maps of the corresponding layers of the encoder through skip connections, which are used to progressively restore the spatial size of the multi-scale feature maps and finally output the anatomical structure segmentation probability map.

[0015] The sign detection module is used to process one or more specific target anatomical regions; for each target anatomical region, the sign detection module performs the following operations:

[0016] From the anatomical structure segmentation probability map, extract the single-channel probability map corresponding to the target anatomical region as the probability channel of the target anatomical region; where each probability value in the probability channel represents the predicted probability that the corresponding pixel in the input image belongs to the target anatomical region; set the probability threshold of the target anatomical region, modify the probability values ​​above the probability threshold to 1, and modify them to 0, so as to convert the probability channel into a binary mask map of the target anatomical region;

[0017] A binarized mask image is used to spatially attention-weight the multi-scale feature map to obtain a weighted multi-scale feature map. Cross-scale fusion and spatial pooling are then performed on the weighted multi-scale feature map to generate a depth feature vector of the target anatomical region. The depth feature vector is then input into a fully connected network for analysis to obtain a sign feature vector. Each dimension of the feature vector corresponds to a predefined sign related to the target anatomical region, and its value represents the presence intensity of the corresponding sign.

[0018] The evidence reasoning module includes a feature fusion network, whose input is the sign feature vector of all target anatomical regions output by the sign discovery module; the feature fusion network is used to integrate the sign feature vectors used and output a global probability distribution of the target disease classification.

[0019] When calculating the probability distribution of the target disease classification, the feature fusion network receives the confidence score vectors of each symptom feature vector from the confidence prediction subnetwork, and converts the confidence scores into fusion weights to perform weighted fusion on the corresponding symptom feature vectors to obtain the final probability distribution.

[0020] Furthermore, the confidence prediction sub-network is independent of the feature fusion network; each sub-network takes the feature vector of a single target anatomical region as input, processes it through a fully connected network with the Sigmoid function as the activation layer, and outputs a confidence score vector with the same dimension as the input feature vector; the score of each dimension of the confidence score vector corresponds to the confidence of a feature of the target anatomical region.

[0021] Furthermore, the specific annotation process for the three types of annotations is as follows:

[0022] The annotators annotate each sample image in the training dataset;

[0023] The pixel-level annotation refers to assigning an anatomical structure category label to each pixel in the sample image; through the anatomical structure category labels of each pixel, several anatomical regions are divided, generating a segmentation annotation map with the same spatial size as the sample image;

[0024] Based on the anatomical regions defined by the segmentation and annotation map, the target anatomical region to be analyzed is identified. The region-level annotation process for the target anatomical region to be analyzed is as follows:

[0025] For a predefined sign related to the target anatomical region, determine whether it is within the target anatomical region and generate a binary existence label, where 1 indicates existence and 0 indicates non-existence.

[0026] For a symptom that is determined to exist, a numerical or classification level score is generated based on the symptom type. The symptom type is divided into objective measurement type and subjective evaluation type. For objective measurement type symptom, at least numerical parameters are calculated based on image grayscale, morphological or texture features. For subjective evaluation type symptom, the annotator makes a judgment and gives at least an ordered classification level score.

[0027] For a target anatomical region, a sign annotation vector is finally generated, which consists of all the signs present in the target anatomical region and their measurement values.

[0028] Image-level annotation is based on the known disease categories in the sample image and associates them with the sign annotation vector of the target anatomical region, using the known disease categories in the sample image as the disease diagnosis labels of the sign annotation vector.

[0029] Furthermore, the credibility prediction subnetwork is constrained by an auxiliary supervision signal during training; this auxiliary supervision signal comes from the self-quality score of the feature whose binarized existence label is 1 in the regional annotation; the learning objective of the credibility prediction subnetwork is to make the difference between its output credibility score vector and the self-quality score of the corresponding feature within the deviation range.

[0030] For features with a binarized existence label of 1, a self-quality score reflecting the reliability of their annotation is calculated, as follows:

[0031] The annotation process is completed independently by multiple annotators, and their own quality score is equal to the proportion of all annotators who determine that the phenomenon exists out of the total number of annotators.

[0032] The sample images that have completed the above three types of annotations are randomly divided into non-overlapping training sets, validation sets, and test sets according to a preset ratio; all sample images are subjected to standardized preprocessing operations, the preprocessing of which includes at least: normalizing pixel values ​​to a fixed range, and resampling or cropping the image size to a uniform size.

[0033] Furthermore, during the training process, each sample image in the training set is input into the initial model to obtain the anatomical structure segmentation probability map, the sign feature vector of each target anatomical region, the probability distribution of the target disease classification, and the confidence score vector of each sign; the segmentation annotation map, sign annotation vector, disease diagnosis label, and the sign's own quality score are used as target values, and compared with the corresponding output of the model to calculate the loss values ​​of each item.

[0034] Using the anatomical structure segmentation probability map output by the anatomical perception module as the predicted value and the segmentation annotation map generated by pixel-level annotation as the target value, the loss is calculated; the anatomical segmentation loss consists of two parts, expressed by the formula:

[0035]

[0036] in, Indicates loss due to anatomical segmentation; This represents the Dice similarity coefficient loss, used to optimize the overall segmentation overlap of the anatomical region; These are the weighting coefficients for the corresponding items; The weighted cross-entropy loss;

[0037] The formula for calculating the Dice similarity coefficient loss is as follows:

[0038]

[0039] in, This represents the probability that the i-th pixel in the anatomical structure segmentation probability map belongs to the c-th anatomical structure category; This represents the binary existence label of the i-th pixel in the segmentation annotation image, indicating that it belongs to the c-th anatomical structure category. This is a preset minimum value used to prevent the denominator from being zero; i is the pixel index, N is the number of pixels; c is the index of the anatomical structure category, and C is the number of anatomical structure categories;

[0040] The weighted cross-entropy loss is used to optimize the classification of each pixel, mitigating the impact of the uneven number of pixels of different anatomical structures in the image. Its calculation formula is as follows:

[0041]

[0042] in, The positive weighting coefficients assigned to the c-th anatomical structure category;

[0043] For each target anatomical region, the sign recognition loss is calculated using the sign feature vector output by the sign detection module as the predicted value and the sign annotation vector of that target anatomical region as the target value. The sign recognition loss is calculated separately for the presence and measurement value of the sign, as follows:

[0044]

[0045] in, This indicates a loss in sign recognition; The presence loss is represented by the binary cross-entropy loss function, which is used to calculate the binary presence labels of all features in the feature annotation vector. These are the weighting coefficients for the corresponding items; The loss represents the symptom measurement value, which is calculated only for symptoms with a presence label of 1. For objectively measured symptoms, a smoothed L1 loss function is used for calculation. For subjectively evaluated symptoms, the metric is an ordered classification rank score, and a weighted ordinal regression loss function is used.

[0046] The disease classification loss is calculated using the standard multi-class cross-entropy loss function. The probability distribution of the target disease category output by the evidence reasoning module is used as the predicted value. The probability distribution of the target disease category is represented as a probability distribution vector, where each element represents the probability of belonging to the k-th target disease category. Here, k represents the index of the target disease category. The disease diagnosis label is converted into a one-hot encoded vector with the same dimension as the probability distribution vector, where each element is encoded with 0 and 1, and there is exactly one element that is 1, indicating that the element corresponds to the disease diagnosis label. The one-hot encoded vector is used as the target value to calculate the disease classification loss.

[0047] The calculation logic of the confidence prediction auxiliary loss is as follows: using the confidence score vector of the feature vector of each target anatomical region output by the confidence prediction sub-network as the prediction value, and the quality score of the sign itself as the target value, the confidence prediction auxiliary loss is calculated; the confidence prediction auxiliary loss adopts the mean squared error loss function, and its calculation is only for signs with a binarized existence label of 1.

[0048] The total loss function during training is obtained by weighted summation of the losses mentioned above, and the calculation formula is as follows:

[0049]

[0050] in, This represents the total loss function value; For disease classification loss value; To predict the auxiliary loss value for credibility; These are the weighting coefficients of their respective terms in the total loss function.

[0051] Furthermore, once the initial model meets the preset convergence conditions, the training of the initial model is considered complete, and all parameters of the anatomical perception module, the sign detection module, the evidence reasoning module, and the credibility prediction sub-network are solidified and saved as the final medical image recognition model.

[0052] The medical image recognition model defines a standardized structured output format; for any new input image to be recognized, after analysis by the medical image recognition model, the output includes relevant diagnostic criteria comprising the following four core data components:

[0053] Based on the probability distribution of the target disease classification output by the feature fusion network, the disease category corresponding to the highest probability value is taken as the disease category diagnosed in the current image to be identified.

[0054] Output a list of target anatomical regions, which includes at least: the name of the anatomical structure category identified by the anatomical perception module, the boundary of the target anatomical region in the image to be identified, all features in the target anatomical region with a binarized existence label of 1, and the corresponding measurement value of each feature; the boundary of the target anatomical region in the image to be identified is obtained through a segmentation annotation map.

[0055] Output the confidence score vector for each feature vector predicted by the confidence prediction subnetwork.

[0056] The present invention also provides a method for generating an image recognition model, wherein the method for generating an image recognition model is executed by the aforementioned image recognition model generating device, and the specific steps include:

[0057] Step 1: Construct an initial model, which includes three modules connected in sequence: an anatomical perception module, a sign detection module, and an evidence reasoning module. The anatomical perception module is used to output the anatomical structure segmentation probability map of the sample image. The sign detection module generates sign feature vectors for predefined target anatomical regions based on the probability map. The evidence reasoning module is used to integrate all sign feature vectors to output the probability distribution of the target disease classification.

[0058] Step 2: Integrate a credibility prediction subnetwork into the evidence reasoning module to calculate the credibility score vector of the sign feature vector for each target anatomical region, so as to characterize the reliability of the sign feature vector in its sample images.

[0059] Step 3: Associate each sample image with three types of annotations: pixel-level annotations, region-level annotations, and image-level annotations; using the sample image as input, the three types of annotations as supervision annotations, and the outputs of the three modules as labels, to form the training dataset;

[0060] Step 4: Train the initial model using the training dataset. The total loss function during training includes anatomical segmentation loss, sign recognition loss, disease classification loss, and confidence prediction auxiliary loss.

[0061] Step 5: After training is completed, save the parameters of the initial model to obtain the medical image recognition model; input the image to be recognized into the medical image recognition model, diagnose the disease category of the image to be recognized according to the probability distribution of the target disease classification, and output the relevant diagnostic basis simultaneously.

[0062] The technical effects and advantages provided by the present invention in the above technical solution are as follows:

[0063] This invention introduces three cascaded modules: anatomical perception, sign detection, and evidence reasoning. Combined with multi-level supervision at the pixel, region, and image levels, the constructed initial model not only outputs accurate disease classification results but also simultaneously provides key anatomical regions supporting the decision, specific identified pathological signs, and their quantitative values. This transforms the diagnostic process from a black box to a white box, significantly enhancing the model's interpretability and clinical applicability, enabling physicians to understand and verify the model's judgment basis. Secondly, this invention innovatively integrates a credibility prediction sub-network into the evidence reasoning module, allowing the model to automatically assess the credibility of sign features extracted from different anatomical regions and weight them in the final disease probability fusion. This mechanism effectively improves the model's decision robustness in situations with poor image quality, ambiguous region segmentation, or atypical sign presentation. The model can then indicate to physicians the differences in reliability among different pieces of evidence in the diagnostic conclusion, enhancing the trust and safety of clinical applications. By designing a multi-task joint training framework, this invention can efficiently utilize multi-level medical annotation data containing a certain degree of uncertainty for end-to-end optimization. While improving the performance of each sub-task, it promotes collaboration between modules, ultimately resulting in an integrated medical image recognition system that is accurate in diagnosis, clear in its basis, and has assessable reliability. Attached Figure Description

[0064] Figure 1 This is a schematic diagram of the device structure of the present invention;

[0065] Figure 2 This is a schematic diagram of the overall method flow of the present invention. Detailed Implementation

[0066] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to specific embodiments.

[0067] It should be noted that, unless otherwise defined, the technical or scientific terms used in this invention should have the ordinary meaning understood by one of ordinary skill in the art to which this invention pertains. The terms "first," "second," and similar terms used in this invention do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Terms such as "comprising" or "including" mean that the element or object preceding the word encompasses the elements or objects listed following the word and their equivalents, without excluding other elements or objects. Terms such as "connected" or "linked" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. Terms such as "upper," "lower," "left," and "right" are used only to indicate relative positional relationships; when the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0068] Example:

[0069] Please see Figure 1 The present invention provides an image recognition model generation apparatus, comprising:

[0070] The initial model building module is used to build an initial model, which includes three modules connected in sequence: an anatomical perception module, a sign detection module, and an evidence reasoning module. The anatomical perception module is used to output an anatomical structure segmentation probability map of the sample image. The sign detection module generates sign feature vectors for a predefined target anatomical region based on the probability map. The evidence reasoning module is used to integrate all sign feature vectors to output the probability distribution of the target disease classification.

[0071] In this embodiment, the anatomical perception module is configured as a medical image semantic segmentation network. This network receives a sample image (e.g., a chest X-ray or a brain MRI slice) as input. Its function is to analyze each pixel in the input sample image and classify it into predefined anatomical structure categories. For example, in a chest X-ray, these categories might be "left lung," "right lung," "heart," or "mediastinum." The module's output is an anatomical structure segmentation probability map with the same spatial dimensions (height × width) as the input image. Each pixel in this probability map is no longer a single grayscale or RGB value, but a vector of length C (where C is the total number of predetermined anatomical structure categories). The c-th element in this vector represents the probability that the current pixel belongs to the c-th type of anatomical structure, and the sum of all C probability values ​​is 1. For example, for a pixel, its output vector may be [0.01, 0.02, 0.95, 0.02], which correspond to the probabilities of "background", "heart", "left lung" and "right lung" respectively, indicating that the model believes that the pixel belongs to the left lung tissue with a 95% confidence.

[0072] To achieve accurate segmentation, the medical image semantic segmentation network employs an encoder-decoder structure and is equipped with skip connections, such as U-Net or its variants. This structure can simultaneously utilize both deep semantic information and shallow detail information of the image.

[0073] An encoder consists of multiple stages, each typically containing a convolutional layer, an activation function (such as ReLU), and a downsampling layer (such as a max-pooling layer or a convolutional layer with a stride of 2). The encoder's purpose is to progressively extract multi-scale feature maps from the input image and reduce its spatial size. For example, an input 256x256 image, after passing through the encoder, may successively yield feature maps of different scales, such as 128x128, 64x64, 32x32, and 16x16. Deeper, smaller-scale feature maps contain high-level, category-related semantic information (such as "this is lung tissue"), while shallower, larger-scale feature maps contain more spatial details and edge information (such as the boundaries of the lungs).

[0074] The decoder also consists of multiple stages, each typically containing upsampling layers (such as transposed convolutions or interpolations) and convolutional layers. The decoder's goal is to progressively recover the spatial dimensions of the feature maps, ultimately outputting a segmentation probability map of the same size as the input. A key design feature is that each stage of the decoder uses skip connections to concatenate or add channels of the corresponding scale feature maps from the encoder. This allows the decoder to directly utilize the same-scale spatial information preserved by the encoder when upsampling to recover details, significantly improving the accuracy of segmentation boundaries. For example, when the decoder upsamples a 32x32 feature map to 64x64, it simultaneously receives a 64x64 feature map generated from the encoder stage. After fusing the two, a convolutional process is applied, utilizing both the high-level semantics of the deep features and the fine structure of the shallow features.

[0075] Before inputting sample images into the anatomical perception module, standardized preprocessing must be performed to ensure training stability and model generalization ability. Preprocessing operations include at least:

[0076] Pixel value normalization: The dynamic range of raw pixel values ​​in medical images (such as Hounsfield units in CT scans, grayscale values ​​in X-rays) can be very large. They need to be linearly normalized to a fixed range, such as [0,1] or [-1,1].

[0077] Spatial size standardization: Due to the diverse sources of medical images, their sizes may vary. All images need to be resampled or cropped to a uniform size (e.g., 256x256 pixels). Resampling (e.g., bilinear interpolation) is preferred to preserve complete anatomical information. If cropping is used, ensure that key anatomical structures are within the cropping area; a cropping strategy centered on specific anatomical points (e.g., the hilum) can be employed.

[0078] Data augmentation: To increase data diversity and prevent overfitting, data augmentation can be performed online during training, such as random rotation (small angle), translation, scaling, elastic deformation, and adding Gaussian noise.

[0079] The sign detection module serves as a bridge connecting anatomical information and pathological evidence. Based on the probability map output by the anatomical perception module, this module generates sign feature vectors with clear clinical semantics for one or more specific target anatomical regions, such as "left upper lobe" or "right middle lobe nodule region." For each predetermined target anatomical region (denoted as the r-th region), the sign detection module performs the following operations:

[0080] Probability Channel Extraction: From the anatomical structure segmentation probability map, extract a single-channel probability map corresponding to the target anatomical region, which serves as the probability channel for that target anatomical region. Each probability value in this channel represents the predicted probability that the corresponding pixel in the input image belongs to that target anatomical region. For example, if the target region is "left lung," then extract the probability value of each pixel in the probability map belonging to the "left lung" class to form a single-channel probability map. .

[0081] Thresholding generation mask: Sets a probability threshold for the target anatomical region. ,Will The probability value of each pixel in the middle and Comparison: If If a value is found to be 1, the corresponding position in the binary mask image is set to 1; otherwise, it is set to 0, thus converting the probability channel into a binary mask image of the target anatomical region. The binary mask image represents the pixels in the sample image that belong to the target anatomical region. Here, i represents the pixel index. This probability threshold is usually preset to 0.5, a commonly used intermediate value in binary classification problems. Its rationale is that the segmentation probability represents the confidence that a pixel belongs to a certain class, and 0.5 is the watershed for confidence. In practical applications, it can be fine-tuned based on the segmentation accuracy on the validation set (such as the Dice coefficient). For example, if the segmentation result is found to be too aggressive (containing too much background), the threshold can be appropriately increased (e.g., 0.6); if it is too conservative (missing some regions), the threshold can be appropriately decreased (e.g., 0.4).

[0082] The sign detection module shares the multi-scale feature maps extracted by the encoder of the anatomical perception module. Its core idea is to use the region mask obtained in the previous step to focus on the features related to that region.

[0083] The binarized mask image is upsampled (using bilinear interpolation) to the same spatial size as the feature map at each scale output by the encoder. The upsampled mask corresponding to the i-th scale is denoted as... .

[0084] For each scale of feature map, perform element-wise multiplication: the weighted feature map at scale I is equal to the original feature map at scale i. First-scale mask Among them, symbols This indicates that elements at corresponding positions are multiplied.

[0085] This operation is a hard spatial attention mechanism. It focuses feature extraction entirely on the l-th scale mask. The feature values ​​of pixels with a median value of 1 (i.e., the target anatomical region) are forced to zero or significantly suppressed at locations with a median value of 0 (background or non-target regions). This effectively eliminates the interference of irrelevant regions on subsequent sign analysis, achieving anatomical guidance for feature extraction.

[0086] The weighted multi-scale feature maps are fused across scales and spatially pooled to generate a depth feature vector for the target anatomical region. Specifically, weighted feature maps at different scales are fused. A typical method is feature pyramid fusion: first, the deep, small-scale feature map (containing high-level semantic information) is upsampled, and then it is added element-wise or concatenated along the channel dimension with the shallow, large-scale feature map (containing more spatial details). Global average pooling (GPA) is then performed on the fused feature map. This operation compresses the spatial dimension of the entire two-dimensional feature map, calculates the average value of all pixels in each feature channel, and finally generates a one-dimensional depth feature vector for the target anatomical region. The GPA operation can aggregate information from the entire target region and endow the model with a certain degree of translation invariance.

[0087] The deep feature vector is input into a fully connected network for analysis. The number of neurons in the final output layer of this network is equal to the predefined total number of clinical signs associated with the target anatomical region. The output of this layer is the sign feature vector, denoted as... Each dimension corresponds to a predefined sign related to the target anatomical region (for example, for the lung nodule region, signs may include "spiculation", "lobulation", "calcification", etc.), and its value represents the intensity of the presence of the corresponding sign. Let be the number of signs in the r-th target anatomical region. The output layer typically uses the Sigmoid activation function, which applies a sigmoid activation function to each anatomical region. The value constraint is within the (0,1) interval, which is easy to understand intuitively as the probability of existence or significance.

[0088] The evidence reasoning module is the decision center of the model, responsible for integrating symptom evidence from all target anatomical regions to classify the disease. This module includes a feature fusion network, whose input is the symptom feature vectors of all target anatomical regions output by the symptom detection module. This feature fusion network integrates the used symptom feature vectors to output a global probability distribution for the target disease classification.

[0089] When calculating the probability distribution of the target disease classification, the feature fusion network converts the confidence scores of each symptom feature vector into weights and performs weighted fusion of the symptom feature vectors to obtain the final probability distribution. Specifically, when integrating evidence, the feature fusion network does not treat all symptom features equally; it weights each symptom feature vector... Its corresponding overall confidence level (e.g., taking the average of the confidence score vectors to obtain a scalar confidence level). (Or use other aggregation methods) to associate them. Specifically, during the fusion process (e.g., before splicing or adding), each can be... Multiplied by a weight derived from its credibility. The function f can be a linear function, for example... It can also be a nonlinear function, such as (Softmax weighted). This indicates that... Indicates all target anatomical regions Summation is performed. This gives higher-confidence regional evidence greater weight in the decision-making process, while suppressing lower-confidence evidence, thus improving the model's robustness in situations with poor local quality or inaccurate segmentation. The weighted fused features are then processed by the final classifier (usually a fully connected layer with a softmax activation function), outputting a probability distribution for the target disease classification. ,in, The total number of target diseases (such as "health", "pneumonia", "lung cancer", etc.). This represents the probability that the model considers the input image to belong to the k-th target disease category, and all The sum is 1.

[0090] The credibility integration module is used to integrate a credibility prediction subnetwork into the evidence reasoning module to calculate the credibility score vector of the sign feature vector of each target anatomical region to characterize the reliability of the sign feature vector in its sample images.

[0091] In this embodiment, the credibility prediction subnetwork is structurally independent of the feature fusion network. This means it has its own independent parameters, and its learning objective is both related to and different from the disease classification task. Its task is not to directly predict the disease, but to evaluate the quality of the evidence.

[0092] This subnetwork takes a single feature vector as input and outputs a confidence score vector with all dimensions ranging from [0,1] through a fully connected network with a sigmoid function as the activation layer. The score for each dimension of the confidence score vector corresponds to the confidence level of a feature in the target anatomical region. A score closer to 1 indicates that the model is more confident in the reliability of the feature's recognition; a score closer to 0 indicates that the feature may originate from image noise, blurred boundaries, or uncertainty in the model's segmentation of the region.

[0093] The confidence prediction subnetwork is preferably a lightweight fully connected neural network (MLP) to ensure efficiency and prevent overfitting. In this embodiment, the input layer is the received symptom feature vector. One or more hidden layers are configured to learn the complex nonlinear relationship between symptom features and reliability. For example, it could be a [structure / mechanism]... A fully connected layer with 2 neurons and using the ReLU activation function. The output layer is a... A fully connected layer with 10 neurons must have the sigmoid function as its activation function. The sigmoid function compresses the output of each neuron to the range (0,1), perfectly matching the range requirement of the confidence score.

[0094] Besides simple MLPs, other architectures can be chosen based on task complexity, such as MLPs with attention mechanisms. Adding a self-attention mechanism after the hidden layers allows the network to consider the features of other features when evaluating the credibility of a particular feature, simulating the impact of correlation between features on credibility. Bayesian Neural Networks (BNNs) treat the network weights as a distribution rather than fixed values, and their outputs naturally contain a measure of uncertainty, which can be directly used as a credibility score. However, BNNs typically have higher computational costs.

[0095] Regardless of the network structure used, network weights are typically initialized using Xavier uniform initialization or He initialization, with biases initialized to 0 to ensure stability and convergence speed in the early stages of training.

[0096] The multi-class annotation association module is used to associate each sample image with three types of annotations: pixel-level annotation, region-level annotation, and image-level annotation. The sample image is used as input, the three types of annotations are used as supervision annotations, and the outputs of the three modules are used as labels to form the training dataset.

[0097] In this embodiment, the specific annotation process for the three types of annotations is as follows:

[0098] The annotation work was completed by experts with medical imaging diagnostic qualifications (annotators) on a dedicated annotation platform. Each sample image was associated with these three types of annotations, forming a complete annotation group. The annotation of each sample image in the training dataset by the annotators specifically included:

[0099] The pixel-level annotation refers to assigning a label of an anatomical structure category to each pixel in the sample image. This is the lowest level of supervision used to train the anatomical perception module.

[0100] The annotator uses tools such as a brush and a polygonal lasso to precisely outline each predefined anatomical structure (e.g., left lung, right lung, heart) on the image. The annotation system assigns a corresponding category ID to each outlined pixel (e.g., 1 for left lung, 2 for right lung, 3 for heart, 0 for background). Finally, a segmentation mask with the exact same spatial dimensions as the original sample image is generated, where the value of each pixel is its anatomical category label. This mask is a single-channel integer matrix.

[0101] By labeling the anatomical structure category of each pixel, several anatomical regions were divided, generating a segmentation annotation map with the same spatial size as the sample image.

[0102] Region-level annotation is performed based on the anatomical regions generated by pixel-level annotation and serves as the direct source of supervision signals for the sign detection module and the confidence prediction subnetwork. The process is performed one by one for the target anatomical region to be analyzed. Specifically, based on the anatomical regions defined by the segmentation annotation map, the target anatomical region to be analyzed (e.g., the segmented "left upper lobe" region) is identified. For the target anatomical region to be analyzed, the region-level annotation process is as follows:

[0103] For predefined signs related to the target anatomical region (e.g., for the lung region, the list might include "ground-glass opacity," "consolidation," "nodules," "reticular opacity," "pleural effusion," etc.), determine whether they are within the target anatomical region and generate a binary presence label, where 1 indicates presence and 0 indicates absence. The predefined signs need to be predefined by clinical experts based on key signs for diagnosing the target disease; they serve as a bridge connecting image features and clinical semantics.

[0104] For any identified sign, a numerical or categorical rating is generated based on the sign type. Sign types are categorized as objective measurement and subjective assessment. For objective measurement signs, numerical parameters are calculated based on at least image grayscale, morphological, or textural features. For example, regarding nodule size, for a nodule sign, the annotator clicks on the center of the nodule or outlines its approximate boundary on the image, and the system automatically calculates its maximum diameter, volume (3D), and average CT value (HU), among other numerical parameters. For example, regarding ground-glass opacity, for a ground-glass opacity region, the annotator outlines the region, and the system automatically calculates the ratio of the average grayscale value of that region to the average grayscale value of normal lung tissue. Finally, a numerical metric is generated for the sign (e.g., "Maximum diameter: 8.2 mm").

[0105] For signs assessed subjectively, the annotator makes a judgment and assigns at least one ordered classification level score. Its attributes depend on the physician's subjective experience assessment. For example, regarding the degree of consolidation, there is no absolutely objective single numerical value for consolidation; three physicians can assess its severity. The annotator, based on experience, selects one of the ordered classification levels, such as "1-mild," "2-moderate," and "3-severe." Ultimately, an ordered classification level score is generated as a metric for the sign.

[0106] For a target anatomical region, the "presence labels" and "measures" of all its predefined features are organized in a fixed order to form the feature annotation vector for that region. For features that do not exist, their measure fields can be set to 0 or None and are ignored when calculating the training loss. For example, suppose the predefined features of the lung region are: [Ground-glass opacity (GGO), Constipation (Cons), Nodule (Nod)]. An image annotation result is: GGO present (mild), Constipation absent, Nod present (diameter 12mm). Then its feature annotation vector may be represented as: [Presence: 1, 0, 1; Measure: 1 (grade), 0, 12.0 (mm)].

[0107] Image-level annotation, based on known disease categories in the sample images, associates them with the sign annotation vectors of the target anatomical region, using the known disease categories in the sample images as the disease diagnostic labels for the sign annotation vectors. These labels typically come from the patient's electronic medical records, pathology reports, or the final formula from an expert panel. This constitutes the ultimate supervised target for disease classification in the evidence reasoning module. It is usually an integer category ID (e.g., 0: healthy, 1: pneumonia, 2: lung cancer) or a one-hot encoded vector.

[0108] In this embodiment, the credibility prediction subnetwork is constrained by an auxiliary supervision signal during training; the auxiliary supervision signal comes from the self-quality score of the feature whose binarized existence label is 1 in the regional annotation; the learning objective of the credibility prediction subnetwork is to make the difference between its output credibility score vector and the self-quality score of the corresponding feature within the deviation range.

[0109] For features with a binarized existence label of 1, a self-quality score reflecting the reliability of their annotation is calculated, as follows:

[0110] The annotation process is completed independently by multiple annotators. The self-quality score is equal to the proportion of annotators who correctly identified the presence of the feature out of the total number of annotators. For example, if five doctors independently examine a region on the same image, and three identify the presence of a "ground-glass opacity" feature while two do not, the self-quality score is 3 / 5, or 0.6. This score directly reflects the inter-observer consistency of the feature; higher consistency and closer scores indicate a clearer and easier-to-identify feature. It directly utilizes the inherent observer differences in medical image annotation as a measure of uncertainty, providing natural and clinically meaningful data.

[0111] The sample images with the above three types of annotations are randomly divided into non-overlapping training, validation, and test sets according to a preset ratio. A common ratio is 70% training set, 15% validation set, and 15% test set. The specific ratio needs to be adjusted according to the total amount of data to ensure that each subset, especially the validation and test sets, has enough data to reliably evaluate performance. All sample images undergo standardized preprocessing operations, including at least: normalizing pixel values ​​to a fixed range and resampling or cropping the image size to a uniform size. Pixel-level preprocessing (normalization and resampling) has already been completed; this section mainly describes the preprocessing of the labeled data:

[0112] Synchronous Transformation of Segmentation Annotations: When the original image undergoes spatial resizing (resampling / cropping), its corresponding segmentation annotations must be processed synchronously using the exact same geometric transformation parameters (and interpolation methods). Key point: The segmentation annotations should be resampled using nearest neighbor interpolation to prevent non-integer confusion values ​​from being generated in the interpolation process for the class labels.

[0113] Coordinate correction of feature annotation vectors: If the image is cropped and some target anatomical regions are partially or completely outside the cropped area, the affected region-level annotations need to be updated or removed.

[0114] Unified data format: Organize pixel-level labeled images, region-level symptom label vectors (and their own quality scores), and image-level disease labels into a unified format that is easy for deep learning frameworks (such as PyTorch and TensorFlow) to read. For example, a custom Dataset class can be used to return a dictionary or tuple containing all elements when loaded.

[0115] The multi-task joint training module is used to train the initial model using the training dataset. The total loss function during the training process includes anatomical segmentation loss, sign recognition loss, disease classification loss, and confidence prediction auxiliary loss.

[0116] In this embodiment, supervised learning is used for training. In each iteration, a batch of data is taken from the training set, and each sample image is input into the initial model to obtain an anatomical structure segmentation probability map, feature vectors of each target anatomical region, probability distribution of the target disease classification, and confidence score vectors for each sign. The segmentation annotation map, sign annotation vectors, disease diagnosis labels, and the sign's own quality score are used as target values, compared with the model's corresponding outputs, and various loss values ​​are calculated. The essence of training is to continuously adjust all learnable parameters of the model through the backpropagation algorithm to minimize the difference between the predicted value and the supervised target.

[0117] Using the anatomical structure segmentation probability map output by the anatomical perception module as the predicted value and the segmentation annotation map generated by pixel-level annotation as the target value, the loss is calculated; the anatomical segmentation loss consists of two parts, expressed by the formula:

[0118]

[0119] in, Indicates loss due to anatomical segmentation; This represents the Dice similarity coefficient loss, used to optimize the overall segmentation overlap of the anatomical region; These are the weighting coefficients for the corresponding items; This is the weighted cross-entropy loss.

[0120] Weighting coefficient Used for balance and The relative importance of the two losses. They are typically initially set to 1.0, indicating equal importance. This can be fine-tuned on the validation set; if discontinuities in the segmentation boundaries are found, the value can be increased appropriately. Weight (increase) If the overall overlap of the regions is found to be poor, adjustments can be made accordingly. The weight (equivalent to reducing) ).

[0121] The formula for calculating the Dice similarity coefficient loss is as follows:

[0122]

[0123] in, This represents the probability that the i-th pixel in the anatomical structure segmentation probability map belongs to the c-th anatomical structure category; This represents the binary existence label of the i-th pixel in the segmentation annotation image, indicating that it belongs to the c-th anatomical structure category. is a preset minimum value used to prevent the denominator from being zero; i is the pixel index, N is the number of pixels; c is the index of the anatomical structure category, and C is the number of anatomical structure categories.

[0124] Dice similarity coefficient loss ( Dice loss is one of the two key loss functions used in medical image semantic segmentation tasks. Dice loss optimizes the overall segmentation effect by measuring the degree of overlap between the predicted segmented region and the ground truth labeled region. It has good robustness to class imbalance problems. The smaller the value, the higher the overlap between the predicted region and the ground truth region and the better the segmentation effect. The larger the value, the more significant the difference between the two and the inaccurate segmentation.

[0125] For example, suppose there are only two classes (background c=0, left lung c=1), and the image has 4 pixels. The predicted probability p and the true label g are as follows:

[0126] Pixel 1: p{1,1}=0.9, g{1,1}=1;

[0127] Pixel 2: p{1,2}=0.8, g{1,2}=1;

[0128] Pixel 3: p{1,3}=0.1, g{1,3}=0 (background);

[0129] Pixel 4: p{1,4}=0.2, g{1,4}=0 (background);

[0130] Assumption Since it is a local minimum, we ignore it and calculate... It equals 0.15.

[0131] parameter The denominator is a very small positive number (such as 1e-7) to prevent numerical calculation errors caused by a zero denominator. This is a common technique for numerical stability.

[0132] The weighted cross-entropy loss is used to optimize the classification of each pixel, mitigating the impact of the uneven number of pixels of different anatomical structures in the image. Its calculation formula is as follows:

[0133]

[0134] in, This is the positive weighting coefficient assigned to the c-th anatomical structure category; this weighting coefficient is inversely proportional to the pixel frequency of this anatomical structure in the training dataset, and can be specifically calculated as the total number of pixels / (C * the number of pixels in the c-th anatomical structure category).

[0135] The weighted cross-entropy loss optimizes the pixel-level classification by assigning weights inversely proportional to pixel frequency to different anatomical structure categories. This forces the model to give equal importance to anatomical structures that appear less frequently and are smaller in size in the image. The smaller the value, the lower the classification error per pixel, while the larger the value, the poorer the pixel-level classification performance. The combination of the two methods can supervise the model from both the macro-region consistency and micro-pixel accuracy levels, effectively improving the overall accuracy and balance of complex anatomical structure segmentation.

[0136] In weighted cross-entropy loss, a weight coefficient is assigned to each anatomical structure category, inversely proportional to the pixel frequency of that category. This is reasonable because of the severe class imbalance prevalent in medical images. For example, in a chest X-ray, the number of pixels in the background (non-organ regions) is often much greater than the number of pixels in a specific lung lesion area. Without weighting, the loss function would be dominated by the category with the most pixels (such as the background), causing the model to tend to ignore the segmentation of small objects (such as small nodules). By using the calculation method of "total number of pixels / (C * number of pixels in the c-th class)", higher weights can be automatically assigned to categories with fewer pixels, thereby balancing the influence of each category during training. This forces the model to learn to identify sparse but clinically significant anatomical structures equally diligently, ultimately achieving more balanced and comprehensive segmentation performance and avoiding model failure due to data bias.

[0137] For example, suppose the training set contains 3 images of 100 pixels each, totaling 300 pixels, and there are 3 classes (C=3): background (c=0), left lung (c=1), and right lung (c=2). The statistics show that: background has 180 pixels, left lung has 70 pixels, and right lung has 50 pixels. , , This formula assigns greater weights to smaller categories (fewer pixels) (e.g., right lung weight 2.0) and smaller weights to larger categories (e.g., background) (0.556), thus forcing the model to pay equal attention to the segmentation accuracy of small objects and preventing the model from tending to predict all pixels as background in order to reduce loss.

[0138] For each target anatomical region, the sign recognition loss is calculated using the sign feature vector output by the sign detection module as the predicted value and the sign annotation vector of that target anatomical region as the target value. The sign recognition loss is calculated separately for the presence and measurement value of the sign, as follows:

[0139]

[0140] in, This indicates a loss in sign recognition; The existence loss of the feature is represented by the binary cross-entropy loss function, which is used to calculate the binary existence label of all features in the feature annotation vector. These are the weighting coefficients for the corresponding items; This represents the loss for symptom metrics, calculated only for symptoms with a presence label of 1. For objectively measurable symptoms, whose metrics are continuous numbers (such as size and density values), a smoothed L1 loss function is used. This loss is less sensitive to outliers than the L2 loss function, resulting in more stable training. The calculation formula is:

[0141]

[0142] in, This represents the loss of the symptom metric for the s-th symptom, where s is the index of the symptom with an existence label of 1. This represents the difference between the predicted value and the actual value.

[0143] For subjectively evaluated signs, the metric is an ordered classification rank score, using a weighted ordinal regression loss function. The core idea is to treat the Q ordered ranks as Q-1 binary classification tasks (i.e., whether a value is greater than or equal to rank q). For example, the prediction network outputs a Q-1 dimensional vector, where each element uses a sigmoid function to represent the probability of being greater than or equal to rank q. The loss is calculated as the weighted binary cross-entropy between these Q-1 binary probabilities and the Q-1 binary labels converted from the true ranks.

[0144] Weighting coefficient This is used to balance the importance of existence judgments and metric regression. The initial value is typically set to 0.5.

[0145] The disease classification loss is calculated using the standard multi-class cross-entropy loss function. The probability distribution of the target disease category output by the evidence reasoning module is used as the predicted value. The probability distribution of the target disease category is represented as a probability distribution vector, where each element represents the probability of belonging to the k-th target disease category. Here, k represents the index of the target disease category. The disease diagnosis label is converted into a one-hot encoded vector with the same dimension as the probability distribution vector, where each element is encoded with 0 and 1, and there is exactly one element that is 1, indicating that the element corresponds to the disease diagnosis label. The one-hot encoded vector is used as the target value to calculate the disease classification loss.

[0146] The calculation logic of the confidence prediction auxiliary loss is as follows: using the confidence score vector of the feature vector of each target anatomical region output by the confidence prediction sub-network as the prediction value, and the quality score of the sign itself as the target value, the confidence prediction auxiliary loss is calculated; the confidence prediction auxiliary loss adopts the mean squared error loss function, and its calculation is only for signs with a binarized existence label of 1.

[0147] The total loss function during training is obtained by weighted summation of the losses mentioned above, and the calculation formula is as follows:

[0148]

[0149] in, This represents the total loss function value; For disease classification loss value; To predict the auxiliary loss value for credibility; These are the weighting coefficients of their respective terms in the total loss function.

[0150] Different loss functions have significantly different numerical scales and convergence speeds, requiring a balance. Specifically, the first approach uses a manual network search method, trying different weight combinations on the validation set (e.g., within the range of [0.1, 0.5, 1, 2, 5]) to select the combination that optimizes overall performance on the validation set (e.g., maximizing disease classification accuracy while maintaining the Dice coefficient for segmentation and the F1 score for sign recognition). An initial weight of 1.0 is suggested as a starting point. The second approach uses automatic uncertain weighting; a more advanced strategy is to allow the model to automatically learn the weights for each task during training. Each loss weight... Treating it as a learnable log-variance parameter, the total loss becomes ,in These are learnable parameters. This method allows the model to automatically assign weights based on the uncertainty of the task, but it is more complex to implement. `Task` is an index of any loss term in the model. This is the unweighted, raw loss value calculated for that subtask in the current training batch. For example, when hour, It is calculated according to the formula. value.

[0151] Appropriate weight allocation ensures that the model does not favor any particular area during optimization. For example, if If the weights are too large, the model may prematurely focus on fitting disease labels and neglect the accuracy of anatomical segmentation, leading to a decline in the input quality of the sign detection module and ultimately affecting generalization ability. The goal of joint optimization is to find a set of weights that allow all subtasks to work synergistically to achieve optimal performance on the validation set.

[0152] During training, an adaptive optimizer, such as Adam or AdamW, is used. AdamW typically offers better generalization performance. Employing cosine annealing or cosine annealing with warm restart as the learning rate scheduling strategy helps the model escape local optima. Batch size is set based on GPU memory, typically 8, 16, 32, etc. Continuously monitor the total loss and core metrics (such as disease classification accuracy) on the validation set. When the validation set loss no longer decreases over several consecutive epochs (e.g., 20 epochs, i.e., the patience value), or when the core metrics no longer improve, trigger early stopping, halt training, and roll back to the model parameters that best perform on the validation set. In addition to loss, evaluation metrics for each task should also be monitored, such as the Dice coefficient for segmentation, the average accuracy of sign recognition, and the AUC for disease classification, to ensure balanced performance across multiple tasks.

[0153] Through this multi-task joint training step, the model simultaneously receives feedback and adjusts its parameters at the pixel, region, and image levels, as well as the metacognitive level of confidence. This design forces the model to learn an intrinsically consistent representation: good anatomical segmentation helps in the discovery of accurate signs, and accurate signs and their confidence assessments together lead to robust disease diagnosis. Ultimately, the model not only learns to diagnose, but also learns how to diagnose based on reliable evidence, like an expert.

[0154] The model application and diagnosis module is used to save the parameters of the initial model after training to obtain a medical image recognition model; input the image to be recognized into the medical image recognition model, diagnose the disease category of the image to be recognized according to the probability distribution of the target disease classification, and output the relevant diagnostic basis simultaneously.

[0155] In this embodiment, when the initial model meets the preset convergence condition, the training of the initial model is considered to be completed. All parameters (such as convolutional kernel weights, fully connected layer weights and biases) of the anatomical perception module, sign detection module, evidence reasoning module and credibility prediction sub-network are solidified and saved as the final medical image recognition model.

[0156] The medical image recognition model defines a standardized structured output format, such as a JSON object or a protocol-specific dictionary. For any new input image to be recognized, after analysis by the medical image recognition model, the output includes relevant diagnostic criteria comprising the following four core data components:

[0157] Based on the probability distribution of the target disease classification output by the feature fusion network, the disease category corresponding to the highest probability value is taken as the disease category diagnosed for the current image to be identified. In addition to the most likely category corresponding to the highest probability value, the top three disease categories and their probabilities can also be output for reference.

[0158] Output a list of target anatomical regions, including at least: the name of the anatomical structure category identified by the anatomical perception module (e.g., "left upper lobe" or "right lower lobe"). The boundary of the target anatomical region in the image to be identified, i.e., a description of its spatial location in the image. This is obtained through the post-processed segmentation results output by the anatomical perception module, such as binarizing the corresponding category channels in the segmentation probability map using the same probability threshold as during training to obtain a binary mask, and then using a contour search algorithm to extract the outer polygon coordinate sequence of the binary mask / using the minimum bounding rectangle to describe the boundary. All signs with a binarized existence label of 1 within the target anatomical region and their corresponding metric values. The boundary of the target anatomical region in the image to be identified is obtained through the segmentation annotation map. For this region, list all signs determined to exist by the sign detection module. A sign existence determination threshold is required here; if the predicted probability value of a sign is not less than the sign existence determination threshold, the sign is considered to exist. The threshold for determining the presence of a symptom is typically set to 0.5. A preferred approach is to select a threshold that yields a higher F1 score based on the precision and recall curves for different symptom types on the validation set. The output confidence prediction subnetwork provides a confidence score vector for each symptom feature vector predicted. For example, for the "nodule" symptom, the output predicts a diameter of "8.5 mm"; for "solidity," the output predicts a grade of "2-moderate." Different symptom types have varying recognition difficulties and image clarity. For relatively clear symptom types like "nodules," the optimal threshold may be close to 0.5; while for symptom types with blurred boundaries like "ground-glass opacity," the optimal threshold may need to be lowered (e.g., 0.3) to capture more suspected cases, or increased (e.g., 0.7) to reduce false positives. Optimizing the threshold individually for each symptom is a crucial step in achieving optimal clinical applicability.

[0159] Please see Figure 2 The present invention also provides a method for generating an image recognition model, wherein the method for generating an image recognition model is executed by the aforementioned image recognition model generating device, and the specific steps include:

[0160] Step 1: Construct an initial model, which includes three modules connected in sequence: an anatomical perception module, a sign detection module, and an evidence reasoning module. The anatomical perception module is used to output the anatomical structure segmentation probability map of the sample image. The sign detection module generates sign feature vectors for predefined target anatomical regions based on the probability map. The evidence reasoning module is used to integrate all sign feature vectors to output the probability distribution of the target disease classification.

[0161] Step 2: Integrate a credibility prediction subnetwork into the evidence reasoning module to calculate the credibility score vector of the sign feature vector for each target anatomical region, so as to characterize the reliability of the sign feature vector in its sample images.

[0162] Step 3: Associate each sample image with three types of annotations: pixel-level annotations, region-level annotations, and image-level annotations; using the sample image as input, the three types of annotations as supervision annotations, and the outputs of the three modules as labels, to form the training dataset;

[0163] Step 4: Train the initial model using the training dataset. The total loss function during training includes anatomical segmentation loss, sign recognition loss, disease classification loss, and confidence prediction auxiliary loss.

[0164] Step 5: After training is completed, save the parameters of the initial model to obtain the medical image recognition model; input the image to be recognized into the medical image recognition model, diagnose the disease category of the image to be recognized according to the probability distribution of the target disease classification, and output the relevant diagnostic basis simultaneously.

[0165] The above formulas are all dimensionless calculations. The formulas are derived from software simulations based on a large amount of collected data to obtain the most recent real-world results. The preset parameters in the formulas are set by those skilled in the art according to the actual situation.

[0166] The above embodiments can be implemented, in whole or in part, by software, hardware, firmware, or any other combination thereof. When implemented in software, the above embodiments can be implemented, in whole or in part, as a computer program product. Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution.

[0167] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment, depending on actual needs.

[0168] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application.

Claims

1. An apparatus for generating an image recognition model, characterized in that, include: The initial model building module is used to build an initial model, which includes three modules connected in sequence: an anatomical perception module, a sign detection module, and an evidence reasoning module. The anatomical perception module is used to output an anatomical structure segmentation probability map of the sample image. The sign detection module generates sign feature vectors for a predefined target anatomical region based on the probability map. The evidence reasoning module is used to integrate all sign feature vectors to output the probability distribution of the target disease classification. The credibility integration module is used to integrate the credibility prediction subnetwork into the evidence reasoning module to calculate the credibility score vector of the sign feature vector of each target anatomical region to characterize the reliability of the sign feature vector in its sample images. The multi-class annotation association module is used to associate each sample image with three types of annotations: pixel-level annotations, region-level annotations, and image-level annotations. The sample image is used as input, the three types of annotations are used as supervision annotations, and the outputs of the three modules are used as labels to form the training dataset. The multi-task joint training module is used to train the initial model using the training dataset. The total loss function during the training process includes anatomical segmentation loss, sign recognition loss, disease classification loss, and confidence prediction auxiliary loss. The model application and diagnosis module is used to save the parameters of the initial model after training to obtain a medical image recognition model; input the image to be recognized into the medical image recognition model, diagnose the disease category of the image to be recognized according to the probability distribution of the target disease classification, and output the relevant diagnostic basis simultaneously. The anatomical perception module is set as a medical image semantic segmentation network. Its function is to classify each pixel in the input sample image into a predefined anatomical structure category, thereby outputting an anatomical structure segmentation probability map with the same size as the input image. The sign detection module is used to process one or more specific target anatomical regions; for each target anatomical region, the sign detection module performs the following operations: From the anatomical structure segmentation probability map, extract the single-channel probability map corresponding to the target anatomical region as the probability channel of the target anatomical region; where each probability value in the probability channel represents the predicted probability that the corresponding pixel in the input image belongs to the target anatomical region; set the probability threshold of the target anatomical region, modify the probability values ​​above the probability threshold to 1, and modify them to 0, so as to convert the probability channel into a binary mask map of the target anatomical region; The evidence reasoning module includes a feature fusion network, whose input is the sign feature vectors of all target anatomical regions output by the sign discovery module; the feature fusion network is used to integrate the sign feature vectors used to output a global probability distribution of the target disease classification.

2. The image recognition model generation device according to claim 1, characterized in that, Each pixel in the anatomical structure segmentation probability map corresponds to a vector, representing the probability that the pixel belongs to each anatomical structure category; The medical image semantic segmentation network adopts an encoder and decoder structure; the encoder consists of multiple downsampling layers and convolutional layers, which are used to progressively extract multi-scale feature maps of the input sample image and reduce its spatial size. The decoder consists of multiple upsampling layers and convolutional layers, and is fused with the multi-scale feature maps of the corresponding layers of the encoder through skip connections to gradually recover the spatial size of the multi-scale feature maps and finally output the anatomical structure segmentation probability map. A binarized mask image is used to spatially attention-weight the multi-scale feature map to obtain a weighted multi-scale feature map. Cross-scale fusion and spatial pooling are then performed on the weighted multi-scale feature map to generate a depth feature vector of the target anatomical region. The depth feature vector is then input into a fully connected network for analysis to obtain a sign feature vector. Each dimension of the feature vector corresponds to a predefined sign related to the target anatomical region, and its value represents the presence intensity of the corresponding sign. When calculating the probability distribution of the target disease classification, the feature fusion network receives the confidence score vectors of each symptom feature vector from the confidence prediction subnetwork, and converts the confidence scores into fusion weights to perform weighted fusion on the corresponding symptom feature vectors to obtain the final probability distribution.

3. The image recognition model generation device according to claim 2, characterized in that, The confidence prediction subnetwork is independent of the feature fusion network. Each subnetwork takes the feature vector of a single target anatomical region as input, processes it through a fully connected network with the Sigmoid function as the activation layer, and outputs a confidence score vector with the same dimension as the input feature vector. The score of each dimension of the confidence score vector corresponds to the confidence of a sign in the target anatomical region.

4. The image recognition model generation device according to claim 3, characterized in that, The specific annotation process for the three types of annotations is as follows: The annotators annotate each sample image in the training dataset; The pixel-level annotation refers to assigning an anatomical structure category label to each pixel in the sample image; through the anatomical structure category labels of each pixel, several anatomical regions are divided, generating a segmentation annotation map with the same spatial size as the sample image; Based on the anatomical regions defined by the segmentation and annotation map, the target anatomical region to be analyzed is identified. The region-level annotation process for the target anatomical region to be analyzed is as follows: For a predefined sign related to the target anatomical region, determine whether it is within the target anatomical region and generate a binary existence label, where 1 indicates existence and 0 indicates non-existence. For a symptom that is determined to exist, a numerical or classification level score is generated based on the symptom type. The symptom type is divided into objective measurement type and subjective evaluation type. For objective measurement type symptom, at least numerical parameters are calculated based on image grayscale, morphological or texture features. For subjective evaluation type symptom, the annotator makes a judgment and gives at least an ordered classification level score. For a target anatomical region, a sign annotation vector is finally generated, which consists of all the signs present in the target anatomical region and their measurement values. Image-level annotation is based on the known disease categories in the sample image and associates them with the sign annotation vector of the target anatomical region, using the known disease categories in the sample image as the disease diagnosis labels of the sign annotation vector.

5. The image recognition model generation apparatus according to claim 4, characterized in that, During training, the credibility prediction subnetwork is constrained by an auxiliary supervision signal; this auxiliary supervision signal comes from the self-quality score of the feature whose binarized existence label is 1 in the regional annotation; the learning objective of the credibility prediction subnetwork is to make the difference between its output credibility score vector and the self-quality score of the corresponding feature within the deviation range. For features with a binarized existence label of 1, a self-quality score reflecting the reliability of their annotation is calculated, as follows: The annotation process is completed independently by multiple annotators, and their own quality score is equal to the proportion of all annotators who determine that the phenomenon exists out of the total number of annotators. The sample images that have completed the above three types of annotations are randomly divided into non-overlapping training sets, validation sets, and test sets according to a preset ratio; All sample images undergo standardized preprocessing operations, including at least: normalizing pixel values ​​to a fixed range, and resampling or cropping the image size to a uniform size.

6. The image recognition model generation apparatus according to claim 3, characterized in that, During training, each sample image in the training set is input into the initial model to obtain the anatomical structure segmentation probability map, the sign feature vector of each target anatomical region, the probability distribution of the target disease classification, and the confidence score vector of each sign. The segmentation annotation map, sign annotation vector, disease diagnosis label, and the sign's own quality score are used as target values ​​and compared with the corresponding output of the model to calculate the loss values. Using the anatomical structure segmentation probability map output by the anatomical perception module as the predicted value and the segmentation annotation map generated by pixel-level annotation as the target value, the loss is calculated; the anatomical segmentation loss consists of two parts, expressed by the formula: in, Indicates loss due to anatomical segmentation; This represents the Dice similarity coefficient loss, used to optimize the overall segmentation overlap of the anatomical region; These are the weighting coefficients for the corresponding items; The weighted cross-entropy loss; The formula for calculating the Dice similarity coefficient loss is as follows: in, This represents the probability that the i-th pixel in the anatomical structure segmentation probability map belongs to the c-th anatomical structure category; This represents the binary existence label of the i-th pixel in the segmentation annotation image, indicating that it belongs to the c-th anatomical structure category. This is a preset minimum value used to prevent the denominator from being zero; i is the pixel index, N is the number of pixels; c is the index of the anatomical structure category, and C is the number of anatomical structure categories; The weighted cross-entropy loss is used to optimize the classification of each pixel, mitigating the impact of the uneven number of pixels of different anatomical structures in the image. Its calculation formula is as follows: in, The positive weighting coefficients assigned to the c-th anatomical structure category; For each target anatomical region, the sign recognition loss is calculated using the sign feature vector output by the sign detection module as the predicted value and the sign annotation vector of that target anatomical region as the target value. The sign recognition loss is calculated separately for the presence and measurement value of the sign, as follows: in, This indicates a loss in sign recognition; The presence loss is represented by the binary cross-entropy loss function, which is used to calculate the binary presence labels of all features in the feature annotation vector. These are the weighting coefficients for the corresponding items; The loss represents the symptom measurement value, which is calculated only for symptoms with a presence label of 1. For objectively measured symptoms, a smoothed L1 loss function is used for calculation. For subjectively evaluated symptoms, the metric is an ordered classification rank score, and a weighted ordinal regression loss function is used. The disease classification loss is calculated using the standard multi-class cross-entropy loss function. The probability distribution of the target disease category output by the evidence reasoning module is used as the predicted value. The probability distribution of the target disease category is represented as a probability distribution vector, where each element represents the probability of belonging to the k-th target disease category. Here, k represents the index of the target disease category. The disease diagnosis label is converted into a one-hot encoded vector with the same dimension as the probability distribution vector, where each element is encoded with 0 and 1, and there is exactly one element that is 1, indicating that the element corresponds to the disease diagnosis label. The one-hot encoded vector is used as the target value to calculate the disease classification loss. The calculation logic of the confidence prediction auxiliary loss is as follows: using the confidence score vector of the feature vector of each target anatomical region output by the confidence prediction sub-network as the prediction value, and the quality score of the sign itself as the target value, the confidence prediction auxiliary loss is calculated; the confidence prediction auxiliary loss adopts the mean squared error loss function, and its calculation is only for signs with a binarized existence label of 1. The total loss function during training is obtained by weighted summation of the losses mentioned above, and the calculation formula is as follows: in, This represents the total loss function value; For disease classification loss value; To predict the auxiliary loss value for credibility; These are the weighting coefficients of their respective terms in the total loss function.

7. The image recognition model generation apparatus according to claim 1, characterized in that, Once the initial model meets the preset convergence conditions, the training of the initial model is considered complete. All parameters of the anatomical perception module, sign detection module, evidence reasoning module, and credibility prediction sub-network are then fixed and saved as the final medical image recognition model. The medical image recognition model defines a standardized structured output format; for any new input image to be recognized, after analysis by the medical image recognition model, the output includes relevant diagnostic criteria comprising the following four core data components: Based on the probability distribution of the target disease classification output by the feature fusion network, the disease category corresponding to the highest probability value is taken as the disease category diagnosed in the current image to be identified. Output a list of target anatomical regions, which includes at least: the name of the anatomical structure category identified by the anatomical perception module, the boundary of the target anatomical region in the image to be identified, all features in the target anatomical region with a binarized existence label of 1, and the corresponding measurement value of each feature; the boundary of the target anatomical region in the image to be identified is obtained through a segmentation annotation map. Output the confidence score vector for each feature vector predicted by the confidence prediction subnetwork.

8. A method for generating an image recognition model, characterized in that, The method for generating an image recognition model is executed by the image recognition model generating apparatus according to any one of claims 1-7, and the specific steps include: Step 1: Construct an initial model, which includes three modules connected in sequence: an anatomical perception module, a sign detection module, and an evidence reasoning module. The anatomical perception module is used to output the anatomical structure segmentation probability map of the sample image. The sign detection module generates sign feature vectors for predefined target anatomical regions based on the probability map. The evidence reasoning module is used to integrate all sign feature vectors to output the probability distribution of the target disease classification. Step 2: Integrate a credibility prediction subnetwork into the evidence reasoning module to calculate the credibility score vector of the sign feature vector for each target anatomical region, so as to characterize the reliability of the sign feature vector in its sample images. Step 3: Associate each sample image with three types of annotations: pixel-level annotations, region-level annotations, and image-level annotations; using the sample image as input, the three types of annotations as supervision annotations, and the outputs of the three modules as labels, to form the training dataset; Step 4: Train the initial model using the training dataset. The total loss function during training includes anatomical segmentation loss, sign recognition loss, disease classification loss, and confidence prediction auxiliary loss. Step 5: After training is completed, save the parameters of the initial model to obtain the medical image recognition model; input the image to be recognized into the medical image recognition model, diagnose the disease category of the image to be recognized according to the probability distribution of the target disease classification, and output the relevant diagnostic basis simultaneously.

Images