Knowledge distillation with adaptive asymmetric label sharpening for semi-supervised fracture detection in chest x-rays

Abstract

A knowledge distillation method for fracture detection includes obtaining medical images in chest X-rays, including region-level labeled images, image-level diagnosis positive images, and image-level diagnosis negative images; performing a supervised pre-training process on the region-level labeled images and the image-level diagnosis negative images to train a neural network to generate pre-training weights; and performing a semi-supervised training process on the image-level diagnosis positive images using the pre-training weights. A teacher model is employed to generate pseudo ground truth (GT) on the image-level diagnosis positive images to supervise the training of a student model, the pseudo GT is processed by an adaptive asymmetric label sharpening (AALS) operator to generate sharpened pseudo GT to provide positive detection responses on the image-level diagnosis positive images.

Description

Technical Field

[0001] This application claims priority to U.S. Provisional Patent Application No. 63 / 130,838, filed December 27, 2020. It also claims priority to U.S. Patent Application No. 17 / 214,400, filed March 26, 2021. The entire contents of both are incorporated herein by reference.

[0002] This application relates to the field of knowledge distillation of medical images, and more specifically, to methods, electronic devices, and computer program products for knowledge distillation of fracture detection in chest X-rays. Background Technology

[0003] Over the past decade, computer-aided diagnostics (CAD) of medical images has been extensively researched. In recent years, substantial progress has been made in developing deep learning-based CAD systems for diagnosing various pathologies. State-of-the-art CAD solutions are typically developed based on large-scale expert annotation. However, the high cost of large-scale annotation in the medical field due to the required medical expertise has hindered the development of deep learning-based CAD solutions for applications that do not yet possess such large-scale annotation capabilities. Summary of the Invention

[0004] One aspect of this disclosure provides a knowledge distillation method for fracture detection. The knowledge distillation method includes: acquiring a medical image from a chest X-ray, the medical image including a region-level labeled image, an image-level diagnostic positive, and an image-level diagnostic negative; performing a supervised pre-training process on the region-level labeled image and the image-level diagnostic negative to train a neural network to generate pre-trained weights; and performing a semi-supervised training process on the image-level diagnostic positive using the pre-trained weights. A teacher model is used to generate a pseudo-ground reality (GT) on the image-level diagnostic positive to supervise the training of a student model. The pseudo-GT is processed by an adaptive asymmetric label sharpening (AALS) operator to generate a sharpened pseudo-GT to provide a positive detection response on the image-level diagnostic positive.

[0005] Another aspect of this disclosure provides an electronic device. The electronic device includes: a memory containing a computer program stored thereon; and a processor coupled to the memory, which, when executed by the processor, implements the method described above.

[0006] Another aspect of this disclosure provides a non-transitory computer-readable storage medium storing a computer program that, when executed by the processor, implements the method described above.

[0007] Other aspects of this disclosure will be understood by those skilled in the art based on the specification, claims and drawings. Attached Figure Description

[0008] Figure 1 An example knowledge distillation method for fracture detection disclosed in this invention is shown.

[0009] Figure 2 This invention illustrates another example of a knowledge distillation method for fracture detection disclosed in this invention.

[0010] Figure 3 An example electronic device for knowledge distillation for fracture detection, based on the present invention, is shown.

[0011] Figure 4 An example of the asymmetric marker sharpening function disclosed in this invention is shown.

[0012] Figure 5 An example of the model performance disclosed in this invention using a subset of image-level diagnostic positive films is shown.

[0013] Figure 6 An example of the fracture detection results disclosed in this invention is shown.

[0014] Figure 7 Examples of FROC (Free Response Receiver Operational Characteristics) curves showing the detection results of rib fractures (left) and clavicle fractures (right) using different methods disclosed in this invention are shown. Detailed Implementation

[0015] The technical solutions of the embodiments of the present invention will be described below with reference to the accompanying drawings. Where possible, the same reference numerals will be used throughout the drawings to refer to the same or similar parts. Obviously, the described embodiments are only some embodiments of the present invention and do not represent all embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention. Some terms used in this disclosure will be explained below.

[0016] Various embodiments provide methods, electronic devices, and computer program products for knowledge distillation methods for fracture detection. For example, knowledge distillation methods can be performed using adaptive asymmetric label sharpening (AALS) to perform semi-supervised fracture detection on chest X-rays (CXR). Therefore, a reliable, robust, and accurate fracture detection model for CXR can be provided using limited expert annotations and a rich clinical diagnostic record.

[0017] In one embodiment, image-level CXR (e.g., labeled / unlabeled) and region-level labeled CXR can be used to develop fracture detection models to identify the classification and localization of fractures (including, for example, rib and clavicle fractures and spinal fractures) based on CXR.

[0018] As used herein, the term “chest X-ray” or CXR may refer to X-ray imaging results and / or X-ray examinations related to the chest, including the ribs, clavicle, and / or spine.

[0019] Image-level tags can be obtained efficiently and on a large scale, for example, by mining hospital image archives and clinical records. In one example, image-level tags can be obtained by finding matching diagnostic codes and / or keywords in clinical records. Image-level tags can include positive tags (e.g., for a positive diagnosis) and negative tags (e.g., for a negative diagnosis).

[0020] Region-level labeling can be done manually by experts and is more costly. For example, image-level diagnostic positive CXR can be annotated by experts to provide region-level labeling, such as in the form of bounding boxes.

[0021] Figure 1 Example knowledge distillation methods for fracture detection are shown according to various embodiments of this disclosure. Figure 2 Another example of a knowledge distillation method for fracture detection according to various embodiments of the present disclosure is shown.

[0022] exist Figure 1 At step 110, medical images of CXR were obtained, including region-level labeled images (R), image-level diagnostic positive images (P), and image-level diagnostic negative images (N).

[0023] In an exemplary implementation, a large number of CXRs (e.g., approximately 65,845 CXRs) can be obtained based on a medical center's trauma registry. CXRs with image-level positive and negative diagnostic markers can be identified by matching diagnostic codes and keywords in the clinical records (e.g., including 6,792 positive CXRs and 59,051 negative CXRs). Among the CXRs with image-level positive diagnostic markers, many may be annotated by experts to provide region-level markings in the form of bounding boxes.

[0024] The methods, apparatus, and / or products disclosed herein can effectively utilize CXR with regional and image-level markings under extremely unbalanced positive-to-negative ratios.

[0025] In some embodiments, a neural network can be trained to generate a probability map indicating the location of a detected fracture. Since the shape and scale of fractures can vary considerably, a Feature Pyramid Network (FPN) with a ResNet-50 backbone can be employed to address the challenge of scale variation by fusing multi-scale features. In some embodiments, the training includes at least a supervised pre-training process and a semi-supervised training process. During pre-training, a fracture detection model is trained via supervised learning using region-level labeled images (R) and image-level diagnostic negatives (N). During semi-supervised training, image-level diagnostic positives (P) are further utilized to facilitate the training of this disclosure.

[0026] exist Figure 1 At step 120, a supervised pre-training process is performed on the region-level labeled image (R) and the image-level diagnostic negative (N) to train the neural network (e.g., using a student model) to generate pre-trained weights.

[0027] A neural network can be trained using only region-level labeled images and image-level diagnostic negatives, where pixel-level supervision signals (or pixel-level supervision) can be generated. In one embodiment, the pixel-level supervision signal can be derived from doctor-labeled bounding boxes, for example, by assigning pixels within the bounding boxes as "one" and "zero" elsewhere. For example, for a region-level labeled image (R) in CXR, this is done by assigning "one" ("1") to pixels within the bounding boxes (e.g., as region-level labels) and assigning "zero" ("0") elsewhere. For an image-level diagnostic negative (N) in CXR, a ground truth mask consisting entirely of "zero" is generated.

[0028] During training, the same supervised loss used in the supervised pre-training process for region-level labeled images (R) and image-level diagnostic negatives (N) can be used. For example, the pixel-level binary cross-entropy (BCE) loss between the predicted probability map and the generated ground truth mask can be used for training on region-level labeled images (R) and image-level diagnostic negatives (N).

[0029] BCE loss (L) sup As shown below: Where x represents the region-level annotated image (R) or image-level diagnostic negative (N) in a chest X-ray, y represents the pixel-level supervision mask corresponding to X, and f θ (x) represents the probabilistic graph output of the neural network parameterized by θ.

[0030] In such Figure 2In another example shown, a supervised pre-training process can be performed on an “expert label set” that includes region-level labeled images (R) and image-level diagnostic negatives (N) to use the BCE loss (L) between the predicted probability map and the GT mask generated based on the expert annotations. sup (Student models are used to train neural networks.)

[0031] In cases of extreme imbalance between region-level labeled images (R) and image-level diagnostic negatives (N) (e.g., in one example, 808 for R versus 59,861 for N), a pre-trained model may tend to have low detection sensitivity, for example, producing low probabilities at fracture sites. A semi-supervised training process can then be performed.

[0032] exist Figure 1 At step 130, a semi-supervised training process is performed on the image-level diagnostic positive film (P). For example, a teacher-student paradigm that includes a student model and a teacher model can be adopted. Figure 2 It also includes examples of teacher-student paradigms according to various embodiments of this disclosure.

[0033] like Figure 2 As shown, the teacher model is used to generate pseudo ground truth (GT) on the image-level diagnostic positive (P) to supervise the training of the student model. The student model learns from the pseudo GT generated by the teacher model on the image-level diagnostic positive (P). The teacher and student models share the same network architecture (including, for example, ResNet-50 with Feature Pyramid Network (FPN)) and both use data from... Figure 1 The pre-trained weights obtained in the exemplary supervised learning step at step 120 are initialized.

[0034] In one embodiment, the student model is trained via backpropagation and also as... Figure 2 As shown, the teacher model is iteratively updated during training using the exponential moving average (EMA) of the student model weights.

[0035] The weights of the teacher model are updated as follows: θ′ t =αθ t-1 +(1-α)θ t (2) Where θ′ t and θ t α and t represent the weights of the teacher and student models in training step t, respectively, and α is a smoothing coefficient that controls the speed of knowledge updates. For example, α can be set to 0.999 as needed.

[0036] In various embodiments, the CXR of the region-level label set (R), the image-level positive label set (P), and the image-level negative label set (N) can all be used to train the teacher-student model.

[0037] return Figure 1 In step 130, during the semi-supervised training process, the adaptive asymmetric label sharpening (AALS) process can be further performed in the adaptive asymmetric label sharpening (AALS) operator by using the pseudo ground truth (GT) or pseudo GT map generated by the teacher model on the image-level diagnostic positive (P). This can thus generate sharpened / enhanced pseudo GT.

[0038] The sharpened pseudo-GT(y′) of image (x) is represented as: in Let θ' represent the teacher model at step t. t Let S(·) represent the weights of the teacher model during training step t, and let S(·) denote adaptive asymmetric label sharpening (AALS). The sharpened pseudo-GT(y′) and the student model's predictions are... The KL divergence between them is calculated as the additional loss L. semi (or KLD loss):

[0039] Therefore, the total loss used to train the student network includes:

[0040] In other knowledge distillation models, pseudo-GTs are generated on unlabeled data to supervise student models. Since no knowledge is given about the unlabeled data, the pseudo-GTs are either used directly or subjected to symmetric softening or sharpening in other baseline models. In contrast, the knowledge distillation method of this disclosure can be implemented in the following situations: 1) CXR image-level diagnostic positive films contain visible fracture sites, and 2) due to the imbalance of positive and negative ratios, pseudo-GTs tend to have low sensitivity (i.e., low probability of fracture sites).

[0041] Therefore, the maximum value of the pseudo-GT map may be low. To enhance the activation of the student model at the fracture site, AALS was employed. y′=S(y)=expit(A·logit(y)+(1-A)·logit(T)) (6) Where expit(·) and logit(·) represent the Sigmoid function and its inverse function, and A and T control the intensity and center of the sharpening operator, respectively.

[0042] In one embodiment, the effects of A and T used to control the sharpening of intensity and center are as follows: Figure 2 As shown in the figure, after "sharpening", the intensity and center of the "enhanced pseudo-GT" are enhanced compared to the "pseudo-mask".

[0043] The asymmetric sharpening (e.g., AALS) disclosed herein is used to enhance low probabilities in pseudo-GT, where T < 0.5 and is therefore applicable to the AALS process. In one example, T = 0.4 can be optionally used.

[0044] In some cases, due to the imbalance of training data and the possibility that some fracture sites (e.g., with low probability values) may still be missed in the pseudo-GT(y), max(S(y), y) is used as a label-sharpening function to avoid over-penalizing student model activations for fracture sites with low probability values ​​in the pseudo-GT(y).

[0045] The sharpening intensity A is dynamically selected based on the maximum probability in the pseudo-GT image, written as: A = A0 - (A0 - 1)y max (7) Where y max A is the maximum probability in the pseudo-GT graph, and A0 is a hyperparameter controlling the maximum allowed sharpening intensity. The sharpening intensity A is related to the maximum probability y. max Negative correlation. When y max As y approaches 1, A approaches its minimum value of 1, making S(·) an identity mapping. max When A decreases, it increases towards A0, resulting in stronger sharpening of the pseudo-GT. Dynamic A is needed because the sharpening operator is asymmetric. If a constant A > 1 is used, the sharpening operation will always expand the activation region in the pseudo-GT map, leading to the model generating a probability map with excessively large activation regions. Adaptive sharpening intensity is used when the fracture site is reliably detected in CXR (i.e., y...). max Approaching 1), the sharpening operation degenerates into an identity mapping to avoid continuously expanding the activation region.

[0046] Therefore, this disclosure provides an effective supervised learning solution, including supervised pre-training and semi-supervised training processes, to effectively train fracture detection models (e.g., CAD models) by fully utilizing clinical diagnoses with imbalanced data distributions. The CXR fracture detection method of this disclosure better addresses imbalanced data distributions and leverages image-level labeling of unlabeled data.

[0047] Furthermore, this paper discloses a teacher-student mechanism in which a teacher model is used to generate pseudo-ground facts (GT) on image-level diagnostic positive images to supervise the training of a student model. Unlike other knowledge distillation methods that directly use pseudo-GT or through symmetric sharpening / softening, the adaptive asymmetric label sharpening (AALS) disclosed herein addresses the low sensitivity of the teacher model caused by imbalanced data distribution and provides a positive detection response to image-level diagnostic positive CXR images.

[0048] In various embodiments, the knowledge distillation method for fracture detection disclosed herein can be applied to one or more electronic devices.

[0049] In various embodiments, the electronic device is capable of automatically performing numerical calculations and / or information processing according to pre-configured or stored instructions. The hardware of the electronic device may include, but is not limited to, microprocessors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), digital signal processors (DSPs), and embedded devices. The electronic device can be any electronic product capable of interacting with a user, such as personal computers, tablets, smartphones, desktop computers, laptops, handheld computers, personal digital assistants (PDAs), game consoles, interactive network television (IPTV), and smart wearable devices. The electronic device can interact with the user via a keyboard, mouse, remote control, touchscreen, or voice control device. The electronic device may also include network devices and / or user devices. Network devices may include, but are not limited to, cloud servers, single network servers, server groups consisting of multiple network servers, or cloud computing systems consisting of multiple hosts or network servers. The electronic device can be in a network. The network may include, but is not limited to, the Internet, wide area networks (WANs), metropolitan area networks (MANs), local area networks (LANs), and virtual private networks (VPNs).

[0050] Figure 3 A structural diagram of an exemplary electronic device for performing the knowledge distillation method for fracture detection according to various embodiments of the present disclosure is shown.

[0051] Reference Figure 3 An exemplary electronic device includes a memory 310 storing a computer program and a processor 320 coupled to the memory 310 and configured to execute the knowledge distillation method for fracture detection disclosed herein when the computer program is executed.

[0052] Memory 310 may include volatile memory such as random access memory (RAM) and non-volatile memory such as flash memory, hard disk drive (HDD), or solid-state drive (SSD). Memory 310 may also include various combinations of the above-described memories. Processor 320 may include a central processing unit (CPU), an embedded processor, a microcontroller, and programmable devices such as application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), and programmable logic arrays (PLDs).

[0053] This disclosure also provides a computer-readable storage medium for storing a computer program. The computer program can be loaded into a processor of a computer or a programmable data processing apparatus, such that the computer program is executed by the processor of the computer or the programmable data processing apparatus to implement the methods of this disclosure.

[0054] Various embodiments also provide a computer program product. This computer program product includes a non-transitory computer-readable storage medium and program instructions stored therein. The program instructions can be configured to be executed by a computer to enable the computer to perform operations including the knowledge distillation method for fracture detection disclosed herein.

[0055] Example 1

[0056] The knowledge distillation method for fracture detection disclosed herein is performed by an electronic device including a workstation. For example, the fracture detection model is executed on a workstation with a single Intel Xeon E5-2650v4 CPU @ 2.2GHz, 128GB RAM, and four NVIDIA Quadro RTX 8000 GPUs. All example methods are implemented in Python 3.6 and PyTorch v1.6. ImageNet pre-trained weights are used to initialize the backbone network of the student model. All example methods employ the Adam optimizer. The model of this disclosure is trained for 25 epochs using a learning rate of 4e-5, a weight decay of 0.0001, and a batch size of 48. All images are padded to squares and resized to 1024x1024 for network training and inference. Random rotation, horizontal flipping, intensity, and contrast jitter are performed to augment the training data. After each training epoch, the trained model is evaluated on a validation set, and the model with the highest area under the receiver operating characteristic curve (AUROC) is selected as the best inference model.

[0057] Example 2

[0058] The methods, electronic devices, and computer program products of this disclosure were evaluated on a real-world dataset of all (N=65,843) CXR images taken at the Trauma Center of Chang Gung Memorial Hospital between 2008 and 2016.

[0059] Based on clinical diagnostic records, image-level markers for rib and clavicle fractures were assigned to CXR images. Of a total of 65,843 CXR images, 6,792 had diagnostically positive markers, indicating at least one fracture type, while 59,051 had diagnostically negative markers indicating two fracture types (including rib and clavicle fractures). Of the 6,792 image-level diagnostic positive CXR images, 808 were randomly selected by two experienced trauma surgeons for expert annotation. Annotations were confirmed by the best available information, including the original CXR images, radiologist reports, clinical diagnoses, advanced imaging modality findings, etc. All experiments were performed using a 70% / 10% / 20% five-fold cross-validation, used for training, validation, and test splits, respectively.

[0060] Example 3

[0061] The fracture classification and localization performance was evaluated using a publicly available score detection model. The widely used classification metric AUROC (area under the receiver operating characteristic curve) was used to evaluate classification performance. For object detection, the maximum classification score across all predicted bounding boxes was used as the classification score. To generate the probability map, the maximum value of the probability map was used as the classification score.

[0062] The fracture localization performance of different methods was also evaluated, including the example methods of this disclosure and other baseline methods. As disclosed, since only probabilistic maps can be generated, the standard FROC (Free Response Receiver Operational Property) metric based on bounding box prediction was used. As used herein, a modified FROC metric was used to evaluate the localization performance of all compared methods, including the example methods of this disclosure and other baseline methods.

[0063] If the center of the bounding box at the fracture site is activated, the fracture site is considered recalled. Activated pixels outside the bounding box are considered false positives. Therefore, the modified FROC measures the fracture recall rate and the average ratio of false positive pixels per image. To calculate the modified FROC for the object detection method, the predicted bounding boxes are converted into binary masks using different thresholds, with pixels inside the predicted box being positive and pixels outside the box being negative. To quantify localization performance, the FROC score is calculated as the average recall rate across ten false positive rates ranging from 1% to 10%.

[0064] The example methods disclosed herein are compared with other baseline methods in the following three categories: 1) Weakly supervised methods: CheXNet, a representative state-of-the-art X-ray CAD method trained purely using image-level labels; 2) Object detection methods: including the anchor-based detector RetinaNet and the anchorless detector FCOS; 3) Semi-supervised methods: including II-Model, Temporal Ensemble, and Mean Teacher, as well as the state-of-the-art medical image SSL method. For all evaluated methods, ResNet-50 was used as the backbone network. Both RetinaNet and FCOS detection methods employed FPN. Table 1

[0065] Table 1 summarizes the quantitative results of all comparative methods compared to the example method of this disclosure. FROC scores are reported for localization performance. In the more challenging rib fracture detection task, Mean Teacher is the most competitive baseline method, with a measured AUROC of 0.9155 and an FROC score of 0.8540. The method of this disclosure measures an AUROC of 0.9318 and an FROC score of 0.8914, which significantly outperforms Mean Teacher by 1.63% in AUROC and 3.74% in FROC score.

[0066] Experiments based on the method disclosed herein show that, in rib / clavicle fracture detection, the area under the receiver operating characteristic curve (AUROC) is 0.9318 / 0.9646, and the free response receiver operating characteristic (FROC) score is 0.8914 / 0.9265. Compared with other state-of-the-art baseline methods, the method disclosed herein significantly improves AUROC by 1.63% / 0.86% and FROC by 3.74% / 3.81%, respectively, in rib / clavicle fracture detection.

[0067] Figure 7 The FROC curves for rib fracture (left) and clavicle fracture (right) detection results using different methods, including the example method of this disclosure, are shown. In the simpler clavicle fracture detection task, CheXNet and Li-method reported the highest AUROC (i.e., above 0.95) among the baseline methods. Mean Teacher provided the strongest FROC score of 0.8884 among the baseline methods. The method of this disclosure also outperformed all baseline methods on the clavicle fracture detection task, reporting an AUROC of 0.9646 and an FROC of 0.9265. Notably, the three knowledge distillation methods—II-Model, Temporal Ensemble, and Mean Teacher—outperformed supervised detection methods. The advantage was more significant on the easier clavicle fracture detection task. This is mainly because clavicle fractures have simpler geometric characteristics and similar visual patterns, allowing knowledge distillation methods to effectively learn from the pseudo-GT of unlabeled data. However, the advantage of knowledge distillation methods was less significant on the more complex rib fracture detection task. Due to the complex visual patterns of rib fractures and the limited positive region labeling data, pseudo-GT maps exhibit low sensitivity (i.e., the supervised pre-trained model reports a low FROC score of 0.7267), which restricts the knowledge transferred to the distillation model. Using AALS, the method disclosed in this paper effectively transfers more knowledge to the student model, thus achieving significantly improved performance compared to other knowledge distillation methods.

[0068] CheXNet and Li-method significantly outperformed other baseline knowledge distillation methods on the AUROC metric for clavicle fractures, but no performance advantage was observed on the AUROC metric for rib fractures. This is because CheXNet and Li-method specifically use positive image-level labels, while baseline knowledge distillation methods do not. In particular, CheXNet is trained through weakly supervised learning using only image-level labels, while Li-method utilizes positive image-level labels in a multi-instance learning manner.

[0069] Compared to publicly available methods, baseline knowledge distillation treats image-level positive films as unlabeled data. While weakly supervised learning and multi-instance learning are effective for learning simpler clavicle fractures, they perform poorly for more complex rib fractures. Furthermore, CheXNet and the Li-method exhibit poor localization performance. CheXNet provides localization visualization via class activation maps (CAMs). Since CAM values ​​cannot be compared across images, the FROC of CheXNet results cannot be computed. The Li-method does not report meaningful FROC scores because it consistently produces excessively large activation regions. For CheXNet and the Li-method, their poorer localization performance compared to other methods is qualitatively verified, such as… Figure 6 The example fracture detection results shown illustrate the fracture bounding boxes detected by GT and FCOS.

[0070] Example 4

[0071] The AALS disclosed herein was validated through experiments using different sharpening intensities A0 and center T. First, to analyze the effect of the marked sharpening center T, AALS was evaluated using T = 0.2, 0.3, 0.4, and 0.5; the results are summarized in Table 2. Optimal detection performance was obtained using T = 0.4, with the highest / second highest AUROC scores of 0.9318 / 0.9646 and the highest FROC scores of 0.8914 / 0.9265 in rib / clavicle fracture detection. Table 2 Table 3

[0072] Please note that for clavicle fracture classification, the best AUROC score of 0.9661 obtained at T=0.2 is only slightly better than the score at t=0.4. Sharpening centers represent a trade-off between sensitivity and specificity. It is noteworthy that the method of this disclosure consistently outperforms other baseline methods using all four T values. Secondly, as... Figure 4As shown in Table 3, the effect of sharpening intensity was studied by fixing the center T = 0.4 and evaluating A0 = 1, 4, 8, and 16. As summarized in Table 3, marker sharpening with an intensity A0 = 4 produced the best detection performance. For A0 = 1, no marker sharpening was applied, which led to a performance degradation. For A0 = 8 and 16, marker sharpening became too aggressive (e.g., ...). Figure 4 As shown in the image, this can also lead to false positives in the sharpened pseudo-GT, resulting in a slight performance degradation.

[0073] The involvement of the image-level positive set P was further investigated. Figure 5 The classification and detection performance of subsets of P using different proportions (0%, 20%, 60%, 100%) is shown for ribs and clavicles, where 0% and 100% correspond to supervised pre-trained student models and publicly available methods, respectively. It was observed that a larger P improved the classification AUROC and detection FROC scores. This confirms that CAD model training can benefit from utilizing image-level markers from clinical diagnoses.

[0074] Although exemplary embodiments have been described in the specification to illustrate the principles and implementations of this disclosure, the descriptions of the embodiments above are merely for the purpose of helping to understand the methods and core ideas of this disclosure. Furthermore, those skilled in the art can modify the specific implementations and application scope based on the ideas of this disclosure. Therefore, the content of this specification should not be construed as limiting the present invention.

Claims

1. A knowledge distillation method for fracture detection, characterized in that, include: Acquire medical images from chest X-rays, including region-level marked images, image-level diagnostic positives, and image-level diagnostic negatives; A supervised pre-training process is performed on the region-level labeled image and the image-level diagnostic negative to train the neural network to generate pre-trained weights; as well as A semi-supervised training process is performed on the image-level diagnostic positive image using the pre-trained weights, wherein a teacher model is used to generate a pseudo-ground reality ground truth (GT) on the image-level diagnostic positive image to supervise the training of the student model. The pseudo-ground reality GT is processed by an adaptive asymmetric label sharpening operator to generate a sharpened pseudo-ground reality GT to provide a positive detection response on the image-level diagnostic positive image. In the semi-supervised training process, the student model is trained via backpropagation, and the teacher model is iteratively updated during training using the exponential moving average of the student model weights. The teacher model weights are updated in the following manner: in and The weights of the teacher model and the student model in training step t are respectively represented, and α is a smoothing coefficient that controls the speed of knowledge update. Processing the pseudo-ground reality GT generated using a teacher model on the image-level diagnostic positive image x in an adaptive asymmetric marker sharpening operator to generate a sharpened pseudo-ground reality GT, wherein the sharpened pseudo-ground reality GT of the image-level diagnostic positive image x includes: Where y' represents the sharpened pseudo-ground real-world ground plane (GT). Let S(·) represent the teacher model in training step t, and let S(·) represent adaptive asymmetric label sharpening.

2. The method according to claim 1, characterized in that, The supervised pre-training process includes: For the region-level labeled image, a GT mask is generated by assigning "-" to pixels within the labeled bounding box and assigning "zero" elsewhere; For the image-level diagnostic negative, generate a GT mask consisting entirely of "zeros"; and The same supervised loss is used in the supervised pre-training process for the region-level labeled image and the image-level diagnostic negative.

3. The method according to claim 2, characterized in that: The same supervision loss includes pixel-level binary cross-entropy loss, where: BCE stands for binary cross-entropy. L sup This represents pixel-level binary cross-entropy loss. x represents the region-level labeled image or the image-level diagnostic negative. y represents the pixel-level supervision mask corresponding to the chest X-ray, and This represents the probability graph output of the neural network parameterized by θ.

4. The method according to claim 1, characterized in that, The method also includes: Each teacher and student model is initialized using pre-trained weights generated from the supervised pre-training process.

5. The method according to claim 1, characterized in that: The sharpened pseudo-ground real-time ground truth and the student model's predictions The KL divergence between them is calculated as an additional loss: ,and The total loss used to train the student network includes: ,in, L semi Indicates additional losses. L sup Let L represent the pixel-level binary cross-entropy loss, and L represent the total loss of the student network.

6. The method according to claim 5, characterized in that: The image-level diagnostic radiograph includes visible fracture sites; and The adaptive asymmetric marker sharpening on the image-level diagnostic positive film includes: , Where, expit(·) and logit(·) represent the Sigmoid function and its inverse function, respectively, and A and T control the sharpening intensity and sharpening center of the adaptive asymmetric marker sharpening operator, respectively. in: , y max A0 is the maximum probability in the pseudo-GT, and A0 is a hyperparameter that controls the maximum allowed sharpening intensity.

7. The method according to claim 6, characterized in that, The method also includes: For asymmetric sharpening, select a dynamic sharpening intensity A, and choose a sharpening center with a value less than 0.

5.

8. An electronic device, characterized in that, include: A memory containing computer programs stored thereon; as well as A processor coupled to the memory, and which, when the computer program is executed by the processor, implements the method as described in any one of claims 1 to 7.

9. A non-transitory computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 7.

Images