Knowledge distillation-based kidney stone classification method and classification system
By designing channel attention and cross-layer sample relationship distillation functions to update student network parameters, the problem of inaccurate kidney stone identification in existing technologies is solved, achieving accurate identification of kidney stone types and improving model accuracy.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- CHANGZHOU ANKANG MEDICAL EQUIP
- Filing Date
- 2026-01-23
- Publication Date
- 2026-04-21
AI Technical Summary
Existing knowledge distillation methods cannot accurately identify the type of kidney stone, effectively extract channel attention feature information and sample relationships, and fail to capture cross-layer structural difference knowledge.
By acquiring channel attention features and cross-layer sample relationships, a channel attention distillation loss function and a channel attention relationship distillation function are designed. Combined with cross-layer sample relationship knowledge distillation, the student network parameters are updated to improve the model's accuracy.
It improves the accuracy of kidney stone classification, enhances the performance of the small model in kidney stone identification, and can better identify different types of kidney stones.
Smart Images
Figure CN121904477A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of image recognition technology, and in particular relates to a kidney stone classification method and system based on knowledge distillation. Background Technology
[0002] The formation mechanism of kidney stones is complex. Common types of stones include calcium oxalate stones, calcium phosphate stones, uric acid stones, magnesium ammonium phosphate stones, and cystine stones. Different types of stones vary significantly in hardness, composition, and sensitivity to shock waves; therefore, accurate identification of the stone type is necessary.
[0003] In recent years, deep learning has been widely applied in computer vision, such as image classification, image segmentation, and object detection. As the depth and width of networks increase, the number of network parameters also increases. Larger models require more memory and computing power, and the hardware required for deployment is often very expensive. While more complex, larger models offer better accuracy than smaller models, they are difficult to deploy in end-user applications. Smaller models are easier to deploy, but their accuracy is somewhat lacking. Therefore, compressing models with minimal impact on performance has become a popular research area.
[0004] To make models more suitable for real-world applications, large models need to be compressed. Current model compression methods mainly include pruning, quantization, and knowledge distillation. Among these, knowledge distillation is a highly effective and feasible method. Originally proposed by Hinton, knowledge distillation transfers knowledge from a large model (teacher model) to a smaller model (student model), thereby improving the performance of the smaller model, achieving a balance between model accuracy and size, and making the model easier to apply on the terminal. However, using knowledge distillation to improve the ability of patterns to identify kidney stones has not yet been fully studied. Previous knowledge distillation methods have the following problems: 1. Directly using MSE to align channel attention fails to fully explore the relationships between channels in the attention feature information. 2. Failure to distill the relationships between samples based on channel attention features. 3. Current relational knowledge distillation work mainly transfers the sample relationships of peer layers, failing to capture cross-layer structural differences.
[0005] Therefore, since directly applying knowledge distillation to identify kidney stones cannot accurately identify the types of kidney stones, a new knowledge distillation-based kidney stone classification method and system needs to be designed.
[0006] It should be noted that the information disclosed in this background section is only for understanding the background technology of the present application concept, and therefore, the above description is not considered to constitute prior art information. Summary of the Invention
[0007] This disclosure provides at least one method and system for classifying kidney stones based on knowledge distillation.
[0008] In a first aspect, embodiments of this disclosure provide a method for classifying kidney stones based on knowledge distillation, including: The processor module preprocesses the original kidney stone images, then inputs the preprocessed images into the teacher and student networks. Channel attention features are obtained based on the output to obtain the channel attention distillation loss function. Relationships between images are constructed based on the channel attention features to obtain the channel attention relationship distillation function. Then, the channel attention cross-layer relationship distillation function is obtained based on the cross-layer sample relationship. After obtaining the total loss of network training, the network parameters are updated through the backpropagation algorithm. After training is completed, the classification probability of kidney stones is obtained based on the real-time acquired kidney stone images.
[0009] In one optional implementation, the method for preprocessing the original kidney stone image using a processor module includes: The processor module is configured to perform preprocessing operations on the dataset, namely cropping, scaling, and rotating the original kidney stone images in the dataset, and using the preprocessed kidney stone image data as input to the teacher network and the student network.
[0010] In one alternative implementation, the method of inputting the preprocessed kidney stone image into the teacher network and student network includes: The processor module is configured to input preprocessed kidney stone image data into the teacher network and the student network, and obtain intermediate features and logit outputs from the teacher network and the student network.
[0011] In one optional implementation, the method for obtaining channel attention features based on the output results to obtain the channel attention distillation loss function includes: The processor module is configured to acquire channel attention features through global average pooling: ; Among them, F T(l) For intermediate features of the teacher network; F S(l) The intermediate features of the student network are represented by T; the teacher network by S; the student network by l; and the index of the convolutional layer by GAP. ) represents the global average pooling operation; CA( () indicates channel alignment; Based on the obtained channel attention features, calculate the channel attention distillation loss function L. CAD : ; Where N is the number of samples in a mini-batch; i is the index of the sample; τ represents the temperature parameter; softmax( ) represents the softmax activation function; KL( , ) represents the Kullback-Leibler function.
[0012] In one optional implementation, the method for constructing relationships between images using channel attention features to obtain channel attention relationship distillation functions includes: The processor module is configured to construct relationships between images based on channel attention features: ; Among them, R T(l) and R S(l) The relation matrix for a mini-batch sample has a size of N*N; Obtain the channel attention relation distillation function L from the relation matrix. CARD : ; Among them, MSE ( , ) represents the mean squared error loss function.
[0013] In one optional implementation, the method for obtaining the channel attention cross-layer relationship distillation function based on the cross-layer sample relationship includes: The processor module is configured to transmit cross-layer relationship knowledge based on channel attention relationships. The cross-layer relationship is calculated as follows: ; Among them, D T(l) For cross-level relationships in the teacher network, F S(l) For cross-level relationships in the student network; The cross-layer sample relationship knowledge distillation representation based on channel attention is as follows: .
[0014] In one optional implementation, the method of updating network parameters via backpropagation after obtaining the total loss of network training includes: The processor module is configured to obtain the total loss during network training: ; Where α, β, and γ are hyperparameters of the balance loss term; L CE It is the cross-entropy loss between the student network and the prediction results; ; Where, H( , ) represents the cross-entropy loss function; z S For the logit output of the student network, y true Represents the true label of the image; Use L total Train the student network and update the network parameters through backpropagation; ; Where, θ S The parameter represents the student network, and δ represents the learning rate.
[0015] In one alternative implementation, the processor module is configured to input real-time acquired kidney stone images into the trained student network after training is completed, obtain the classification probability of the kidney stone images belonging to each type of kidney stone, and the type with the highest probability is taken as the classification result output by the student network.
[0016] Secondly, embodiments of this disclosure also provide a kidney stone classification system based on knowledge distillation, comprising: A preprocessing module is configured to preprocess the raw kidney stone images; The input module is configured to input preprocessed kidney stone images into the teacher network and the student network; The channel attention distillation loss function module is configured to obtain channel attention features based on the output to obtain the channel attention distillation loss function; The channel attention relation distillation function module is configured to construct relationships between images based on channel attention features to obtain channel attention relation distillation functions; The channel attention cross-layer relationship distillation function module is configured to obtain the channel attention cross-layer relationship distillation function based on the channel attention cross-layer relationship; The network parameter update module is configured to update the student network parameters through backpropagation of the total training loss; The output module is configured to obtain the classification probability of kidney stones based on real-time acquired kidney stone images after training is completed.
[0017] Thirdly, this disclosure also provides a computer program product, including a computer program / instruction that, when executed by a processor, implements the steps of the above-described knowledge distillation-based kidney stone classification method.
[0018] The beneficial effects of this invention are as follows: This knowledge distillation-based kidney stone classification method preprocesses the original kidney stone images using a processor module, then inputs the preprocessed kidney stone images into the teacher network and student network. Channel attention features are obtained based on the output results to obtain the channel attention distillation loss function. Furthermore, relationships between images are constructed based on the channel attention features to obtain the channel attention relationship distillation function. Then, the channel attention cross-layer relationship distillation function is obtained based on the cross-layer sample relationships. After obtaining the total loss of network training, the network parameters are updated using a backpropagation algorithm. Finally, after training, the classification probability of kidney stones is obtained based on real-time acquired kidney stone images, thereby improving the model's performance in kidney stone classification and facilitating accurate identification of kidney stone types.
[0019] Other features and advantages of the invention will be set forth in the following description, and will be apparent in part from the description, or may be learned by practicing the invention. The objects and other advantages of the invention are realized and obtained through the structures particularly pointed out in the description and the drawings.
[0020] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, preferred embodiments are described in detail below with reference to the accompanying drawings. Attached Figure Description
[0021] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.
[0022] Figure 1 A flowchart of a knowledge distillation-based kidney stone classification method provided in this disclosure embodiment; Figure 2 This is a block diagram illustrating the principle of a classification system provided in an embodiment of this disclosure. Detailed Implementation
[0023] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0024] As used herein, the phrases “in one embodiment,” “according to one embodiment,” “in some embodiments,” etc., generally refer to the fact that a particular feature, structure, or characteristic following the phrase can be included in at least one embodiment of this disclosure. Therefore, a particular feature, structure, or characteristic can be included in more than one embodiment of this disclosure, such that these phrases do not necessarily refer to the same embodiment. As used herein, the terms “example,” “exemplary,” etc., are used to “serve as an example, instance, or illustration.” Any implementation, aspect, or design described herein as “example” or “exemplary” is not necessarily to be construed as preferred or superior to other implementations, aspects, or designs. Rather, the use of the terms “example,” “exemplary,” etc., is intended to present concepts in a specific manner.
[0025] The formation mechanism of kidney stones is complex and closely related to various factors such as metabolic abnormalities, infections, and dietary structure. Common types of stones include calcium oxalate stones, calcium phosphate stones, uric acid stones, magnesium ammonium phosphate stones, and cystine stones. Different types of stones vary significantly in hardness, composition, and sensitivity to shock waves.
[0026] In recent years, deep learning has been widely applied in computer vision, such as image classification, image segmentation, and object detection. As the depth and width of networks increase, the number of network parameters also increases. Larger models require more memory and computing power, and the hardware required for deployment is often very expensive. While more complex, larger models offer better accuracy than smaller models, they are difficult to deploy in end-user applications. Smaller models are easier to deploy, but their accuracy is somewhat lacking. Therefore, compressing models with minimal impact on performance has become a popular research area.
[0027] Deep learning has been widely applied in the field of medical imaging. This paper proposes a lightweight gradient booster to identify infectious and non-infectious kidney stones by combining radiomics features with deep features from convolutional neural networks. The kidneys are first segmented using a 3D U-Net, followed by gradient-based anisotropic denoising, thresholding, and region growing. Finally, a 13-layer convolutional neural network classifier distinguishes between kidney stones and false positives. StoneNet is proposed, employing global average pooling instead of traditional flattening layers to significantly reduce the number of parameters and improve model robustness.
[0028] To make models more suitable for real-world applications, large models need to be compressed. Current model compression methods mainly include pruning, quantization, and knowledge distillation. Among these, knowledge distillation is a highly effective and feasible method. Originally proposed by Hinton, knowledge distillation transfers knowledge from a large model (teacher model) to a smaller model (student model), thereby improving the performance of the smaller model, achieving a balance between model accuracy and size, and making the model easier to apply on the terminal. However, using knowledge distillation to improve the ability of patterns to identify kidney stones has not yet been fully studied. Previous knowledge distillation methods have the following problems: 1. Directly using MSE to align channel attention fails to fully explore the relationships between channels in the attention feature information. 2. Failure to distill the relationships between samples based on channel attention features. 3. Current relational knowledge distillation work mainly transfers the sample relationships of peer layers, failing to capture cross-layer structural differences.
[0029] It should be noted that similar reference numerals and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. Some embodiments of the present invention will now be described in detail with reference to the accompanying drawings. Unless otherwise specified, the following embodiments and features can be combined with each other.
[0030] like Figure 1 As shown, at least one disclosed embodiment provides a kidney stone classification method based on knowledge distillation, including: preprocessing the original kidney stone image through a processor module, then inputting the preprocessed kidney stone image into a teacher network and a student network, obtaining channel attention features based on the output results to obtain a channel attention distillation loss function; constructing relationships between images based on the channel attention features to obtain a channel attention relationship distillation function, then obtaining a channel attention cross-layer relationship distillation function based on cross-layer sample relationships, updating network parameters through a backpropagation algorithm after obtaining the total loss of network training, and obtaining the classification probability of kidney stones based on real-time acquired kidney stone images after training, thereby improving the model's performance in kidney stone classification and facilitating accurate identification of kidney stone types.
[0031] In one alternative implementation, the method for preprocessing the original kidney stone images via the processor module includes: the processor module being configured to perform preprocessing operations on the dataset, namely cropping, scaling, and rotating the original kidney stone images in the dataset, and using the preprocessed kidney stone image data as input to the teacher network and the student network.
[0032] In one alternative implementation, the method of inputting the preprocessed kidney stone image into the teacher network and the student network includes: the processor module being configured to input the preprocessed kidney stone image data into the teacher network and the student network, and to obtain intermediate features and logit outputs of the teacher network and the student network.
[0033] In one optional implementation, the method for obtaining channel attention features based on the output results to obtain the channel attention distillation loss function includes: the processor module is configured to obtain the channel attention features through Global Average Pooling (GAP). ; Among them, F T(l) For intermediate features of the teacher network; F S(l) The intermediate features of the student network are represented by T; the teacher network by S; the student network by l; and the index of the convolutional layer by GAP. ) represents the global average pooling operation; CA( () indicates channel alignment; 1*1 convolution, batch normalization, and ReLU function are used to align the number of channels in the teacher-student network; Based on the obtained channel attention features, calculate the channel attention distillation loss function L. CAD : ; Where N is the number of samples in a mini-batch; i is the index of the sample; τ represents the temperature parameter; softmax( ) represents the softmax activation function; KL( , () represents the Kullback-Leibler function. Using temperature-softened channel attention allows for a greater representation of the similarity information between channels.
[0034] In one optional implementation, the method for constructing relationships between images based on channel attention features to obtain a channel attention relationship distillation function includes: passing sample relationships helps student networks learn discriminative information between samples, making them more applicable to different downstream tasks; the processor module is configured to construct relationships between images based on channel attention features. ; Among them, R T(l) and R S(l) The relation matrix for a mini-batch sample has a size of N*N; Obtain the channel attention relation distillation function L from the relation matrix. CARD : ; Among them, MSE ( , ) represents the mean squared error loss function.
[0035] In an optional implementation, the method for obtaining the channel attention cross-layer relationship distillation function based on the cross-layer sample relationship includes: the cross-layer relationship can represent the sample relationship process between different network layers; existing methods ignore the knowledge transfer of cross-layer relationships; the processor module is configured to transfer cross-layer relationship knowledge based on the channel attention relationship, and the cross-layer relationship is calculated as follows: ; Among them, D T(l) For cross-level relationships in the teacher network, F S(l) For cross-level relationships in the student network; The sample relationships constructed by channel attention, and the differences in sample structure between the front and back layers of the teacher-student network; The cross-layer sample relationship knowledge distillation representation based on channel attention is as follows: .
[0036] In one optional implementation, the method of updating network parameters via backpropagation after obtaining the total loss of network training includes: the processor module being configured to obtain the total loss of network training. ; Where α, β, and γ are hyperparameters of the balance loss term; L CE It is the cross-entropy loss between the student network and the prediction results; ; Where, H( , ) represents the cross-entropy loss function; z S For the logit output of the student network, y true Represents the true label of the image; Use L total Train the student network and update the network parameters through backpropagation; ; Where, θ S The parameter represents the student network, and δ represents the learning rate.
[0037] In one alternative implementation, the processor module is configured to input real-time acquired kidney stone images into the trained student network after training is completed, obtain the classification probability of the kidney stone images belonging to each type of kidney stone, and the type with the highest probability is taken as the classification result output by the student network.
[0038] In this embodiment, 300 kidney stone images were used for training out of 500 images obtained in collaboration with medical institutions, and 200 images were used for testing. Using a teacher network (ResNet14) for training and a student network (ResNet8), a classification success rate of 75.4% was achieved on the test dataset, a 13.3% improvement over the baseline result of 62.1%, validating the effectiveness of the proposed method.
[0039] like Figure 2 As shown, at least one other disclosed embodiment also provides a kidney stone classification system based on knowledge distillation, comprising: a preprocessing module configured to preprocess the original kidney stone image; an input module configured to input the preprocessed kidney stone image into a teacher network and a student network; a channel attention distillation loss function module configured to obtain channel attention features based on the output result to obtain a channel attention distillation loss function; a channel attention relationship distillation function module configured to construct the relationship between images based on the channel attention features to obtain a channel attention relationship distillation function; a channel attention cross-layer relationship distillation function module configured to obtain a channel attention cross-layer relationship distillation function based on the cross-layer relationship of channel attention; a network parameter update module configured to update the student network parameters through backpropagation of the total training loss; and an output module configured to obtain the classification probability of the kidney stone based on the real-time acquired kidney stone image after training is completed.
[0040] At least one other disclosed embodiment also provides a computer program product, including a computer program / instructions that, when executed by a processor, implement the steps of the knowledge distillation-based kidney stone classification method described above.
[0041] In summary, this knowledge distillation-based kidney stone classification method preprocesses the original kidney stone images using a processor module. The preprocessed images are then input into the teacher and student networks. Channel attention features are obtained from the output results to acquire the channel attention distillation loss function. Furthermore, relationships between images are constructed based on these channel attention features to obtain the channel attention relationship distillation function. Then, the cross-layer channel attention relationship distillation function is obtained based on cross-layer sample relationships. The total training loss is calculated, and the student network parameters are updated via backpropagation. Finally, after training, the classification probability of kidney stones is obtained based on real-time acquired kidney stone images. This improves the model's performance in kidney stone classification, facilitating accurate identification of kidney stone types.
[0042] The disclosures and other solutions, examples, embodiments, modules, and functional operations described in this document can be implemented in digital electronic circuits, or computer software, firmware, or hardware, including the structures disclosed in this document and their structural equivalents, or combinations thereof. The disclosures and other embodiments can be implemented as one or more computer program products, i.e., one or more modules of computer program instructions encoded on a tangible and non-volatile computer-readable medium for execution by a data processing apparatus or for controlling the operation of a data processing apparatus. The computer-readable medium can be a machine-readable storage device, a machine-readable storage substrate, a storage device, a material composition that influences machine-readable propagated signals, or a combination thereof. In addition to hardware, the apparatus may also include code that creates an execution environment for a computer program, such as code constituting processor firmware, a protocol stack, a database management system, an operating system, or a combination thereof. The propagated signals are artificially generated signals, such as machine-generated electrical, optical, or electromagnetic signals, which are generated to encode information for transmission to a suitable receiver device.
[0043] Computer programs (also known as programs, software, software applications, scripts, or code) can be written in any programming language (including compiled or interpreted languages) and can be deployed in any form, including as standalone programs or as modules, components, subroutines, or other units suitable for use in a computing environment. A computer program does not necessarily correspond to a file in a file system. A program can be stored in a portion of a file that holds other programs or data (e.g., one or more scripts stored in a markup language document), in a single file dedicated to that program, or in multiple coordinating files (e.g., a file storing one or more modules, subroutines, or portions of code). Computer programs can be deployed and executed on one or more computers located at a single site or distributed across multiple sites interconnected by a communication network.
[0044] The processing and logic flows described in this document can be executed by one or more programmable processors that execute one or more computer programs to perform functions by manipulating input data and generating outputs. The processing and logic flows can also be executed by special-purpose logic circuitry, and the devices can be implemented as special-purpose logic circuitry, such as FPGAs (Field-Programmable Gate Arrays) or ASICs (Application-Specific Integrated Circuits).
[0045] For example, processors suitable for executing computer programs include general-purpose and special-purpose microprocessors, as well as any one or more of any type of digital computer. Typically, the processor receives instructions and data from read-only memory or random access memory, or both. The basic components of a computer are a processor that executes instructions and one or more storage devices that store the instructions and data. Typically, a computer will also include one or more mass storage devices for storing data, such as magnetic disks, magneto-optical disks, or optical disks, or operatively coupled to receive data from or transfer data to mass storage devices, or both. However, a computer does not necessarily have such devices. Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media, and memory devices, including, for example, semiconductor memory devices such as erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), and flash memory devices; magnetic disks, such as internal hard disks or removable disks; magneto-optical disks; and optical disc read-only memory (CD ROM) and digital versatile optical disc read-only memory (DVD-ROM). The processor and memory may be supplemented by dedicated logic circuitry or integrated into dedicated logic circuitry.
[0046] While several embodiments are provided in this disclosure, it should be understood that the disclosed systems and methods may be embodied in many other specific forms without departing from the spirit or scope of this disclosure. The present examples are intended to be illustrative rather than restrictive and are not limited to the details given. For example, various elements or components may be combined or integrated into another system, or certain features may be omitted or not implemented.
[0047] In the several embodiments provided herein, it should be understood that the disclosed apparatus and methods can also be implemented in other ways. The apparatus embodiments described above are merely illustrative; for example, the flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram and / or flowchart, and combinations of blocks in block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.
[0048] Based on the above-described preferred embodiments of the present invention, and through the foregoing description, those skilled in the art can make various changes and modifications without departing from the inventive concept. The technical scope of this invention is not limited to the contents of the specification, but must be determined according to the scope of the claims.
Claims
1. A method for classifying kidney stones based on knowledge distillation, characterized in that, include: The original kidney stone image is preprocessed by the processor module, and then the preprocessed kidney stone image is input into the teacher network and the student network. Channel attention features are obtained based on the output results to obtain the channel attention distillation loss function; and... The relationship between images is constructed based on the channel attention features to obtain the channel attention relationship distillation function. Then, the channel attention cross-layer relationship distillation function is obtained based on the cross-layer sample relationship. After obtaining the total loss of network training, the network parameters are updated through the backpropagation algorithm. After training is completed, the classification probability of kidney stones is obtained based on the real-time acquired kidney stone images.
2. The kidney stone classification method based on knowledge distillation as described in claim 1, characterized in that, The method for preprocessing the original kidney stone image using a processor module includes: The processor module is configured to perform preprocessing operations on the dataset, namely cropping, scaling, and rotating the original kidney stone images in the dataset, and using the preprocessed kidney stone image data as input to the teacher network and the student network.
3. The kidney stone classification method based on knowledge distillation as described in claim 2, characterized in that, The method for inputting the preprocessed kidney stone image into the teacher network and student network includes: The processor module is configured to input preprocessed kidney stone image data into the teacher network and the student network, and obtain intermediate features and logit outputs from the teacher network and the student network.
4. The kidney stone classification method based on knowledge distillation as described in claim 3, characterized in that, The method for obtaining channel attention features based on the output results to obtain the channel attention distillation loss function includes: The processor module is configured to acquire channel attention features through global average pooling: ； Among them, F T(l) For intermediate features of the teacher network; F S(l) The intermediate features of the student network are represented by T; the teacher network by S; the student network by l; and the index of the convolutional layer by GAP. ) represents the global average pooling operation; CA( () indicates channel alignment; Based on the obtained channel attention features, calculate the channel attention distillation loss function L. CAD : ； Where N is the number of samples in a mini-batch; i is the index of the sample; τ represents the temperature parameter; softmax( ) represents the softmax activation function; KL( , ) represents the Kullback-Leibler function.
5. The kidney stone classification method based on knowledge distillation as described in claim 4, characterized in that, The method for constructing relationships between images using channel attention features to obtain channel attention relationship distillation functions includes: The processor module is configured to construct relationships between images based on channel attention features: ； Among them, R T(l) and R S(l) The relation matrix for a mini-batch sample has a size of N*N; Obtain the channel attention relation distillation function L from the relation matrix. CARD : ； Among them, MSE ( , ) represents the mean squared error loss function.
6. The kidney stone classification method based on knowledge distillation as described in claim 5, characterized in that, The method for obtaining the channel attention cross-layer relationship distillation function based on cross-layer sample relationships includes: The processor module is configured to transmit cross-layer relationship knowledge based on channel attention relationships. The cross-layer relationship is calculated as follows: ； Among them, D T(l) For cross-level relationships in the teacher network, F S(l) For cross-layer relationships in student networks; The cross-layer sample relationship knowledge distillation representation based on channel attention is as follows: 。 7. The kidney stone classification method based on knowledge distillation as described in claim 6, characterized in that, The method for updating network parameters using the backpropagation algorithm after obtaining the total loss of network training includes: The processor module is configured to obtain the total loss during network training: ； Where α, β, and γ are hyperparameters of the balance loss term; L CE It is the cross-entropy loss between the student network and the prediction results; ； Where, H( , ) represents the cross-entropy loss function; z S For the logit output of the student network, y true Represents the true label of the image; Use L total Train the student network and update the network parameters through backpropagation; ； Where, θ S The parameter represents the student network, and δ represents the learning rate.
8. The kidney stone classification method based on knowledge distillation as described in claim 7, characterized in that, The processor module is configured to input real-time acquired kidney stone images into the trained student network after training is completed, obtain the classification probability of the kidney stone images belonging to each type of kidney stone, and the type with the highest probability is the classification result output by the student network.
9. A kidney stone classification system based on knowledge distillation, characterized in that, include: A preprocessing module is configured to preprocess the raw kidney stone images; The input module is configured to input preprocessed kidney stone images into the teacher network and the student network; The channel attention distillation loss function module is configured to obtain channel attention features based on the output to obtain the channel attention distillation loss function; The channel attention relation distillation function module is configured to construct relationships between images based on channel attention features to obtain channel attention relation distillation functions; The channel attention cross-layer relationship distillation function module is configured to obtain the channel attention cross-layer relationship distillation function based on the channel attention cross-layer relationship; The network parameter update module is configured to update the student network parameters through backpropagation of the total training loss; The output module is configured to obtain the classification probability of kidney stones based on real-time acquired kidney stone images after training is completed.
10. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instruction is executed by the processor, it implements the steps of the knowledge distillation-based kidney stone classification method according to any one of claims 1-8.