Knowledge distillation method and device based on category perception boundary optimization, and storage medium

By introducing target class decoupling loss, non-target class decoupling loss, and category-aware boundary loss into the knowledge distillation network, the category-aware boundary of the student network is optimized, solving the problem that the student network in the prior art has difficulty distinguishing category boundaries, and achieving stronger classification discrimination ability and higher fine-grained image classification accuracy.

CN122242645APending Publication Date: 2026-06-19QILU UNIVERSITY OF TECHNOLOGY (SHANDONG ACADEMY OF SCIENCES)
View PDF 0 Cites 0 Cited by

Patent Information

Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
QILU UNIVERSITY OF TECHNOLOGY (SHANDONG ACADEMY OF SCIENCES)
Filing Date
2026-05-21
Publication Date
2026-06-19

AI Technical Summary

Technical Problem

Existing decoupled knowledge distillation methods do not optimize category boundaries, making it difficult for student networks to effectively distinguish the boundaries of different categories during the knowledge distillation process. This results in poor classification ability, especially in fine-grained image classification tasks where it is difficult to obtain accurate image classification results.

Method used

A knowledge distillation network is constructed to optimize the category-aware boundary of the student network through target class decoupling loss, non-target class decoupling loss, and category-aware boundary loss. This includes calculating the feature cosine similarity between the teacher network and the student network, concatenating the distribution difference coefficients, applying linear transformations to the fully connected layers, and processing with batch normalization layers to optimize the category prediction probability distribution of the student network.

Benefits of technology

It enhances the classification and discrimination capabilities of student networks, enabling more accurate identification of similar categories and improving the accuracy of fine-grained image classification.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN122242645A_ABST
    Figure CN122242645A_ABST
Patent Text Reader

Abstract

This invention discloses a knowledge distillation method, device, and storage medium based on category-aware boundary optimization, relating to the fields of machine learning and computer vision. In this application, the category difficulty coefficient and the difference coefficient are concatenated into an input vector; a fully connected layer performs a linear transformation and learnable weighted fusion on the input vector, outputting sample-level adaptive intermediate parameters; the sample-level adaptive parameters are multiplied element-wise by a stage decay coefficient to obtain batch-level intermediate parameters; a batch normalization layer processes the batch-level intermediate parameters to obtain global adaptive boundary parameters; a category-aware boundary loss is calculated based on the global adaptive boundary parameters; and the knowledge distillation network is trained based on the training set, validation set, and the total network loss including the boundary loss to obtain a knowledge distillation network model. The method described in this application can optimize the category-aware boundary of a student network, resulting in a student network model with stronger classification and discrimination capabilities.
Need to check novelty before this filing date? Find Prior Art