Low-complexity environmental sound recognition method and system based on course temperature self-distillation

Abstract

The invention relates to the technical field of artificial intelligence, in particular to a low-complexity environmental sound recognition method and system based on course temperature self-distillation, and the method comprises the steps: constructing a course temperature self-distillation model based on a lightweight backbone network, and constructing a teacher model and a student model based on the backbone network; a learning parameter is distributed for each training sample, a course temperature parameter is obtained through re-parameterization processing, a total loss function containing teacher loss and student loss is constructed, and a backbone network parameter, a student branch parameter and the course temperature parameter are jointly optimized through a gradient descent algorithm based on the total loss function; after training is completed, all student branches and course temperature parameters are removed, a backbone network and a final classifier are reserved, and an input sample is processed through the backbone network and the final classifier to obtain a recognition result; according to the invention, the recognition precision and efficiency of the low-complexity environment sound recognition system can be improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, in particular to a low-complexity environmental sound recognition method and system based on curriculum temperature self-distillation. BACKGROUND

[0002] Environmental sound recognition (ESR) technology is one of the core applications of edge artificial intelligence (EdgeAI), which aims to identify specific acoustic events (such as glass breaking, baby crying) or scenes (such as parks, streets) from continuous audio signals. With the popularity of the Internet of Things (IoT) and smart hardware, ESR technology is widely used in: intelligent security field, to realize active early warning by identifying glass breaking, abnormal shouting, etc.; in the smart home scenario, to identify baby crying, abnormal operation sound of household appliances to provide intelligent services; in the industrial internet, to identify abnormal operation sound of equipment to realize predictive maintenance; in health care, to identify falling sound and loud coughing sound to ensure the safety of the elderly. These application scenarios have common and stringent requirements, i.e. the recognition model must be deployed on embedded devices or edge nodes with highly limited computing power, memory and power consumption, and realize real-time, high-precision and low-power inference, which has led to an urgent need for low-complexity environmental sound recognition (Low-Complexity ESR) technology. The core challenge is how to maximize the recognition accuracy and robustness of the model under the constraint of limited model complexity (parameter quantity and calculation quantity). SUMMARY

[0003] To solve the above problems, the present application provides a low-complexity environmental sound recognition method and system based on curriculum temperature self-distillation, which introduces a curriculum temperature self-distillation mechanism to realize sample-level adaptive distillation, establishes an intelligent curriculum learning mechanism, improves the efficiency and quality of knowledge transfer, and at the same time maintains the requirement of low complexity deployment, providing a high-precision and high-efficiency solution for environmental sound recognition on edge devices.

[0004] To achieve the above purpose, the present application provides the following technical solutions: On the one hand, the present application provides a low-complexity environmental sound recognition method based on curriculum temperature self-distillation, which comprises the following steps: S100, a curriculum temperature self-distillation model based on a lightweight backbone network is constructed, and a teacher model and a student model are constructed based on the backbone network; S200, a learnable parameter is assigned to each training sample, and a curriculum temperature parameter is obtained through reparameterization processing, which is used to modulate the softening intensity in the knowledge distillation process; S300, constructing a total loss function comprising a teacher loss and a student loss, the teacher loss being a cross-entropy loss of the teacher model, and the student loss comprising a task loss, a feature distillation loss and a logical distillation loss of the student model; S400, jointly optimizing the backbone network parameters, the student branch parameters and the curriculum temperature parameters based on the total loss function by a gradient descent algorithm; S500, removing all the student branches and the curriculum temperature parameters after the training is completed, retaining the backbone network and the final classifier, and processing an input sample through the backbone network and the final classifier to obtain a recognition result.

[0005] Optionally, in S100, the teacher model and the student model are constructed based on the backbone network, comprising: S110, dividing the backbone network into K consecutive modules in terms of depth, and for an input sample, the teacher model being a combination of output features of a deepest layer, i.e., the Kth module, and the final classifier, wherein the output of the teacher model for the input sample is a teacher logical output; S120, adding student branches respectively after the first K-1 shallow or middle modules of the backbone network, each student branch comprising a student module and a student classifier, the student module being composed of an attention module and a feature alignment network, and being used for converting and enhancing the shallow layer features output by the corresponding module to align with the teacher feature dimension, the student classifier being a fully connected layer and being used for outputting a student logical value, and the kth student model being composed of the student module and the student classifier after the kth module.

[0006] Optionally, in S200, the curriculum temperature parameter is obtained through a reparameterization process, comprising: S210, assigning a learnable parameter to each training sample; S220, processing the learnable parameter through function, and adding a very small normal number to obtain the curriculum temperature parameter, the calculation formula being: ; wherein, is a very small normal number, the function ensures that the output is always positive, is the learnable parameter of the training sample , , is the index of the training sample.

[0007] Optionally, in S300, the total loss function comprising the teacher loss and the student loss is constructed, comprising: S310, obtaining a cross-entropy loss of the teacher model itself as the teacher loss, the teacher loss being obtained by calculating the cross-entropy of the predicted output of the teacher model for a training sample and the real label; S320, obtain the task loss, the feature distillation loss and the logical distillation loss of each student branch as the student loss, wherein the task loss is obtained by calculating the cross entropy of the predicted output of the student model on the training sample and the real label, the feature distillation loss is obtained by calculating the L2 loss between the teacher feature and the student feature converted by the student module, and the logical distillation loss is obtained by calculating the KL divergence between the distribution of the student logical value softened by the course temperature parameter and the distribution of the teacher logical output softened by the course temperature parameter; S330, weight and sum the teacher loss and the student loss of each student branch to obtain the total loss function.

[0008] Optionally, in S310, the calculation process of the teacher loss is: For each training sample in the training sample set, obtain the component corresponding to the real label position in the teacher logical output output by the teacher model, calculate the logarithm of the ratio of the exponential value of the component to the sum of the exponential values of all components of the teacher logical output, take the average value of the logarithm results of all training samples and take the negative value as the teacher loss.

[0009] Optionally, in S320, the calculation process of the student loss is: S321, for each training sample in the training sample set, obtain the component corresponding to the real label position in the student logical value output by the student model, calculate the logarithm of the ratio of the exponential value of the component to the sum of the exponential values of all components of the student logical value, take the average value of the logarithm results of all training samples and take the negative value as the task loss; S322, for each training sample in the training sample set, obtain the teacher feature output by the teacher model and the student feature converted by the student module, calculate the square of the L2 norm between the teacher feature and the student feature, and take the average value of the L2 norm square results of all training samples as the feature distillation loss; S323, for each training sample in the training sample set, divide the student logical value and the teacher logical output by the course temperature parameter of the sample respectively to obtain the student distribution and the teacher distribution by the Softmax function, calculate the KL divergence between the student distribution and the teacher distribution, and take the average value of the KL divergence results of all training samples as the logical distillation loss; S324, multiply the task loss, the feature distillation loss and the logical distillation loss by the corresponding weight coefficients respectively and add them to obtain the student loss of a single student branch.

[0010] Optionally, in S400, the process of joint optimization is: Initialize the backbone network parameters, student branch parameters, and course temperature parameters. In each training iteration, calculate the total loss function based on the current parameters. Calculate the gradient of the total loss function with respect to the backbone network parameters, student branch parameters, and course temperature parameters using the gradient descent algorithm. Update the backbone network parameters, student branch parameters, and course temperature parameters according to the preset learning rate. Repeat the iterative process until the model training is complete.

[0011] Optionally, in S500, the step of processing the input samples through a backbone network and a final classifier to obtain the recognition result includes: S510: Input samples are fed into the preserved backbone network, and features are extracted from the input samples through the backbone network to obtain deep features; S520, the deep features are input into the final classifier, the final classifier performs classification processing on the deep features, and outputs the predicted label corresponding to the input sample. The predicted label is the recognition result of the environmental sound.

[0012] pass The function calculates the probability distribution of a sample belonging to each category, and selects the category with the highest probability value as the predicted label for the input sample. This predicted label is the recognition result of the environmental sound. The formula for calculating the predicted label is: ;in, It is a predicted label. This indicates that the backbone network responds to the input samples. The output features obtained after feature extraction yes function, The category with the highest probability is selected as the final prediction result.

[0013] On the other hand, embodiments of the present invention provide a low-complexity environmental sound recognition system based on course temperature self-distillation, comprising: At least one processor; At least one memory for storing at least one program; When the at least one program is executed by the at least one processor, the at least one processor performs the method described above.

[0014] On the other hand, embodiments of the present invention provide a computer-readable storage medium storing a processor-executable program, which, when executed by a processor, is used to perform the above-described method.

[0015] The beneficial effects of the present application are: the present application discloses a low-complexity environmental sound recognition method and system based on course temperature self-distillation, and the present application dynamically allocates a learnable course temperature parameter for each training sample by introducing a course temperature self-distillation mechanism, realizes sample-level adaptive distillation, can accurately modulate the softening intensity in the knowledge distillation process according to the feature complexity and learning difficulty of different environmental sound samples, and thus optimizes the knowledge transfer efficiency. At the same time, a joint optimization framework containing teacher loss and student loss is constructed, the cross-entropy loss of the teacher model and the task loss, feature distillation loss and logical distillation loss of the student model are weighted and fused, and the student model is guided to fully learn discriminative knowledge from the deep features and output logic of the teacher model. In the model structure design, multiple modules are divided based on the lightweight backbone network, the student branch containing the attention mechanism and the feature alignment network is added after the shallow or middle layer module, the multi-stage and multi-level knowledge transfer is realized, and the representation ability of the student model to the environmental sound features is effectively improved. After training, only the backbone network and the final classifier are retained, all student branches and course temperature parameters are removed, the low-complexity deployment requirement is ensured, and the high-precision recognition performance of the model is maintained. The method not only provides an environmental sound recognition solution with high precision and high efficiency for edge devices, but also has good expansibility and adaptability through modular design and general framework, and can be flexibly applied to different types of edge computing scenes and environmental sound recognition tasks. BRIEF DESCRIPTION OF DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0017] Figure 1 is a flowchart of a low-complexity environmental sound recognition method based on course temperature self-distillation in an embodiment of the present application;

[0018] Figure 2 is a whole architecture diagram of a net cage recognition model in an embodiment of the present application;

[0019] Figure 3 is a mechanism flowchart of course temperature in an embodiment of the present application. DETAILED DESCRIPTION

[0020] The disclosed concept, specific structure and generated technical effects will be described clearly and completely in combination with embodiments and drawings to fully understand the purposes, schemes and effects of the present disclosure. It should be noted that the embodiments in the present application and the features in the embodiments can be combined with each other without conflict.

[0021] In order to make the purposes, technical schemes and advantages of the present application clearer, the present application will be further described in combination with the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and do not limit the present application. When the following description relates to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementation described in the following exemplary embodiments does not represent all the implementations consistent with the embodiments of the present application. They are only examples of devices and methods consistent with some aspects of the embodiments of the present application as described in the appended claims.

[0022] It can be understood that the terms "first", "second" and the like used in the present application can be used herein to describe various concepts, but unless specifically stated, these concepts are not limited by these terms. These terms are only used to distinguish one concept from another. For example, without departing from the scope of the embodiments of the present application, the first information can also be referred to as the second information, and similarly, the second information can also be referred to as the first information. Depending on the context, the word "if" as used herein can be interpreted as "when" or "when" or "in response to determining".

[0023] The terms "at least one", "multiple", "each", "any" and the like used in the present application include one, two or more than two, multiple includes two or more than two, each refers to each of the corresponding multiple, and any refers to any one of the multiple.

[0024] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as understood by those skilled in the art to which the present application belongs. The terms used herein are only for the purpose of describing the embodiments of the present application and are not intended to limit the present application.

[0025] The technical terms related to the present application will be explained first as follows:

[0026] Environmental sound recognition (ESR): a broad field of audio recognition, aiming to recognize and classify various sound events (such as glass breaking, baby crying) or scenes (such as park, street) in audio streams. It includes but is not limited to acoustic scene classification (ASC), sound event detection (SED), abnormal sound detection, etc.

[0027] Knowledge Distillation (KD): A model compression technique that improves the performance of a lightweight student model by having it mimic the output or intermediate features of a large teacher model.

[0028] Self-Distillation (SD): A form of knowledge distillation where the teacher and student models originate from different parts or training stages of the same model.

[0029] Hard Labels: A form of label that is typically a one-hot vector (e.g., [1, 0, 0]) indicating only the unique correct class for a sample. It provides an absolute decision.

[0030] Soft Labels: A form of label that is a probability distribution vector (e.g., [0.9, 0.09, 0.01]) representing the model's confidence in each class for a sample. It provides a probabilistic insight with rich information.

[0031] Temperature Parameter (T): A hyperparameter in knowledge distillation used to control the smoothness of the output distribution. Increasing the temperature (T > 1) produces smoother soft labels, revealing the similarity relationships between classes (i.e., "dark knowledge").

[0032] Curriculum Temperature: The core concept proposed by the invention, referring to the adaptive temperature parameter dynamically assigned to each training sample, which is automatically learned through gradient descent algorithm during the training process.

[0033] In related technologies, although knowledge distillation and self-distillation techniques have achieved significant success, they both rely on a global and fixed temperature parameter T. This static, "one-size-fits-all" strategy has fundamental theoretical flaws, especially in complex ESR tasks, which are amplified:

[0034] Inability to cope with the huge sample diversity in ESR tasks: Environmental sound data has high complexity. A clear "alarm sound" sample and a "street noise" sample mixed with wind and rain sound have a huge difference in learning difficulty. Fixed temperature cannot achieve differentiated teaching: for simple samples, too high distillation strength may be redundant; while for truly difficult samples, fixed softening strength may not be enough to provide effective supervision.

[0035] Inability to adapt to the dynamics of model training: the model's perception of sample difficulty changes dynamically at different training stages. A sample considered difficult in the early stages of training may become easy after the model's capabilities improve. The fixed temperature strategy cannot achieve an intelligent curriculum learning process from easy to difficult.

[0036] Limitations on the efficiency of knowledge transfer: The above limitations make it difficult for traditional distillation strategies to maximize the potential of lightweight models, resulting in a bottleneck in performance improvement. In order to achieve the ultimate goal of low-complexity ESR, this bottleneck must be broken through.

[0037] Based on the in-depth study of current knowledge distillation and self-distillation techniques, we find that the existing methods have the following inherent limitations:

[0038] (1) Static temperature strategy cannot adapt to sample diversity:

[0039] Traditional knowledge distillation and self-distillation methods use a unified fixed temperature parameter T, ignoring the significant difficulty differences between different training samples. In the environmental sound recognition task, the complexity of samples varies greatly - the learning difficulty of the model for clear "alarm sound" and mixed background noise "street environment sound" is completely different. The fixed temperature strategy cannot achieve targeted knowledge transfer: for simple samples, too high distillation strength may cause overfitting; while for difficult samples, the fixed temperature setting may not provide enough supervision information.

[0040] (2) Lack of dynamic adaptability in training process:

[0041] The model's understanding of the same sample changes dynamically at different training stages. Existing methods use a fixed temperature throughout the training period, which cannot achieve an adaptive learning process from easy to difficult. In the early stages of training, the model needs smoother soft labels to establish basic feature representation; while in the later stages of training, it needs sharper labels to learn detailed classification boundaries. The fixed temperature strategy cannot meet this dynamic demand.

[0042] (3) Limited efficiency of knowledge transfer:

[0043] Due to the above two limitations, traditional distillation methods cannot maximize the efficiency of knowledge transfer. Fixed temperature is either too conservative, failing to fully exploit the dark knowledge of the teacher model, or too aggressive, causing the student model to learn difficult. This "one-size-fits-all" strategy makes the performance improvement of lightweight models encounter a bottleneck, failing to fully exploit their potential.

[0044] (4) Insensitivity to data distribution characteristics:

[0045] Environmental sound data has obvious long-tail distribution and inter-class similarity characteristics. The fixed temperature strategy cannot be optimized for these characteristics, especially when dealing with minority classes or similar classes.

[0046] In view of the above technical defects, the present application aims to provide a new solution, the specific purposes are as follows:

[0047] (1) Realize sample-level adaptive distillation:

[0048] The primary purpose of the present application is to break through the limitation of fixed temperature and give each training sample an independent and learnable curriculum temperature parameter. By designing a dynamic temperature adjustment mechanism, the model can automatically adjust the distillation strength according to the specific characteristics of the sample, realizing the real "teaching students according to their aptitude".

[0049] (2) Establish an intelligent curriculum learning mechanism:

[0050] The second purpose is to introduce the idea of curriculum learning into the knowledge distillation process, so that the model can automatically identify the difficulty of the sample and develop a corresponding learning plan. Through gradient backpropagation, the temperature parameter is automatically optimized to realize progressive learning from simple samples to difficult samples, simulating the human learning and cognitive process.

[0051] (3) Improve the efficiency and quality of knowledge transfer:

[0052] The present application aims to maximize the efficiency of knowledge transfer through a dynamic temperature mechanism, enabling lightweight student models to more effectively learn the knowledge of teacher models. Especially when dealing with difficult samples and edge cases, by adaptively adjusting the temperature parameter, the accuracy and integrity of knowledge transfer are ensured.

[0053] (4) Maintain low complexity deployment requirements:

[0054] While achieving performance improvement, the present application strictly maintains the computational complexity of the inference stage unchanged. All additional calculations only occur in the training stage, ensuring that the final deployed model still meets the resource constraint requirements of edge devices.

[0055] (5) Provide a universal solution framework:

[0056] The final purpose is to establish a universal adaptive distillation framework that is not only suitable for acoustic scene classification, but also can be extended to other environmental sound recognition tasks such as sound event detection, abnormal sound recognition, etc., providing a new technical path for low complexity audio processing.

[0057] Reference Figure 1 As Figure 1 shown is a low-complexity environmental sound recognition method based on curriculum temperature self-distillation provided by an embodiment of the present application, the method comprising the following steps: S100, a curriculum temperature self-distillation model based on a lightweight backbone network is constructed, a teacher model and a student model are constructed based on the backbone network; S200, a learnable parameter is assigned to each training sample, a curriculum temperature parameter is obtained through reparameterization processing, and the curriculum temperature parameter is used to modulate the softening strength in the knowledge distillation process; S300, construct a total loss function comprising a teacher loss and a student loss, the teacher loss being a cross-entropy loss of the teacher model, and the student loss comprising a task loss, a feature distillation loss and a logical distillation loss of the student model; S400, jointly optimize the backbone network parameters, the student branch parameters and the curriculum temperature parameters based on the total loss function by a gradient descent algorithm; S500, remove all student branches and curriculum temperature parameters after the training is completed, retain the backbone network and the final classifier, and process an input sample through the backbone network and the final classifier to obtain a recognition result.

[0058] In the embodiments provided by the present application, a modular adaptive distillation framework is constructed, which can not only seamlessly adapt to acoustic scene classification tasks, but also be flexibly extended to various environmental sound recognition tasks such as sound event detection and abnormal sound recognition. The framework internally adopts a pluggable component design, including a dynamic temperature generation module, a multi-task loss fusion module and a lightweight feature migration module. The modules communicate with each other through standardized interfaces, so that users can replace or add or delete functional modules according to specific task requirements. For example, when processing a sound event detection task, the classification loss can be replaced by a detection-specific loss function, and the optimization strategy of the curriculum temperature parameter can be adjusted to adapt to the requirements of event boundary positioning; for an abnormal sound recognition task, the temperature modulation weight of a small number of classes can be strengthened to improve the sensitivity of the model to abnormal samples. At the same time, the framework provides a unified data preprocessing interface and model evaluation index, supports audio data input in different formats and multi-dimensional performance evaluation, and provides a standardized and reusable technical path for algorithm research and engineering application in the field of low-complexity audio processing.

[0059] In some embodiments, in S100, the teacher model and the student model are constructed based on the backbone network, comprising: S110, dividing the backbone network into K consecutive modules in depth, and for an input sample, the teacher model is the combination of the output features of the deepest layer, i.e. the Kth module, and the final classifier, wherein the output of the teacher model for the input sample is a teacher logical output; S120, adding a student branch after each of the first K-1 shallow or middle modules of the backbone network, each student branch comprising a student module and a student classifier, the student module comprising an attention module and a feature alignment network, for converting and enhancing the shallow layer features output by the corresponding module to align with the teacher feature dimension, and the student classifier being a fully connected layer for outputting a student logical value, and the kth student model being composed of the student module and the student classifier after the kth module.

[0060] In this embodiment, the hierarchical correspondence between the teacher model and the student model is achieved by modularizing the backbone network in depth. The teacher model combines the output features of the deepest module of the network and the final classifier, which can fully utilize the high-level semantic features extracted by the deep network for decision-making. The student branch added after the shallow or middle module focuses on the key feature area through the attention module and completes the dimension matching with the teacher features through the feature alignment network, so that the student model can learn feature representations at different levels in stages, laying a structured foundation for subsequent knowledge distillation.

[0061] In some embodiments, in S200, the course temperature parameter is obtained through the reparameterization process, including: S210, assigning a learnable parameter to each training sample; S220, processing the learnable parameter through function, and adding a small positive number to obtain the course temperature parameter, the calculation formula is: ; wherein, is a small positive number, function ensures that the output is always positive, is the learnable parameter of the training sample , , is the index of the training sample.

[0062] In this embodiment, the learnable parameter is nonlinearly transformed through function, which ensures that the course temperature parameter is always positive, avoiding unreasonable cases where the temperature is zero or negative. The introduction of the small positive number is to prevent the numerical instability problem caused by the course temperature parameter tending to zero during optimization. It can be set to a small value such as 0.00001. This reparameterization design allows the temperature parameter to be optimized together with the network weights through the gradient descent algorithm, automatically learning the optimal distillation strength for each sample during training, and achieving differentiated processing of samples with different complexities.

[0063] In some embodiments, in S300, the total loss function containing the teacher loss and the student loss is constructed, including: S310, obtaining the cross-entropy loss of the teacher model itself as the teacher loss, which is obtained by calculating the cross-entropy between the predicted output of the teacher model for the training sample and the true label; S320, obtain the task loss, the feature distillation loss and the logical distillation loss of each student branch as the student loss, wherein the task loss is obtained by calculating the cross-entropy of the prediction output of the student model on the training sample and the real label, the feature distillation loss is obtained by calculating the L2 loss between the teacher feature and the student feature converted by the student module, and the logical distillation loss is obtained by calculating the KL divergence between the distribution of the student logical value softened by the course temperature parameter and the distribution of the teacher logical output softened by the course temperature parameter; S330, weight and sum the teacher loss and the student loss of each student branch to obtain the total loss function.

[0064] In this embodiment, the total loss function of multi-task cooperative optimization is constructed by weight and summing the teacher loss and the student loss of each student branch. The teacher loss ensures that the deep network can learn high-quality discriminative features, providing a reliable knowledge source for the student model. The task loss of each student branch ensures that the shallow model can directly learn basic classification ability, and the feature distillation loss and the logical distillation loss respectively realize knowledge transfer from the aspects of feature representation and probability distribution. By dynamically adjusting the weight coefficients of each loss, the model can balance the learning priority of knowledge at different levels during the training process. Especially after introducing the course temperature parameter, the logical distillation loss can automatically adjust the softening intensity according to the sample difficulty, reduce the distillation weight for simple samples to avoid overfitting, and enhance the distillation strength for difficult samples to strengthen knowledge transfer, finally realizing the cooperative evolution of the teacher model and the student model and the efficient transfer of knowledge.

[0065] In some embodiments, in S310, the calculation process of the teacher loss is:

[0066] For each training sample in the training sample set, obtain the component corresponding to the real label position in the teacher logical output output by the teacher model, calculate the logarithm of the ratio of the exponential value of the component to the sum of the exponential values of all components of the teacher logical output, take the average value of the logarithm results of all training samples and take the negative value as the teacher loss.

[0067] In this embodiment, the cross-entropy loss of the teacher model on the training sample is calculated, the teacher logical output is softened by the course temperature parameter, and the KL divergence between the distribution of the softened teacher logical output and the distribution of the softened student logical output is calculated. The function is calculated. This loss function promotes the prediction probability distribution of the teacher model to be as close as possible to the real label distribution, ensuring that the teacher model can learn high-level semantic features with strong discriminability, providing a reliable "knowledge source" for subsequent knowledge distillation. During the back propagation process, the gradient of the teacher loss guides the parameter update of the deepest module of the backbone network, optimizing the extraction ability of high-level features.

[0068] In some embodiments, in S320, the calculation process of the student loss is: S321, For each training sample in the training sample set, obtain the component corresponding to the real label position in the student logic value output by the student model, calculate the logarithm of the ratio of the exponent value of this component to the sum of the exponent values ​​of all components of the student logic value, and take the average of the logarithm results of all training samples and take the opposite number as the task loss. S322, for each training sample in the training sample set, obtain the teacher features output by the teacher model and the student features transformed by the student module, calculate the square of the L2 norm between the teacher features and the student features, and take the average of the squared L2 norm results of all training samples as the feature distillation loss. S323, for each training sample in the training sample set, divide the student's logical value and the teacher's logical output by the course temperature parameter of that sample, and then... The function obtains the student distribution and teacher distribution, calculates the KL divergence between the student distribution and teacher distribution, and takes the average of the KL divergence results of all training samples as the logistic distillation loss. S324: Multiply the task loss, feature distillation loss, and logistic distillation loss by their respective weighting coefficients and then sum them to obtain the student loss for a single student branch.

[0069] In this embodiment, a comprehensive learning objective for a single student branch is formed by weighted combination of task loss, feature distillation loss, and logistic distillation loss. Task loss ensures the student model directly learns the category information of the samples; feature distillation loss prompts the student model to transfer deep feature representations from the teacher model; and logistic distillation loss dynamically adjusts the intensity of knowledge transfer through a course temperature parameter, achieving differentiated supervision for samples of varying difficulty. The weight coefficients can be adjusted according to specific task requirements. For example, in acoustic scene classification tasks, the weight of logistic distillation loss can be appropriately increased to strengthen knowledge transfer at the probability distribution level, while in sound event detection tasks, the proportion of feature distillation loss can be increased to improve feature alignment accuracy. Through this collaborative optimization of multi-dimensional losses, the student model can comprehensively absorb the discriminative knowledge of the teacher model while maintaining its lightweight characteristics and inference efficiency.

[0070] In some embodiments, in S400, the joint optimization process is as follows:

[0071] Initialize the backbone network parameters, student branch parameters, and course temperature parameters. In each training iteration, calculate the total loss function based on the current parameters. Calculate the gradient of the total loss function with respect to the backbone network parameters, student branch parameters, and course temperature parameters using the gradient descent algorithm. Update the backbone network parameters, student branch parameters, and course temperature parameters according to the preset learning rate. Repeat the iterative process until the model training is complete.

[0072] In this embodiment, the gradient descent algorithm is used to jointly optimize the backbone network parameters, the student branch parameters and the curriculum temperature parameters, realizing the co-evolution of multiple parameters. During the training iteration process, the total loss function considers the classification performance of the teacher model, the multi-task learning effect of the student model and the dynamic adjustment requirement of the curriculum temperature parameter. The gradient information is transmitted to the backbone network, the student branch and the temperature parameter through the back propagation mechanism. This joint optimization method enables the temperature parameter to adaptively adjust according to the learning state of the model and the sample characteristics. For example, when the model has low recognition accuracy for a certain type of sample, the temperature parameter will automatically increase to enhance the distillation strength for that type of sample. When the model has fully mastered the simple samples, the temperature parameter will decrease to avoid overfitting. Through continuous iteration and updating, each parameter gradually converges to the optimal state, finally realizing the efficient transfer of teacher knowledge to the student model and the significant improvement of the classification performance of the student model.

[0073] In some embodiments, in S500, the input sample is processed by the backbone network and the final classifier to obtain the recognition result, including: S510, inputting the input sample into the reserved backbone network, extracting features of the input sample through the backbone network to obtain deep features; S520, inputting the deep features into the final classifier, classifying the deep features through the final classifier, and outputting the prediction label corresponding to the input sample, which is the recognition result of the environmental sound.

[0074] Through the function, the probability distribution of the sample belonging to each category is calculated, and the category with the maximum probability value is selected as the prediction label of the input sample, which is the recognition result of the environmental sound. The prediction label calculation formula is: ; wherein, is the prediction label, represents the output feature obtained after the input sample is extracted by the backbone network, is the function, used to select the category with the maximum probability as the final prediction result.

[0075] In this embodiment, the design goal of zero additional inference overhead is achieved by removing all student branches and course temperature parameters after training, leaving only the main network and the final classifier. When a new environmental sound sample is input, the audio data is first converted into a feature format acceptable to the model through a unified data preprocessing interface, and then input to the retained backbone network. As a lightweight architecture, the backbone network can efficiently extract features layer by layer from the input sample, gradually abstracting deep features in the time domain, frequency domain, and semantic level from the original audio waveform. These deep features are input to the final classifier, which maps the features to the preset environmental sound category space through fully connected layers, and calculates the probability distribution of the sample belonging to each category through function. Finally, the class with the maximum probability value is selected as the predicted label of the input sample, completing the environmental sound recognition process. This end-to-end inference process not only maintains the low complexity deployment requirement, but also ensures the accuracy and real-time performance of the recognition result through the optimization design of the backbone network, which can meet the application requirements of mobile devices, embedded systems and other resource-constrained scenarios.

[0076] The following are specific embodiments provided by the present application:

[0077] The present application proposes a Curriculum Temperature based Self-Distillation (CTSD) method for low-complexity environmental sound recognition (ESR).

[0078] Network architecture definition and forward propagation:

[0079] The overall architecture of the curriculum temperature self-distillation model is based on a lightweight backbone network (such as MobileNetV2). The core innovation lies in the introduction of a curriculum temperature module for dynamically adjusting the knowledge distillation process. This architecture divides the network into multiple modules and integrates teacher models, student models, and curriculum temperature mechanisms.

[0080] Backbone Network: A lightweight convolutional neural network (such as MobileNetV2) is selected as the backbone model where is the model parameter, represents the parameter set of the backbone network, including the convolution kernel parameters, batch normalization parameters, and other learnable variables of each module. represents the feature extraction process of the backbone network on the input sample, i.e., through convolution, activation, pooling, and other operations of each module to convert the original input into a multi-level feature representation.

[0081] The backbone network is divided into K consecutive modules (Sections) in depth, for example, when K=4, the backbone network is divided into , , , four parts, namely .

[0082] Teacher Model: defined as the combination of the modules (all K modules) that extract the deepest layer output features of the backbone network and the final classifier. Specifically, for an input sample , the teacher feature is , which is processed by the final classifier (typically a fully connected layer plus ) to produce the teacher logical output . Therefore, the teacher model is structurally represented as ; is the backbone network, represents the composite operation of functions, is any input sample, is the deepest layer output feature, i.e., the teacher feature, is the final classifier, is the teacher logical output of the final classifier output.

[0083] Student Model: add student branches after the first K-1 shallow or middle modules of the backbone network. Each student branch is a complete subnetwork, including: Student Module : usually composed of a light attention module and a feature alignment network (e.g., 1x1 convolution), used to convert and enhance the shallow layer features to align with the teacher feature dimension. Where , represents the feature extraction process of the backbone network from the first module to the kth module, k represents the index of the student module.

[0084] Student Classifier : an independent classifier (fully connected layer) for outputting student logical values .

[0085] where, represents the kth student branch, k represents the index of the student branch, represents the student module in the kth student branch for converting and enhancing shallow layer features, represents the student classifier in the kth student branch, represents the first k modules of the backbone network The input sample is denoted as The shallow or middle layer features output by the feature extraction. The feature output by the kth student branch after converting and enhancing the shallow or middle layer features output by the kth module of the backbone network, which is aligned with the teacher feature dimension for subsequent calculation of the feature distillation loss. The student logical value output by the kth student branch after converting and enhancing the shallow or middle layer features output by the kth module of the backbone network. The student logical value output by the kth student branch after converting and enhancing the shallow or middle layer features output by the kth module of the backbone network.

[0086] Therefore, the kth student model is defined as The student logical value is obtained by first classifying the features converted and enhanced by the student module in the kth student branch using the student classifier, and then performing knowledge distillation on the student logical value and the teacher logical output to achieve knowledge transfer from the teacher model to the student model. Through the cascade structure, the student model can fully utilize the feature information of the shallow or middle layer modules and learn the deep knowledge of the teacher model, thereby improving the recognition performance of the overall model.

[0087] The overall architecture is shown in Figure 2 , which shows the integration of data flow, knowledge flow and curriculum temperature module:

[0088] The mechanism process of curriculum temperature is shown in Figure 3 :

[0089] Curriculum temperature is the core innovation of the present application, which dynamically allocates an adaptive temperature parameter to each training sample, replacing the traditional fixed temperature, and realizes intelligent knowledge transfer.

[0090] Parameterization and reparameterization: a learnable parameter is assigned to each training sample . The reparameterization technique ensures that the temperature is always positive: ; where is a very small positive number (such as 0.000001) to prevent numerical instability. The function ensures that the output is always positive, is the learnable parameter of the training sample , , and i is the index of the training sample.

[0091] Function implementation: curriculum temperature is used to modulate the softening strength in the knowledge distillation process. For simple samples, it is automatically reduced, making the teacher label more sharp and increasing the learning strength; for difficult samples, Increase, make teacher label smoother, reduce learning intensity. This simulates the human "from easy to difficult" learning process.

[0092] Course temperature calculation formula, is the most core learnable parameter. It assigns an unconstrained scalar value to each training sample. It can be described in the patent document as "a first parameter associated with the sample". is an activation function that maps to the positive value space. Its expression is . This is to ensure the smoothness and differentiability of the output, facilitating gradient optimization. is a very small positive value, which is a safety design in engineering, aiming to prevent the temperature from being too small or zero at the beginning of training, leading to numerical instability (such as division by zero error). This reflects the robustness and practicality of the scheme.

[0093] Loss function construction:

[0094] The loss function combines teacher model supervision and student model distillation learning, ensuring that the model learns both real labels and teacher knowledge during training.

[0095] Total loss function: ; where, , , , , is a hyperparameter used to balance the loss weight of each item.

[0096] Teacher loss : The cross-entropy loss of the teacher model itself, ensuring its prediction accuracy: ; where M is the total number of categories, is the true label of the sample , N is the total number of training samples, indicates the component of the corresponding true label in the teacher logic output , indicates the jth component of the teacher logic output .

[0097] Student loss: The loss of each student branch consists of three parts: Task loss : The cross-entropy loss of the kth student model, ensuring its basic classification ability: ; where, denotes the k-th student's logical output , denotes the k-th student's logical output 's j-th component.

[0098] feature distillation loss : constrain the similarity between student and teacher features using L2 loss: ; where, is the teacher feature output by the teacher model on the training sample , is the student feature in the k-th student branch after being transformed and enhanced by the student module, denotes the L2 norm.

[0099] logical distillation loss (key part): let the student output distribution approximate the teacher distribution softened by curriculum temperature using KL divergence: ; where, is function, is the KL divergence.

[0100] the symbol in the formula is the standard operator of KL divergence, read as "KL divergence of the prior distribution with respect to the posterior distribution", the prior is the reference distribution, and the posterior is the approximate distribution.

[0101] joint optimization and curriculum mechanism: optimize all parameters, including model parameters and curriculum temperature parameters , by gradient descent algorithm. The optimization process is as follows: ; where, is the learning rate, , the curriculum temperature parameters of the t-th and t+1-th iterations, respectively, denotes the total loss function on the model parameters and curriculum temperature parameters at the t-th iteration.

[0102] The gradient of the curriculum temperature reveals the adaptive mechanism: ; where, ​is the weighted average of student logits. When the student model is not confident enough on the sample ( ), the gradient is negative, and the curriculum temperature increases, reducing the learning intensity; otherwise, the temperature decreases, increasing the learning intensity. This achieves fully automatic curriculum learning without manual intervention. represents a positive correlation relationship and is used to intuitively express the association between the gradient and the adjustment direction of the temperature. represents the partial derivative of the logical distillation loss with respect to the curriculum temperature parameter , which reflects how the temperature parameter is dynamically adjusted according to the learning state of the student model. When the prediction of the student model on the sample is far from the true label, i.e., the true label component of the student logits is less than the corresponding weighted average , the partial derivative is negative, and at this time, the curriculum temperature parameter will increase through gradient descent optimization, so that the distribution of the teacher label is smoother, reducing the difficulty of the student model in learning the difficult sample; when the prediction of the student model on the sample is more accurate, i.e. is greater than , the partial derivative is positive, and the curriculum temperature parameter will decrease, making the teacher label distribution more sharp, enhancing the learning intensity of the student model on the simple sample, so as to realize the adaptive curriculum learning process from easy to difficult, effectively improving the learning efficiency and recognition performance of the model. This dynamic adjustment mechanism is completely based on data-driven, without the need for manual setting of temperature scheduling strategy, greatly enhancing the adaptability and generalization ability of the model on different types of environmental sound samples.

[0103] Inference deployment:

[0104] After training, all student branches and curriculum temperature parameters are removed, and only the original backbone network and the final classifier are retained for inference. Therefore, the present application does not introduce any additional inference overhead, which meets the low complexity requirement. The inference process is: ; wherein, is the predicted label, represents the output feature obtained after the backbone network extracts features from the input sample , is the function, used to select the class with the highest probability as the final prediction result.

[0105] ​The application realizes adaptive knowledge distillation by introducing a curriculum temperature mechanism, effectively solving the limitations of fixed temperature strategies. The technical solution includes detailed network architecture, loss function and optimization process, ensuring that the model intelligently adjusts the learning intensity during the training phase and maintains low complexity during the inference phase, providing a reliable solution for low-complexity environmental sound recognition.

[0106] Experimental verification:

[0107] The following table compares the key features of the present application with the closest prior art.

[0108]

[0109] This comparison shows that the present application has realized a fundamental change from "fixed" to "adaptive" in the core mechanism, occupying a high position in technological innovation.

[0110] Compared with the fixed temperature-based self-distillation method in the related art, the curriculum temperature-based self-distillation (CTSD) method proposed by the present application has the following significant advantages:

[0111] 1. Curriculum temperature mechanism:

[0112] The present application first introduces the concept of "curriculum temperature", which changes the temperature parameter in knowledge distillation from a fixed hyperparameter to a learnable, sample-level adaptive parameter. Each training sample is assigned an independent curriculum temperature value , which is ensured to be positive through the reparameterization technique, thereby dynamically adjusting the intensity of knowledge distillation.

[0113] 2. Joint optimization framework:

[0114] A joint optimization strategy is proposed, which synchronously trains the model parameters θ and all curriculum temperature parameters through the gradient descent algorithm. This optimization method enables the model to automatically adjust the temperature according to the difficulty of the sample.

[0115] without manual intervention or pre-defined curriculum plan.

[0116] 3. Adaptive curriculum learning:

[0117] Through the gradient signal of the curriculum temperature, a fully automatic "easy to difficult" learning process is realized. When the student model has insufficient confidence in the sample, the temperature automatically increases, reducing the learning intensity; otherwise, the temperature decreases, increasing the learning intensity. This simulates the human cognitive process, improving the training efficiency and model performance.

[0118] 4. Multi-task loss integration:

[0119] A comprehensive loss function is designed, which combines the teacher loss, student task loss, feature distillation loss and KL divergence loss based on curriculum temperature. This design ensures that the model maintains the basic classification ability while efficiently transferring the teacher knowledge and modulating the distillation process through curriculum temperature.

[0120] 5. Zero additional inference overhead design:

[0121] All innovative modules (such as student branches, curriculum temperature parameters) are only used in the training stage and are completely removed during inference. The final deployed model only retains the original backbone network, ensuring no additional computation or parameter increase, which meets the low complexity application requirements.

[0122] 6. Universality and scalability:

[0123] The method of the present application is a universal framework suitable for various environmental sound recognition tasks (such as acoustic scene classification, sound event detection), and can be extended to classification problems of other modalities (such as images, texts), which embodies wide applicability.

[0124] The embodiment of the present application also provides a low-complexity environmental sound recognition system based on curriculum temperature self-distillation, comprising: at least one processor; at least one memory for storing at least one program; When the at least one program is executed by the at least one processor, the at least one processor implements the above method.

[0125] The contents in the above method embodiments are applicable to the present embodiment, the functions specifically implemented by the present embodiment are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those of the above method embodiments, which will not be repeated here.

[0126] The embodiment of the present application also provides an electronic device, which comprises a memory and a processor, the memory stores a computer program, and the processor implements the above method when executing the computer program. The electronic device can be any intelligent terminal including a tablet computer, a vehicle-mounted computer, etc.

[0127] It can be understood that the contents in the above method embodiments are applicable to the present device embodiment, the functions specifically implemented by the present device embodiment are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those of the above method embodiments.

[0128] The embodiment of the present application also provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the above method.

[0129] It can be understood that the contents in the above method embodiments are all applicable to the present storage medium embodiments, the present storage medium embodiments specifically implement the same functions as the above method embodiments, and achieve the same beneficial effects as the above method embodiments.

[0130] The present application also provides a computer program product, which comprises computer programs or computer instructions stored in a memory, and a processor of a computer device reads the computer programs or computer instructions from the memory, and the processor executes the computer programs or computer instructions, so that the computer device executes the above method.

[0131] It can be understood that the contents in the above method embodiments are all applicable to the present program product embodiments, the present program product embodiments specifically implement the same functions as the above method embodiments, and achieve the same beneficial effects as the above method embodiments.

[0132] The memory, as a non-transitory computer readable storage medium, can be used to store non-transitory software programs and non-transitory computer executable programs. In addition, the memory can include a high-speed random access memory, and can also include a non-transitory memory, such as at least one magnetic disk storage device, a flash memory device, or other non-transitory solid-state memory device. In some embodiments, the memory can optionally include a memory remotely arranged relative to the processor, and these remote memories can be connected to the processor through a network. Examples of the above network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof.

[0133] As will be appreciated by one of ordinary skill in the art, all or some steps, systems of the above-disclosed methods can be implemented as software, firmware, hardware, or suitable combinations thereof. Some or all of the physical components can be implemented as software executed by a processor, such as a central processing unit, a digital signal processor, or a microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit. Such software can be distributed on computer readable media, which can comprise computer storage media (or non-transitory media) and communication media (or transitory media). As is well known to those of ordinary skill in the art, the term computer storage media includes both volatile and non-volatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Computer storage media include, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can be accessed by a computer. Further, as is well known to those of ordinary skill in the art, communication media typically embodies computer readable instructions, data structures, program modules, or other data in a modulated data signal, such as a carrier wave or other transport mechanism, and includes any information delivery media.

[0134] The apparatus embodiments described above are merely illustrative for separate units described as separate components can or can not be physically separate, i.e. can be located in one place or can be distributed over a plurality of network units. Part or all of the modules can be selected according to actual needs to achieve the purposes of the embodiments.

Claims

1. A low-complexity environmental sound recognition method based on course temperature self-distillation, characterized in that, The method includes the following steps: S100, Construct a course temperature self-distillation model based on a lightweight backbone network, and construct teacher and student models based on the backbone network; S200, assign learnable parameters to each training sample, and obtain course temperature parameters through reparameterization, which are used to modulate the softening intensity in the knowledge distillation process; S300, Construct a total loss function that includes teacher loss and student loss. The teacher loss is the cross-entropy loss of the teacher model, and the student loss includes the task loss, feature distillation loss, and logistic distillation loss of the student model. S400, based on the total loss function, uses gradient descent algorithm to jointly optimize backbone network parameters, student branch parameters and course temperature parameters; After training, the S500 removes all student branches and course temperature parameters, retains the backbone network and the final classifier, and processes the input samples through the backbone network and the final classifier to obtain the recognition results.

2. The method according to claim 1, characterized in that, In S100, the construction of the teacher model and student model based on the backbone network includes: S110 divides the backbone network into K consecutive modules in depth. For the input sample, the teacher model is the combination of the output features of the deepest layer of the network, i.e. the Kth module, and the final classifier. The output of the teacher model to the input sample is the teacher logic output. S120 adds student branches after the first K-1 shallow or mid-layer modules of the backbone network. Each student branch contains a student module and a student classifier. The student module consists of an attention module and a feature alignment network, which is used to transform and enhance the shallow features output by the corresponding module to align with the teacher feature dimension. The student classifier is a fully connected layer used to output the student's logical value. The k-th student model consists of the student module and student classifier after the k-th module.

3. The method according to claim 1, characterized in that, In S200, obtaining the course temperature parameters through reparameterization includes: S210, assign learnable parameters to each training sample; S220, via The function processes the learnable parameters and adds a minimum positive constant to obtain the course temperature parameter. The calculation formula is as follows: ;in, It is a very small positive number. The function ensures that the output is always a positive number. For training samples Learnable parameters, , This is the index of the training samples.

4. The method according to claim 1, characterized in that, In S300, the construction of the total loss function, which includes teacher loss and student loss, includes: S310, Obtain the cross-entropy loss of the teacher model itself as the teacher loss, wherein the teacher loss is obtained by calculating the cross-entropy between the teacher model's predicted output of the training samples and the true labels; S320, obtain the task loss, feature distillation loss and logistic distillation loss of each student branch as the student loss. The task loss is obtained by calculating the cross-entropy between the student model's predicted output of the training samples and the true label. The feature distillation loss is obtained by calculating the L2 loss between the teacher features and the student features after transformation by the student module. The logistic distillation loss is obtained by calculating the KL divergence between the distribution of student logistic values ​​softened by the course temperature parameter and the distribution of teacher logistic output softened by the course temperature parameter. S330 calculates the total loss function by weighting and summing the teacher loss with the student loss for each student branch.

5. The method according to claim 4, characterized in that, In S310, the calculation process for the teacher's loss is as follows: For each training sample in the training sample set, obtain the component corresponding to the real label position in the teacher logic output of the teacher model, calculate the logarithm of the ratio of the exponent value of this component to the sum of the exponent values ​​of all components of the teacher logic output, and take the average of the logarithm results of all training samples and take the opposite number as the teacher loss.

6. The method according to claim 4, characterized in that, In S320, the calculation process for the student's loss is as follows: S321, For each training sample in the training sample set, obtain the component corresponding to the real label position in the student logic value output by the student model, calculate the logarithm of the ratio of the exponent value of this component to the sum of the exponent values ​​of all components of the student logic value, and take the average of the logarithm results of all training samples and take the opposite number as the task loss. S322, for each training sample in the training sample set, obtain the teacher features output by the teacher model and the student features transformed by the student module, calculate the square of the L2 norm between the teacher features and the student features, and take the average of the squared L2 norm results of all training samples as the feature distillation loss. S323, for each training sample in the training sample set, divide the student's logical value and the teacher's logical output by the course temperature parameter of that sample, and then... The function obtains the student distribution and teacher distribution, calculates the KL divergence between the student distribution and teacher distribution, and takes the average of the KL divergence results of all training samples as the logistic distillation loss. S324: Multiply the task loss, feature distillation loss, and logistic distillation loss by their respective weighting coefficients and then sum them to obtain the student loss for a single student branch.

7. The method according to claim 1, characterized in that, In S400, the joint optimization process is as follows: Initialize the backbone network parameters, student branch parameters, and course temperature parameters. In each training iteration, calculate the total loss function based on the current parameters. Calculate the gradient of the total loss function with respect to the backbone network parameters, student branch parameters, and course temperature parameters using the gradient descent algorithm. Update the backbone network parameters, student branch parameters, and course temperature parameters according to the preset learning rate. Repeat the iterative process until the model training is complete.

8. The method according to claim 1, characterized in that, In S500, the process of processing the input samples through the backbone network and the final classifier to obtain the recognition result includes: S510: Input samples are fed into the preserved backbone network, and features are extracted from the input samples through the backbone network to obtain deep features; S520, the deep features are input into the final classifier, the deep features are classified by the final classifier, and the predicted label corresponding to the input sample is output. The predicted label is the recognition result of the environmental sound. pass The function calculates the probability distribution of a sample belonging to each category, and selects the category with the highest probability value as the predicted label for the input sample. This predicted label is the recognition result of the environmental sound. The formula for calculating the predicted label is: ;in, It is a predicted label. This indicates that the backbone network responds to the input samples. The output features obtained after feature extraction yes function, The category with the highest probability is selected as the final prediction result.

9. A low-complexity environmental sound recognition system based on course temperature self-distillation, characterized in that, include: At least one processor; At least one memory for storing at least one program; When the at least one program is executed by the at least one processor, the at least one processor performs the method as described in any one of claims 1 to 8.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 8.

Images