Dark light scene target detection method

Abstract

The invention provides a dark light scene target detection method. The method comprises the following steps: firstly, acquiring target images under different illumination conditions, and performing image data enhancement through gamma change; secondly, pre-training a teacher model by using the enhanced large-scale data set to learn target detection general knowledge; thirdly, constructing distillation loss based on Zensen-Shannon divergence, measuring the prediction result distribution difference of the pre-trained teacher model and the trained middle school student model, and improving the training stability; and finally, designing a self-adaptive label distribution method to determine an image distillation area, realizing dark light scene knowledge distillation by adopting the proposed symmetric loss, and improving the efficiency of knowledge distillation from a large model to a small model. The method can be applied to target detection of dark light scenes of different degrees.

Description

Technical Field

[0001] This invention relates to the field of target detection technology, specifically to a target detection method for low-light scenes. Background Technology

[0002] Object detection, a fundamental computer vision task, has wide applications in various fields, including autonomous perception for unmanned aerial vehicles (UAVs). In recent years, deep learning-based object detection algorithms have achieved excellent target recognition and localization performance under ideal lighting conditions. However, the accuracy of existing deep learning-based object detectors drops significantly in low-light scenarios. The core reason for this significant performance decline is that the powerful capabilities of existing deep learning object detectors are highly dependent on the quality of the input image. Low light results in low overall image brightness, severely reduced contrast, blurred boundaries between objects and backgrounds, and the loss or weakening of crucial edge, texture, and detail features. The core of deep neural networks is learning a non-linear mapping from pixels to semantic features; the lack of this information directly makes it difficult for the network to extract sufficiently effective and discriminative features for target recognition and localization. Furthermore, noise is significantly amplified in low light conditions. In insufficient light, camera sensors increase their sensitivity to obtain sufficient brightness, inevitably introducing a large amount of shot noise. This noise manifests as randomly distributed particles or artifacts in the image. Deep neural networks are trained using data acquired under ideal lighting conditions. They tend to mistake strong, unstructured noise as part of the image content, severely interfering with the model's understanding of the true object structure and leading to numerous false positives and false negatives. Low light also distorts color information in images; in extremely low light, color information becomes highly unreliable. Differences in camera sensor sensitivity to different color channels are amplified under low signal-to-noise ratios, resulting in severe color distortion and shifts. Many object detection algorithms utilize color as an important auxiliary cue (e.g., to distinguish different object categories); the loss or distortion of color information further weakens the model's discriminative ability. Therefore, the inherent information attenuation, noise interference, and limitations of models and data in low-light scenes collectively contribute to the performance degradation of existing deep learning object detectors under low-light conditions.

[0003] To address these issues, existing methods for mitigating the performance degradation of object detection under low-light conditions can be broadly categorized into two types: image restoration or enhancement-based methods and domain-adaptive methods. The former uses image restoration or enhancement algorithms as a preprocessing module to recover details such as color and texture of the target in the image. However, image restoration often leads to local overexposure or noise amplification. The latter attempts to reduce the distributional differences between the source domain (ideal weather) and the target domain (adverse weather) through adversarial training and feature alignment. However, due to the limited size and lack of annotation in the target domain data, these methods often overfit the training dataset, resulting in poor generalization ability of the object detector in real-world scenarios.

[0004] Knowledge distillation methods guide the learning process of small models by using a large model pre-trained on a large dataset. Although the large model is not specifically trained for low light conditions, the large image dataset covers various image degradation modes, allowing the student model to inherit relevant knowledge from the intermediate features or final logic layer of the teacher model. This effectively improves the generalization ability of the small model across various visual tasks. Therefore, knowledge distillation based on a large model under low-light conditions is another possible approach to improve the generalization ability of object detection.

[0005] Two key issues remain unresolved in object detection methods based on large-model knowledge distillation: 1) How to distill reliable localization information from a pre-trained large model instead of low-confidence noisy data? Existing methods propose several loss functions for distillation based on different bounding box representations. Chen et al. used squared difference loss to calculate the difference between the predicted boxes of the teacher and student models. Although squared difference loss measures the difference in the absolute position of the predicted boxes, it cannot represent the uncertainty of the bounding box prediction. Zheng et al. switched the edges of each predicted box from absolute position representation to probability distribution representation and calculated the relative entropy between the teacher and student models. However, relative entropy is sensitive to low-probability regions of a distribution, which can cause the student model to learn the low-confidence predicted box distribution in the teacher model. 2) How to selectively distill key regions instead of distilling all image regions? Li et al. performed knowledge distillation from the detection box regions predicted by the region proposal network. However, the student network may inherit the incorrect prediction results of the region proposal network. Nguyen et al. used label assignment to select distillation regions, which guarantees localization accuracy through ground truth regions, but the sparsity of ground truth regions prevents the student model from efficiently learning localization knowledge. Summary of the Invention

[0006] To address the limitations of existing object detection methods based on large-model knowledge distillation in achieving high-reliability distillation and key region distillation from the teacher model, we propose two components to enhance the localization capabilities of the student model. First, we propose a distillation loss based on Jensen-Shannon divergence to handle the low-confidence output of the teacher model. By introducing an intermediate distribution of the large-small model's predicted output, we mitigate the overfitting of the student model to the noise in the teacher model's output and improve the robustness of the training process. Second, we propose a dynamic threshold adjustment strategy to select distillation regions. By adjusting the constraints of the bounding box ground truth values ​​to appropriate neighboring regions, we expand the positive sample allocation and address the issue of sparse positive sample allocation.

[0007] This invention provides a low-light target detection method based on large model knowledge distillation. The method comprises two stages: adaptive label assignment of positive sample regions of the target detection box and bidirectional localization distillation of the teacher model and student model outputs under low-light image input. The former determines which regions to distill, while the latter determines the distillation method. Specifically, it includes the following steps:

[0008] S1. Collect images of the target to be detected under low light conditions, label the target category and location frame by frame to obtain the dataset, and divide the dataset into training set and test set according to an 8:2 ratio.

[0009] S2. Input the training set data in batches into the ResNet101 teacher model pre-trained on the COCO dataset for feature extraction, combine it with the FPN network to obtain multi-scale features, and predict the probability distribution of the detection box in the multi-scale feature layer.

[0010] S3. Since GFocal is used as the baseline network for object detection, this network outputs the probability distribution of each detection box position. First, the probability distribution of the predicted box is converted into the absolute position of the boundary. One predicted box corresponds to four edge probability distributions. For each predicted edge e, its absolute position is... It can be given by the following formula: Where x is a [e min ,e max For direct continuous variables, Pr(x) is the probability density function of x, where e min =0,e max =1. By regressing the interval [e] of each edge min ,e max Discretize into a uniform distribution of length n [e1,e2,...,e] n In our experiment, the length of n was confirmed to be 16, and the above formula can be written as: Among them, Pr(ei The features are predicted by the teacher model. For the l-th layer of the feature pyramid, a pre-set set of truth boxes is used. and anchor frame set Adaptive label assignment is performed, meaning that for each anchor box, it is determined whether it is a background region or a target region based on the ground truth region. First, the GIoU score set X for each anchor box and ground truth box is calculated. l : Where, x ij Indicates anchor frame With truth value The GIOU score. For each true value The GIOU scores are ranked, and the anchor boxes corresponding to the top k scores are taken as candidate boxes. The mean μ and standard deviation σ of the GIOU of these k anchor boxes and the ground truth box are calculated. An adaptive scaling factor γ is designed to adjust the positive sample allocation threshold α. α=γ(μ+σ) Where γ∈[0-1], μ and σ represent the mean and standard deviation of the confidence scores of the k candidate boxes, respectively. Then, positive sample regions are selected based on α: V l ={v ij |x ij >α} Among them, V l It represents a set of truth boxes. and anchor frame set The matching binary mask matrix, v ij ∈{0,1} represents the anchor frame With truth value Whether it matches or not, v ij =1 represents the anchor frame Positive samples are assigned as positive samples in the target detection loss calculation. By dynamically adjusting the parameter γ, the sparsity problem of positive sample allocation during knowledge distillation and the difficulty of selecting distillation regions can be effectively solved.

[0011] S4. After label assignment, positive samples participate in the calculation of the object detection loss. However, existing methods use KL divergence to force a small model to learn from a large model, which may lead to overfitting to the noise of the large model. Furthermore, the asymmetry, steep gradient, and unboundedness of KL divergence cause oscillations during training. To address these issues, a bidirectional distillation loss is proposed to solve the distillation problem. Assuming z is the logistic value of an edge of the predicted bounding box, represented by n discrete probabilities (we take n=16), the distributions of the teacher model and student model for this edge can be represented as z. T and z S The distribution of these logical values ​​can be rewritten as a probability distribution using the following Softmax formula: Where i and j are index numbers (unlike the previous formula, e in this formula represents...). τ is a temperature parameter; a larger τ indicates smaller differences between elements in the probability distribution p. After applying the Softmax formula, z... T and z S Transformed into probability distribution p T and p S For the predicted probability distribution of each edge e, in order to measure the difference between the two probability distributions of the teacher model prediction and the student model prediction, a bidirectional distillation loss based on the Jensen-Shannon divergence is introduced for both the student and teacher models. in, p T and p M Asymmetric similarity: After calculating the similarity of an edge distribution, the big-small model distillation loss of a detection box is given by the following formula: The reasons for using JS divergence as the localization loss are: 1) bounded distribution distance, 2) symmetry between the student and teacher models, and 3) gradient smoothness during training. Theoretically, the difference between KL and JS divergence during gradient student model updates can be expressed as: The JS divergence is due to the intermediate distribution. Maintaining boundedness, while the KL divergence is in p s The gradient increases rapidly as it approaches zero, which can lead to numerical instability in the early stages of distillation in large models.

[0012] The advantages of this method are that the training process is stable and it can accurately detect targets with a small number of model parameters under low light conditions. It includes multiple categories such as people, cars, bicycles, motorcycles and buses. The detection accuracy mAP50≥60% is achieved on the dense fog weather dataset RTTS and the detection accuracy mAP50≥50% is achieved on the low light weather dataset ExDark. Attached Figure Description

[0013] The accompanying drawings, which form part of this application, are used to provide a further understanding of this application. The illustrative embodiments of this application and their descriptions are used to explain this application and do not constitute an undue limitation of this application.

[0014] Figure 1 This is a diagram of the two-way distillation loss method based on JS divergence of the present invention;

[0015] Figure 2 This is a diagram of the adaptive tag allocation method based on a gating mechanism according to the present invention;

[0016] Figure 3 This is a diagram showing the implementation effect of the present invention on the RTTS public dataset;

[0017] Figure 4 This is a diagram showing the implementation effect of the present invention on the ExDark public dataset; Detailed Implementation

[0018] To better understand the technical solution of the present invention, the embodiments of the present invention will be further described below in conjunction with the accompanying drawings and specific examples. Note that the aspects described below in conjunction with the accompanying drawings and specific embodiments are merely exemplary and should not be construed as limiting the scope of protection of the present invention in any way.

[0019] A flowchart of a low-light target detection method based on large model knowledge distillation is shown below. Figure 1 As shown, the label allocation method in this method is as follows: Figure 2 As shown, the overall process consists of four steps: data collection and labeling, large-scale model pre-training, determining the distillation region, and small-scale model training. Figure 1 The distillation method is shown. Figure 2 The distillation region is revealed, as follows:

[0020] Step S1: First, according to the task requirements, use an RGB camera to acquire images under different lighting conditions (low light, exposure, etc.) and weather conditions (fog, blur, etc.), and label the targets to be detected in the images according to the COCO dataset format. Two public datasets, RTTS and PASCAL VOC, are used as the training and testing sets. The RTTS dataset contains 4322 labeled images, covering different lighting scenes (fog, low light, etc.), and includes five categories: people, bicycles, trucks, buses, and motorcycles. This dataset is used as the testing set. PASCAL VOC includes two versions, 2007 and 2012. Images of the same category as those in the RTTS dataset are sampled from these two datasets for training. To improve the robustness of the model, two methods, synthetic fog and Gamma transformation, are used to augment the images. The mathematical formula for synthetic fog is: I(x)=J(x)t(x)+A(1-t(x)) Where I(x) and J(x) represent the fogged image and its corresponding defogging image, respectively, A represents global atmospheric light, t(x) represents the projection map, and x represents each pixel in the image. The transmittance image is obtained by the following formula: t(x)=e -βd(x) Where d(x) represents the depth of field, and β represents the atmospheric attenuation factor, which is empirically set to [0, 1]. To cope with different lighting conditions, Gamma transformation is used to adjust the image brightness: f(x) = x γ Where f(x) represents the image after brightness adjustment, and γ represents the scaling factor. The value of γ is selected based on empirical values ​​[1.5, 5.0].

[0021] Step S2: Pre-train the teacher model on the large-scale common category object detection dataset COCO The parameter update process of the ResNet101 teacher model is represented as follows: Where, θ T Here, represents the parameters of the teacher model, t represents the number of gradient updates, and β0 represents the training learning rate of the teacher model. L(θ) represents the gradient of the teacher model. T ) represents the total loss during teacher model training.

[0022] Step S3: After obtaining enhanced images with varying lighting and weather conditions in Step S1, and obtaining a pre-trained large model in Step S2, a knowledge distillation method based on logical simulation is used to transfer the general knowledge learned by the large model ResNet101 to the lightweight model ResNet18 for better deployment. Given a batch of input images, the ResNet101 teacher model and the ResNet18 student model perform feature extraction respectively, combine the FPN network to obtain multi-scale features, and predict the probability distribution of detection boxes in the multi-scale feature layer. Since GFocal is used as the baseline network for object detection, this method uses ResNet as the backbone network. The output form of the predicted box is a 4×n probability distribution, where n represents the granularity of dividing the coordinate box interval [0, 1] equally. The larger n is, the smaller the sub-interval, and the greater the computational complexity of the model. Therefore, balancing model complexity and computational accuracy, n is chosen to be 16. By regressing the regression interval [e] of each edge... min ,e max Discretize into a uniform distribution of length n [e1,e2,...,e] n The above formula can be written as: Among them, Pr(e i ) is to obtain each edge e i The probability density function is obtained from the teacher model prediction, where i represents the index number of each variable. Assuming z is the logistic value of an edge of the prediction box, represented by n discrete probabilities, the distributions of the teacher and student models for this edge can be expressed as z = ... T and z SThe distribution of these logical values ​​can be rewritten as a probability distribution using the following Softmax formula: Where i and j are index numbers (unlike the previous examples, e in this formula represents the natural constant). z i and z j Let and represent the i-th and j-th probability values ​​of a probability distribution, respectively; both are scalars. And z... T and z S This represents the probability output of the teacher and student models, and is a vector. τ is a temperature parameter; a larger τ indicates smaller differences between the elements of the probability distribution p. The value of τ was obtained through multiple rounds of ablation experiments, and ultimately, a τ value of 10 resulted in the best detection performance for dark-light targets. After applying the Softmax formula, z... T and z S Transformed into probability distribution p T and p S For the l-th level of the feature pyramid, a pre-defined set of truth boxes is used. and anchor frame set For each anchor box, determine whether it is a background region or a target region based on the ground truth region; first, calculate the GIoU score set X for each anchor box and the ground truth box. l : Where, x ij Indicates anchor frame With truth value The GIOU score can be calculated using the following formula: Where A and B represent two detection boxes, and C represents the smallest rectangle encompassing A and B. For each true value... The GIOU scores are ranked, and the anchor boxes corresponding to the top k scores are taken as candidate boxes. The mean μ and standard deviation σ of the GIOU of these k anchor boxes and the ground truth box are calculated. An adaptive scaling factor γ is designed to adjust the positive sample allocation threshold α. α=γ(μ+σ) Where γ∈[0-1], μ and σ represent the mean and standard deviation of the confidence scores of the k candidate boxes, respectively. Then, positive sample regions are selected based on α: V l ={v ij |x ij >α} Among them, V l It represents a set of truth boxes. and anchor frame set The matching binary mask matrix, v ij ∈{0,1} represents the anchor frame With truth value Whether it matches or not, v ij =1 represents the anchor frame It is assigned as a positive sample to participate in the target detection loss calculation.

[0023] Step S4: After determining the distillation region in Step S3, the distillation loss is then selected, i.e., determining how to use the large model to guide the small model in learning relevant domain knowledge. Traditional methods use KL divergence to measure the difference between the outputs of the large and small models. Where, p T (e i ) and p s (e i ) represent the predictions of the teacher model and the student model for a certain edge in e, respectively. i The probability at point p s =Softmax(z s ),p T =Softmax(z T However, KL divergence forces the student model to strictly learn the probability distribution of the teacher model, making it susceptible to noise from the teacher model. Therefore, JS divergence is used instead of KL divergence to supervise the student model's learning. in, p T and p M Asymmetric similarity: After calculating the similarity of an edge distribution, the big-small model distillation loss L of a detection box is... JS Given by the following formula: Among them, B T and B S Let B represent the predicted bounding boxes of the teacher model and the student model, respectively, and let e represent an edge of the bounding box. The parameter update process of the student model is represented as follows: Where, θ s Here, represents the parameters of the student model, t represents the number of gradient updates, and β1 represents the training learning rate of the student model. L(θ) represents the gradient of the student model. T ) represents the total loss during student model training.

[0024] like Figure 3As shown, the adaptability of the proposed method under different climate and lighting conditions was tested on the RTTS dataset. During the testing phase, no data augmentation was used; the images were directly input into the student model for testing. The test results show the detection performance of the proposed method under different concentrations of fog and lighting conditions.

[0025] The advantage of this method is that by distilling the knowledge of the teacher model into a lightweight student model, it can achieve accurate detection results under varying lighting conditions in actual indoor and outdoor scenes with a small number of parameters, with mAP@50>50%.

[0026] To simplify the explanation of this method, the above figures and text are described as a series of steps. However, it should be understood and understood that these methods are not restricted by the order of operations, because some actions may occur in different orders when implemented according to one or more steps, but those skilled in the art can understand the principle behind the actions.

[0027] Although illustrative specific embodiments of the invention have been described step by step to enable those skilled in the art to understand them, the invention is not limited to the scope of the specific embodiments. Those skilled in the art can make various modifications or variations within the scope of the claims, as long as such variations are within the spirit and scope of the invention as defined and determined by the appended claims.

Claims

1. A target detection method in a low-light scene, characterized in that, Includes the following steps: S1. Use an RGB camera to acquire images under different lighting conditions, and label the target to be detected according to the format of the COCO dataset; S2. Pre-trained teacher model, which enables the teacher model to extract the visual features of general targets, distinguish target categories, and locate target positions; S3. The knowledge distillation method of logical simulation is used to transfer the general knowledge learned by the large model to the lightweight small model. The position of the target detection box is modeled using probability distribution, and an adaptive label allocation strategy is designed to select the image distillation region. S4. After label assignment, positive samples participate in the calculation of target detection loss. To address the problem that the existing method may overfit the noise of the large model by forcing the small model to learn the large model using KL divergence, a bidirectional distillation loss based on Jensen-Shannon divergence is proposed. S5. Input the sample data from the test set into the final trained model and output the target detection results of the small model under low light conditions.

2. The method according to claim 1, characterized in that: The overall framework consists of data acquisition and annotation under different lighting conditions, data augmentation based on atmospheric diffusion model and gamma transform, teacher model pre-training on large-scale general object detection dataset, determination of distillation region using adaptive label assignment method, and design of small model training loss based on Jensen-Shannon divergence.

3. The method according to claim 2, characterized in that, A target detection dataset under low-light conditions is constructed, and atmospheric diffusion model and gamma transform are applied to add and change the illumination distribution of the images to achieve data augmentation. A teacher model is pre-trained on a large-scale general target detection dataset. The knowledge distillation method of logical simulation is used to transfer the general knowledge learned by the large model to the lightweight model. How to use the large model to guide the small model to learn relevant domain knowledge is determined, and the difference in output distribution between the large and small models is evaluated by using a loss function based on Jensen-Shannon divergence.

4. The method according to claim 1, characterized in that: In the dataset preparation and augmentation phase, images under different lighting conditions were first acquired using an RGB camera according to the task requirements, and the targets to be detected in the images were labeled according to the COCO dataset format. Gamma transform was used to augment the images. The mathematical formula for synthesizing fog is: I(x)=J(x)t(x)+A(1-t(x)) Where I(x) and J(x) represent the fogged image and its corresponding defogging image, respectively, A represents global atmospheric light, t(x) represents the projection map, and x represents each pixel in the image; the transmittance image is obtained by the following formula: t(x)=e -βd(x) Where d(x) represents the depth of field, β represents the atmospheric attenuation factor, and β is empirically set to [0, 1]; to cope with different lighting conditions, gamma transform is used to adjust the image brightness: f(x)=x γ Where f(x) represents the image after brightness adjustment, and γ represents the scaling factor, with the value range of γ being [1.5, 5.0].

5. The method according to claim 1, characterized in that: The parameter update process of the pre-trained teacher model ResNet101 is represented as follows: Where, θ T Here, represents the parameters of the teacher model, t represents the number of gradient updates, and β0 represents the training learning rate of the teacher model. L(θ) represents the gradient of the teacher model. T ) represents the total loss during teacher model training.

6. The method according to claim 1, characterized in that: The knowledge distillation method of logical simulation is used to pass the general knowledge learned by the large model ResNet101 to the lightweight model ResNet18. Given a batch of input images, the ResNet101 teacher model and the ResNet18 student model perform feature extraction respectively, and combine the FPN network to obtain multi-scale features, and predict the probability distribution of detection boxes in the multi-scale feature layer. Using ResNet as the backbone network, the output of the predicted bounding box is a 4×n probability distribution, where n represents the granularity of dividing the coordinate box interval [0, 1] proportionally. The larger n is, the smaller the sub-intervals, and the greater the computational complexity of the model. Therefore, balancing model complexity and computational accuracy, n is chosen to be 16. This is achieved by dividing the regression interval of each edge [e...] min ,e max Discretize into a uniform distribution of length n [e1, e2, ..., e n The above formula can be written as: Among them, Pr(e i ) is to obtain each edge e i The probability density function is obtained from the teacher model prediction, where i represents the index number of each variable; assuming z is the logical value of an edge of the prediction box, represented by n discrete probabilities, the distribution of this edge by the teacher model and the student model can be expressed as z T and z S The distribution of these logical values ​​can be rewritten as a probability distribution using the following Softmax formula: Where i, j are index numbers, and e in the formula represents the natural constant; z i and z j Let z represent the i-th and j-th probability values ​​of a probability distribution, respectively, and be a scalar; while z... T and z S This represents the probability output of the teacher and student models, and is a vector; τ is set to 10; after applying the Softmax formula, z... T and z S Transformed into probability distribution p T and p S For the l-th level of the feature pyramid, a pre-defined set of truth boxes is used. and anchor frame set For each anchor box, determine whether it is a background region or a target region based on the ground truth region; first, calculate the GIoU score set X for each anchor box and the ground truth box. l : Where, x ij Indicates anchor frame With truth value The GIOU score can be calculated using the following formula: Where A and B represent two detection boxes, and C represents the smallest rectangle encompassing A and B; for each true value The GIOU scores are ranked, and the anchor boxes corresponding to the top k scores are taken as candidate boxes. The mean μ and standard deviation σ of the GIOU of these k anchor boxes and the ground truth box are calculated. An adaptive scaling factor γ is designed to adjust the positive sample allocation threshold α. α=γ(μ+σ) Where γ∈[0-1], μ and σ represent the mean and standard deviation of the confidence scores of the k candidate boxes, respectively; then, positive sample regions are selected based on α: V l ={v ij |x ij >α} Among them, V l It represents a set of truth boxes. and anchor frame set The matching binary mask matrix, v ij ∈{0,1} represents the anchor frame With truth value Whether it matches or not, v ij =1 represents the anchor frame It is assigned as a positive sample to participate in the target detection loss calculation.

7. The method according to claim 1, characterized in that: To determine how to use a large model to guide a small model in learning relevant domain knowledge, traditional methods use KL divergence to measure the difference in output between the large and small models. Where, p T (e i ) and p S (e i ) represent the predictions of the teacher model and the student model for a certain edge in e, respectively. i The probability at point p S =Softmax(z S ),p T =Softmax(z T ); Learning using a student model supervised by JS divergence: in, p T and p M Asymmetric similarity: After calculating the similarity of an edge distribution, the big-small model distillation loss L of a detection box is... JS Given by the following formula: Among them, B T and B S Let B represent the predicted bounding boxes of the teacher model and the student model, respectively, and let e represent an edge of the bounding box. JS divergence is used as the localization loss, along with 3) gradient smoothness during the training phase. The difference between KL and JS during the student model update can be expressed as: The JS divergence is due to the intermediate distribution. Maintain boundedness; The parameter update process of the student model is represented as follows: Where, θ S Here, represents the parameters of the student model, t represents the number of gradient updates, and β1 represents the learning rate. L(θ) represents the gradient of the student model. T ) represents the total loss during student model training.

Images