Teacher tensor knowledge distillation-driven anti-violation target detection method in electric power safety supervision scene

Abstract

The invention belongs to the technical field of target detection, and particularly relates to a teacher tensor knowledge distillation-driven anti-violation target detection method in an electric power safety supervision scene, and the method comprises the following steps: inputting image data to a student model and a pre-trained teacher model; performing forward propagation on the student model to obtain a student prediction tensor; associating differences between the real tags based on the student prediction tensor and the input image data; acquiring an original teacher output tensor by using the same input image data by calling a bottom layer forward propagation method of a teacher model; calculating knowledge distillation loss based on the difference between the student prediction tensor and the original teacher output tensor; the standard detection loss and the knowledge distillation loss are combined to form total loss; and performing back propagation updating on parameters of the student model based on the total loss so as to complete model training and real-time target detection and detection method updating optimization. According to the invention, the performance of the student model can be significantly improved, and the high efficiency of the model is maintained.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of target detection, and particularly relates to a teacher tensor knowledge distillation driven anti-illegal target detection method in an electric power safety monitoring scene. BACKGROUND

[0002] As a core technology of computer vision, target detection undertakes the key task of identifying illegal behaviors and abnormal states of equipment in the field of electric power safety inspection. Unlike general scenes, electric power safety inspection needs to accurately locate the operation of high-voltage live areas, the illegal use of safety tools (such as operating disconnectors without wearing insulating gloves), and the non-wearing of protective equipment by workers (such as the absence of safety helmets and the damage of insulating shoes), and needs to cope with complex environmental interference (such as strong electromagnetic fields and equipment shielding). Although the YOLO series model based on deep learning has achieved a balance between precision and speed in the general monitoring field, its direct application in electric power safety inspection cannot meet the dual needs of high-precision feature recognition (such as insulator cracks and conductor sag abnormalities) and lightweight deployment of edge devices (such as the NVIDIJetson Nano carried by a substation inspection robot). However, high-performance target detection models usually have large parameter quantities and computational complexity, which limits their deployment on resource-constrained devices (such as edge computing platforms or mobile devices). To solve this problem, model compression technology has emerged, and knowledge distillation (KD) is a widely used and effective technology. The core idea of KD is to train a smaller "student" model to imitate the behavior of a pre-trained, larger "teacher" model, thereby migrating the "knowledge" of the teacher model to the student model. Knowledge distillation can be divided into response-based KD, feature-based KD, and relation-based KD according to the source of knowledge. Response-based KD usually imitates the output (prediction logits or soft labels) of the last layer of the teacher model, and feature-based KD focuses on making the student model imitate the feature maps of the intermediate layers of the teacher model.

[0003] Problems of the prior art: Although KD has achieved remarkable success in tasks such as image classification, its application in the field of object detection still faces challenges. The object detection task is more complex than classification because it involves both the classification and localization (regression) of objects, and many existing KD methods for object detection mainly focus on feature imitation, or when imitating the prediction, mainly focus on the classification task, ignoring the importance of the regression task. Some studies point out that directly imitating the prediction logits of the teacher model is not efficient in object detection, especially in transferring localization information, and its effect is usually not as good as feature imitation, because there may be a large difference between the prediction of the teacher model and the real label assigned to the student, leading to inconsistent optimization objectives; In addition, under non-ideal imaging conditions such as low light environments, the performance of object detection will decrease dramatically. Low-light images often have low contrast, high noise, and color distortion, making feature extraction and object recognition extremely difficult. Although there are some low-light image enhancement techniques, they may introduce artifacts or fail to completely restore lost information, and treating enhancement and detection as two independent steps may not be optimal. Therefore, it is of great significance to develop a method that can directly improve detection performance under low-light conditions while maintaining model lightweight. The existing technology still has room for improvement in implementing knowledge distillation for object detection, especially in response-based distillation, in terms of how to effectively obtain and utilize the prediction information of the teacher model, and how to achieve significant performance improvement in challenging low-light scenarios. SUMMARY

[0004] The purpose of the present application is to provide a teacher tensor knowledge distillation driven anti-violation object detection method in the power safety monitoring scene, which can obtain the prediction output of the teacher model and apply it to the knowledge distillation process, especially in handling detection scenarios such as low light, which can significantly improve the performance of the student model while maintaining the efficiency of the model.

[0005] The technical solutions adopted by the present application are as follows: A teacher tensor knowledge distillation driven anti-violation object detection method in the power safety monitoring scene, comprising the following steps: Input image data to the student model and the pre-trained teacher model; Forward propagation is performed on the student model to obtain a student prediction tensor; Based on the difference between the student prediction tensor and the associated real label of the input image data, a standard detection loss is calculated; By calling the underlying forward propagation method of the teacher model, an original teacher output tensor is obtained using the same input image data; Based on the difference between the student prediction tensor and the original teacher output tensor, a knowledge distillation loss is calculated; combining the standard detection loss with the knowledge distillation loss to form a total loss; performing back propagation update on parameters of the student model based on the total loss to complete model training and real-time target detection and detection method update and optimization, wherein the teacher model parameters remain unchanged.

[0006] According to another aspect of the embodiment of the present application, the knowledge distillation loss includes KL divergence loss of classification logits and L2 loss of boundary box regression parameters, the knowledge distillation loss is calculated using KL divergence between probability distributions derived from classification logits existing in the student prediction tensor and the original teacher output tensor; wherein a temperature factor T is used to soften the classification logits when calculating the KL divergence, and T > 1, the temperature factor T is dynamically adjusted according to the illumination intensity of the input image.

[0007] According to another aspect of the embodiment of the present application, the student prediction tensor contains boundary box coordinates, class confidence and multi-scale feature maps. The standard detection loss includes boundary box regression loss and classification cross-entropy loss.

[0008] According to another aspect of the embodiment of the present application, the original teacher output tensor is obtained by calling the underlying forward method of the teacher model inside the trainer class, and the teacher model is in evaluation mode and the parameters are frozen, and the distillation loss is automatically set to zero and logs are recorded when the gradient is abnormal.

[0009] According to another aspect of the embodiment of the present application, the trainer class is used to run the teacher tensor knowledge distillation driven anti-illegal target detection method, and the trainer class inherits from the basic detection trainer class in the machine learning framework.

[0010] According to another aspect of the embodiment of the present application, the student model and the teacher model are constructed based on the visual target detection algorithm YOLO architecture, the student model is a lightweight yolov10n, the teacher model is yolov10x, and the channel number of the teacher model output tensor is dynamically compressed to the student model dimension through a dynamic channel alignment module (DCAM) in the knowledge distillation process, and the compression parameter is self-adaptively adjusted according to the illumination intensity of the input image during the training process.

[0011] According to another aspect of the embodiment of the present application, a teacher tensor knowledge distillation driven anti-illegal target detection system in the power safety monitoring scene comprises: a student model; a pre-trained and parameter-frozen teacher model; a processor; A memory for storing a program, wherein the processor executes the program to implement the teacher tensor knowledge distillation driven anti-violation object detection method in the power safety scene.

[0012] According to another aspect of the embodiment of the present application, the trainer is configured to automatically set the distillation loss to zero and record logs when the gradient is abnormal, and the gradient direction coordinator analyzes the consistency of the gradient directions of the two types of losses through cosine similarity: If the consistency is lower than a threshold, the gradient of the distillation loss is projected and corrected.

[0013] According to another aspect of the embodiment of the present application, a real-time violation detection system on an edge device is provided, which is deployed with a student model trained by the method in any of the preceding aspects, and the student model is implemented in a self-built power violation dataset.

[0014] According to another aspect of the embodiment of the present application, a computer program product is also provided, which includes a computer program, and the computer program is executed by a processor to implement the method in any of the preceding aspects.

[0015] The technical effects achieved by the present application are: The present application is used for training an object detection model, directly calls the forward method of a teacher model in a trainer to obtain an original output tensor, and uses the original output tensor to calculate a knowledge distillation loss (especially a KL divergence loss based on classification logits), simplifies the knowledge transmission process, and can effectively guide the learning of a student model, the method achieves more than 100 times performance improvement on a low-light Exdark dataset compared with a baseline model, proves the significant advantage of the method in improving detection performance in adverse conditions, and meanwhile, the performance improvement is achieved without sacrificing model efficiency (the model size and inference complexity remain unchanged), which shows that the method of the present application provides an effective way for deploying a high-performance and high-robustness object detection model on a resource-limited device, and is especially suitable for scenes that need to process non-ideal imaging conditions such as low light.

[0016] The present application trains a teacher model and a student model jointly, the student model can learn high-order feature representation of the teacher model, improves the robustness to device occlusion and strong electromagnetic field interference, combines a dynamic temperature factor T and a gradient direction coordinator (GDC), and enhances the detection accuracy of the student model in low-light conditions. BRIEF DESCRIPTION OF DRAWINGS

[0017] Fig. 1 is a method flowchart of the present application; Fig. 2 is a knowledge distillation learning framework in the present application; Fig. 3 is a teacher model distillation flowchart in the present application. DETAILED DESCRIPTION

[0018] In order to make the objects and advantages of the present application clearer, the following will specifically describe the present application with examples. It should be understood that the following description is only used to describe one or several specific embodiments of the present application, and does not strictly limit the scope of protection of the present application.

[0019] It should be noted that the terms "first", "second" and the like in the present application are used to distinguish similar objects, and do not necessarily indicate a specific order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device including a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0020] According to an embodiment of the present application, a method embodiment of a teacher tensor knowledge distillation driven anti-violation target detection method in a power safety monitoring scene is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described herein can be executed in an order different from that shown herein.

[0021] As shown in Figs. 1 to 3 A teacher tensor knowledge distillation driven anti-violation target detection method in a power safety monitoring scene includes the following steps: S1, inputting image data to a student model and a pre-trained teacher model; S2, performing forward propagation on the student model to obtain a student prediction tensor; S3, calculating a standard detection loss based on the difference between the student prediction tensor and the associated real label of the input image data; S4, obtaining an original teacher output tensor using the same input image data by calling the underlying forward propagation method of the teacher model; S5, calculating a knowledge distillation loss based on the difference between the student prediction tensor and the original teacher output tensor; S6, combining the standard detection loss and the knowledge distillation loss to form a total loss; S7, performing backpropagation update on the parameters of the student model based on the total loss to complete model training and real-time target detection and detection method update and optimization, wherein the parameters of the teacher model remain unchanged.

[0022] In the step S1, the input image data is a power operation site monitoring video frame (such as a substation or a power transmission line construction scene), which contains small targets such as high-voltage live area boundary operation, illegal use of safety tools (such as operating an isolation switch without wearing insulated gloves), and operation personnel not wearing protective equipment (such as missing safety hats). The teacher model (such as YOLOv10x) is a pre-trained high-performance model with high precision but large parameter quantity. The student model (such as YOLOv10n) is a lightweight version (the size is compressed to 1 / 3 of the original model) that adapts to the deployment requirements of edge devices (such as NVIDIA Jetson Nano). The detection task is implemented on an edge device (such as NVIDIA Jetson Nano) to achieve 30 FPS real-time inference.

[0023] The student model is a lightweight model deployed on an edge device for real-time detection of illegal behavior. It can be distributedly deployed through an edge device as a carrier, such as a multi-point matrix deployment for important detection points, which can effectively detect dead angles in the visual angle and avoid misjudgment and omission, thereby improving the accuracy of judgment. A short or long distance communication system is provided between the edge device and the teacher model to realize data transmission. The teacher model is a high-precision model that only provides knowledge in the training stage, and the parameters are frozen. The frozen state is that the parameters are not updated.

[0024] As an optional embodiment, the input image data further includes image data preprocessing.

[0025] The input image is pre-segmented for recognition and judgment by the student model or the teacher model in the later stage. The process further analyzes the depth of the input image based on the depth information of the power equipment and the operator in the image to establish a three-dimensional position relationship model, including: A monocular depth estimation model (such as MiDaS) is used to generate a depth map of the image to obtain the relative distance between the power equipment and the operator. The horizontal distance, vertical height, and angle deviation between the operator and the power equipment are calculated with the power equipment as the center reference point. a) Calculate the standard detection loss based on the difference between the output tensor of the student model and the true label; b) Obtain the original teacher output tensor by calling the bottom forward propagation method of the teacher model; c) Calculate the knowledge distillation loss based on the difference between the student prediction tensor and the original teacher output tensor; d) Fuse the standard detection loss and the knowledge distillation loss to generate the total loss function; e) Based on the total loss, the student model parameters are updated by back propagation, and the operation personnel's station position is determined whether it meets the safety specification combined with the depth analysis result, wherein the teacher model parameters remain unchanged.

[0026] In step a, a monocular depth estimation model (such as MiDaS) is used to generate a depth map for the input image, converting the two-dimensional image into a pixel-level depth value matrix, quantifying the relative distance between the power equipment and the operator, wherein the depth map is the depth value (distance from the camera) corresponding to each pixel point in the image, used to construct the three-dimensional spatial information of the scene, and the center reference point is the center of the bounding box of the power equipment (such as the disconnecting switch) as the coordinate origin, calculating the spatial position relationship between the operator and the equipment.

[0027] Further, according to the horizontal coordinate difference between the operator and the equipment in the depth map, the actual lateral distance is calculated, and by the vertical coordinate difference in the depth map, it is determined whether the operator is within the safe operating height range of the equipment, and combined with the relative position of the operator's posture key points (such as hands and shoulders) and the equipment, it is calculated whether the operating angle deviates from the safe range.

[0028] According to the above, the alarm threshold is set, for example, the lateral / vertical distance is less than or greater than the standard operating distance, triggering the out-of-range alarm, setting the operating angle deviation exceeding the inclination or horizontal angle, especially in the operation process, such as limb stiffness caused by electric shock, etc. Condition, determine the operation alarm, and combined with the target detection result (such as whether to wear insulating gloves or protective clothing) comprehensive evaluation, to realize the evaluation of illegal risk.

[0029] As a supplement to the above scheme, the image obtained by the above depth-of-field method can also be used to obtain personnel or equipment position information by multiple monocular cameras. This scheme is already recorded in the prior art, so it will not be described in more detail.

[0030] According to the above scheme, the depth-of-field analysis eliminates the two-dimensional detection errors caused by equipment shielding or electromagnetic interference, improving the recognition accuracy of the out-of-range operation in the high-voltage live area. The recognition of such complex environment can also be realized by inputting the position information of each device into the model to determine the position of the personnel, making the personnel position determination more accurate in complex environments.

[0031] Further, in low-light scenes, the depth estimation model can supplement the missing details of the visible light image to assist in positioning the relative position of small targets (such as insulator cracks) and the operator.

[0032] The depth analysis module as a preprocessing unit only needs a lightweight depth estimation model (such as MiDaS small network), which is optimized with the subsequent teacher-student model distillation method to ensure the feasibility of edge device deployment.

[0033] In step S2, the student model performs multi-scale feature extraction on the input image. This multi-scale feature extraction involves using a pre-trained model and extracting several key layers for feature extraction. Then, a simple Feature Pyramid Network (FPN) is constructed through convolutional operations. Finally, three feature maps at different scales are output, such as layers P3, P4, and P5. After extracting the multi-scale features, the output is the original detection tensor, which includes the following information: 1. Bounding box coordinates (xywh) are used to describe the spatial location of the detected target, such as the rectangular frame of a safety helmet; 2. Category confidence, which represents the probability distribution of the target category. For example, the probability of wearing a safety helmet is 0.95, and the probability of wearing a safety helmet is 0.05. The above probability confidence is used to detect and judge the above-mentioned violations. 3. Multi-scale feature maps: Capture targets of different sizes through feature maps at different levels, such as detecting small targets like insulating gloves at the P3 layer and detecting large targets like out-of-bounds operations at the P5 layer.

[0034] Multi-scale feature maps can enhance the detection capability of small targets in power scenarios.

[0035] As an optional embodiment of this step, in the bounding box coordinates, by describing the spatial position of the detected target, by setting the lower bounding box recognized by the high-definition camera in the sub-center, the identification of sub-targets such as whether the safety helmet buckle is fastened is realized, and a sub-frame attached to and distinguished from the rectangular frame is established.

[0036] Furthermore, the targets in the sub-frames are probabilistically identified by confidence levels. For example, the probability of a safety helmet being fastened is 0.95, and the probability of it not being fastened is 0.05. The wearing of insulating gloves is also assessed to determine whether they are in compliance with regulations, such as whether the straps are closed and whether the insulating side is on the palm side.

[0037] Furthermore, targets within the sub-frames can be stratified using different hierarchical features, such as whether insulated gloves are worn in accordance with regulations (P2 level) and whether protective clothing is worn in accordance with regulations (P5 level).

[0038] As another optional embodiment of this step, within the bounding box, corresponding items such as insulator cracks and corresponding protective equipment types (protective clothing, insulating gloves, electrician's protective rope, etc.), abnormal conductor sag and corresponding protective equipment are also identified. By detecting maintenance targets and identifying the required equipment during the target maintenance process, the operation specifications can be checked to avoid violations, prevent problems before they occur, and provide early warnings to avoid danger.

[0039] Further, at the category confidence level, after identifying the maintenance target, the operation specification is entered or called to determine whether it is in a violation state, such as whether the power supply switch is in the off state, whether the protective clothing or insulating gloves are worn, and the like.

[0040] Further, through multi-scale feature detection, the power supply disconnection is marked as P1 layer, and the insulating gloves or insulating clothing in the operation process are marked as P3 layer, and the like.

[0041] According to the present step, the detection of small targets in the power scene and the identification of corresponding correlation factors in the corresponding scene application can be enhanced to realize the dressing specification in the store operation process.

[0042] According to step S3, the standard detection loss includes a bounding box regression loss and a classification cross-entropy loss, and the expression is: wherein the classification cross-entropy loss is used to measure the difference between the student model prediction category and the true label, such as the label “wearing insulating gloves” and the like; the bounding box regression loss uses CloU loss function to optimize the overlapping area of the predicted box and the true box and the distance of the minimum closure box, and the parameter weight = 0.5, = 1.0, which is optimized by grid search.

[0043] For example, if the loU of the bounding box of the student model and the true label box is 0.7, the CloU loss value is 0.3, and the baseline model loss value is 0.5, the student model positioning accuracy is high.

[0044] According to step S4, the original teacher output tensor is obtained by calling the underlying forward method of the teacher model inside the trainer class, and the teacher model is in the evaluation mode and the parameters are frozen, and the distillation loss is automatically set to zero and the log is recorded when the gradient is abnormal.

[0045] The teacher model parameters are frozen, and the original output tensor is obtained by directly calling its underlying function, including: 1. Classification logits: the original score of the teacher model for each category (such as the logits of “not wearing a safety helmet” is 2.1, and the logits of other categories are -3.0, 0.5); 2. Bounding box regression parameters: the bounding box coordinates (such as the center point xy, width and height wh) predicted by the teacher model; 3. Multi-scale feature map: high-dimensional features in the intermediate layer of the teacher model (such as 256-channel P3 layer features).

[0046] The original output tensor is the original prediction result of the teacher model without post-processing, and contains complete classification and positioning information.

[0047] Through this step, the underlying function can be directly called instead of relying on the encapsulation interface, ensuring that the student model can learn the complete thinking process of the teacher model, and making up for the shortcomings of the prior art that only focuses on classification logits.

[0048] According to step S5, the knowledge distillation loss includes the KL divergence loss of the classification logits and the L2 loss of the bounding box regression parameters, and the knowledge distillation loss uses the KL divergence to calculate between the probability distributions derived from the classification logits existing in the student prediction tensor and the original teacher output tensor. Wherein, when calculating the KL divergence, the classification logits are softened by using a temperature factor T, and T>1, and the temperature factor T is dynamically adjusted according to the illumination intensity of the input image.

[0049] The knowledge distillation loss includes the KL divergence loss of the classification logits and the L2 loss of the bounding box regression parameters, and the knowledge distillation loss uses the KL divergence to calculate between the probability distributions derived from the classification logits existing in the student prediction tensor and the original teacher output tensor. Wherein, when calculating the KL divergence, the classification logits are softened by using a temperature factor T, and T>1, and the temperature factor T is dynamically adjusted according to the illumination intensity of the input image, and the dynamic adjustment basis is: When the brightness value is lower than the threshold value, the temperature factor T is linearly decreased from the default value 4.0 to 1.0.

[0050] Knowledge distillation loss , wherein the KL divergence loss ( ) compares the probability distribution difference between the student model and the teacher model classification logits, and the formula is: ; Wherein, , logits of the teacher model and the student model, respectively; The temperature factor T=4.0 softens the probability distribution, which is used to suppress low confidence noise.

[0051] In addition, the L2 loss ( ) is used to compare the Euclidean distance of the bounding box regression parameters between the teacher model and the student model, and to strengthen the positioning consistency.

[0052] The temperature factor is a dynamic temperature factor, the average brightness value of the input image is estimated by a lightweight light intensity detection module, when the brightness value is less than 50, it is determined that it is a low light threshold, at this time T is linearly reduced from the default value 4.0 to 1.0, the teacher logits distribution is forced to be sharpened, and the class discrimination degree in the low light area is enhanced.

[0053] According to step S6, the total loss is a standard detection loss combined with a knowledge distillation loss , by weighted fusion of standard detection loss and knowledge distillation loss, the student model is driven to learn real labels and "dark knowledge" of the teacher model, such as inter-class similarity and high-order feature correlation.

[0054] Where, the parameters α=0.5, β=1.0, the synergistic promotion effect of joint optimization on classification (+5.2%mAP@0.75) and positioning (+8.7%mAP@0.5) is verified by ablation experiment to realize weight adjustment.

[0055] According to step S7, based on the total loss the student model parameters are updated by back propagation, while the teacher model parameters remain frozen (only as a knowledge source); during training, abnormality is detected by gradient amplitude threshold (such as gradient value exceeding 1e6 or NaN / Inf), triggering the following mechanisms: 1. Set the distillation loss of the current batch to zero; 2. Log (such as "gradient explosion @ batch{i}, logits difference: {diff}"); 3. Automatically reduce the learning rate to 0.5 times the original value to prevent subsequent iteration abnormalities.

[0056] Synchronously in this step, the gradient abnormality handling mechanism ensures the training stability in low light scenes and avoids model collapse caused by noise interference.

[0057] It should be noted that the following embodiments and experiments mentioned are only detailed descriptions of the present application, but not limit the scope of the present application.

[0058] As a further supplement to the above steps, the present anti-violation target detection method can be implemented using common deep learning frameworks, such as PyTorch, in a specific embodiment, the method of the present application is integrated into the Ultralytics YOLOv8 or YOLOv10 framework; the Ultralytics framework provides a set of tools and base classes for training, validating and deploying YOLO series target detection models, the present application realizes the knowledge distillation function by inheriting and modifying the basic trainer class.

[0059] Further, the student model and the teacher model are constructed based on a visual target detection algorithm YOLO architecture, the student model is a lightweight yolov10n, the teacher model is yolov10x, and in the knowledge distillation process, the channel number of the teacher model output tensor is dynamically compressed to the student model dimension through a dynamic channel alignment module (DCAM), and the compression parameter is self-adaptively adjusted according to the illumination intensity of the input image in the training process.

[0060] Further, the trainer class is used to run the teacher tensor knowledge distillation driven anti-violation target detection method, the trainer class is inherited from the basic detection trainer class (KDTrainer) in the machine learning framework, the teacher tensor knowledge distillation driven anti-violation target detection method is integrated in the machine learning training framework, and is realized by inheriting the basic detection trainer class of the Ultralytics YOLO framework, forming the custom trainer class KDTrainer.

[0061] In the trainer class KDTrainer, the rewriting of the loss function (criterion) is the core of the whole knowledge distillation process. Unlike the basic detection trainer, which only focuses on the error between the student model (student model) itself and the real label, the KDTrainer additionally introduces the “dark knowledge” of the teacher model, adjusts the output distribution of the student model, and makes it follow the normal model detection target while also learning more rich inter-class correlation information from the teacher model.

[0062] The process will be divided into three stages below, and the specific intention, process and key details of each stage will be described without referring to any code implementation.

[0063] First stage: acquisition and maintenance of standard detection loss In the training of a deep detection model, it is first necessary to ensure that the student model can complete the most basic detection task. This part mainly relies on the original detection training process in the Ultralytics framework, that is, calling the loss calculation interface of the parent class, inputting the prediction results of the student model and the current batch of images and labels, and obtaining the original detection loss value including classification loss, regression loss (bounding box positioning) and possible auxiliary loss (such as DFL distribution focal loss). These loss terms jointly constitute the “basic loss” in the training process, which ensures the fitting ability of the student model to real samples; by retaining and directly using the loss value of the parent class, both the efficient implementation of the framework for multiple detection losses and the seamless addition of new distillation losses can be achieved, thereby maintaining the scalability and compatibility of the entire loss function.

[0064] At the end of this stage, a set of standard detection loss data has been acquired, which includes both a scalar total loss value and a breakdown of multiple sub-item losses, providing an important basis for subsequent loss stacking and logging.

[0065] Second stage: teacher model inference and distillation loss calculation This stage will only be entered when the teacher model is successfully loaded and the distillation weight is greater than zero; First, in the context of not calculating gradients, the current batch of input images is fed into the forward inference process of the teacher model to obtain the original output tensor without post-processing. Unlike directly using the encapsulation interface, directly calling the underlying forward propagation method can obtain the complete network output, including both class predictions (logits) and original information such as bounding box parameters. Since the teacher model is in evaluation mode and its parameters are frozen at this time, its weights will not be accidentally updated.

[0066] Then, the key tensors for distillation, usually the logits distribution of classification predictions, need to be extracted from the outputs of the student model and the teacher model.

[0067] Through consistent indexing strategies, the outputs of the student model and the teacher model are aligned and extracted, mapping them to the same class dimension interval to ensure the effectiveness of subsequent calculations.

[0068] After extraction, the distillation loss is calculated based on the classic KL divergence (Kullback–Leibler divergence) norm: the logits of the student model are divided by the temperature coefficient (temperature factor T) and normalized by logarithm, and the logits of the teacher model are also applied to the temperature coefficient and normalized by pure normalization. Then, the KL divergence of the distribution in the class dimension is calculated, and then multiplied by the scaling factor of the temperature square and the distillation weight alpha to adjust the absolute strength and relative proportion of the distillation signal. This step fully utilizes the role of the temperature coefficient in softening the prediction distribution and exposing more similarity information between classes, while controlling the influence of the distillation loss in the overall loss system through alpha.

[0069] In specific implementation, intermediate results are also checked for validity: if the gradient calculation result contains non-finite values (such as NaN, Inf) or the tensor dimensions do not match, the distillation loss will be automatically set to zero, and the corresponding warning log will be printed to ensure the robustness and monitorability of the entire training process. This not only avoids the collapse of the entire training caused by an accidental numerical anomaly, but also allows developers to adjust hyperparameters or troubleshoot potential defects in the inference process in a timely manner.

[0070] Third stage: loss fusion and output After the independent calculation of the standard detection loss and the knowledge distillation loss is completed, the KDTrainer performs a weighted summation of the two parts of the loss to obtain the "total loss" scalar that is actually used for backpropagation during the training process. The summation logic follows the simple linear superposition principle, that is, the basic detection loss and the distillation loss are directly added together to form a dual goal of maintaining the fitting to the real label and aligning with the teacher model for the student model parameter update.

[0071] In order to facilitate training monitoring and log visualization, the KDTrainer will also aggregate each loss component (each sub-item in the standard detection loss, as well as the distillation loss) into a unified loss vector and return it to the training loop. In the training framework, this loss vector will be recorded to the console or TensorBoard monitoring tool, thereby helping researchers to understand the learning status of the model in each dimension in real time, such as classification accuracy improvement, regression error reduction, and evolution of distillation consistency signals, etc.

[0072] Through the organic connection of the above three stages, the KDTrainer seamlessly integrates the core elements of knowledge distillation while inheriting the original efficient detection training system of Ultralytics. The standard detection loss ensures that the student model can learn the necessary target detection capability; the temperature adjustment and weight control mechanism enables the distillation loss to play an appropriate guiding role at different training stages and different data distributions; and the numerical validity check and automatic weight reduction strategy adds an important safety insurance to the entire training process. Finally, the total loss returned by the KDTrainer not only drives the student model to gradually approach the discrimination boundary of the real label, but also enables the student model to obtain the advantages of class association and feature representation contained in the teacher model by borrowing the high-order semantic distribution information of the teacher model, under the premise of limited capacity.

[0073] In summary, the loss rewriting module of the KDTrainer is a set of rigorous and flexible loss construction scheme: it inherits and retains the basic loss calculation ability of the detection task from the parent class, extends the training target by introducing the distillation loss, and finally realizes the coordination and unification of the two in a weighted summation manner. This design not only makes full use of the rich knowledge of the teacher model, but also maintains the stability of the training process, achieving a balance between performance, generalization, and training efficiency for small models.

[0074] Integrate and train the above method When training using the method of the present application, first instantiate the KDTrainer class and pass in the necessary configurations such as the teacher model path, temperature factor T and alpha value; then, call the training function of the framework (e.g. the train() method of Ultralytics) and pass in the KDTrainer instance as the trainer; the training process is managed by the KDTrainer, including data loading, model forward propagation (student model), loss calculation (by overriding the criterion method, which internally contains the forward call of the teacher model and the KD loss calculation), backpropagation (only update the student model weights) and optimizer steps.

[0075] Experimental verification and results To verify the effectiveness of the method in the present application, the following experiments were conducted, especially on the Exclusively Dark (Exdark) dataset; the Exdark dataset contains images taken under extremely low light conditions and is a commonly used benchmark for evaluating the robustness of target detection models under poor lighting conditions.

[0076] Experimental setup: Baseline model: use the standard Ultralytics training process to train a lightweight student model architecture (e.g. yolov10n.pt) on the Exdark dataset as the baseline, without using any knowledge distillation.

[0077] Distilled model (method embodiment in the present application): use the same student model architecture (yolov10n), but train it through the KDTrainer of the present application; choose a larger and more powerful model (e.g. yolov10x.pt) as the teacher model. Set the distillation parameters, such as T=4.0 and alpha=0.5.

[0078] Performance indicators: use standard target detection evaluation indicators, including: Precision (precision); Recall (recall); mAP@50 (average precision mean when IoU threshold is 0.5); mAP@50-95 (average mAP when IoU threshold is from 0.5 to 0.95, step size 0.05); Key results: the experimental results are shown in the following table: Result analysis: As can be seen from the above table, the baseline yolov10n.pt model performs very poorly on the Exdark dataset, with all indicators close to zero, which reflects that under extremely low light conditions, the standard target detection model is difficult to work effectively.

[0079] In contrast, the 10nBest.pt model trained using the KDTrainer proposed in the present application (knowledge distillation through direct teacher tensor output) has achieved significant performance improvement in all indicators; for example, mAP@50 is improved from 0.00666 to 0.605, and mAP@50-95 is improved from 0.00370 to 0.382, with an increase of more than 100 times.

[0080] According to the above experiment, the effectiveness of the method in the present application is proved, that is, by directly using the original output tensor of the teacher model for knowledge distillation, the knowledge can be successfully transferred from the teacher model to the lightweight student model.

[0081] Secondly, the above-mentioned improvement is achieved on a low-light dataset, indicating that the method has great potential to overcome the detection difficulties under adverse conditions.

[0082] Thirdly, considering that the distillation method based on prediction / response in the prior art is often considered to have limited effect on the target detection task, the present application achieves such significant gains in this setting, highlighting the unique advantages of directly obtaining and utilizing the original teacher tensor output.

[0083] Finally, it is worth noting that the performance improvement is achieved without increasing the size or inference complexity of the student model, which fully meets the core goal of knowledge distillation as a model compression and efficiency improvement technology.

[0084] According to the above: For training a target detection model, the forward method of the teacher model is directly called in the trainer to obtain the original output tensor, which is used to calculate the knowledge distillation loss (especially the KL divergence loss based on classification logits), thereby simplifying the knowledge transfer process and effectively guiding the learning of the student model; according to the experimental results, the method achieves more than 100 times performance improvement on the low-light Exdark dataset compared to the baseline model, proving its significant advantage in improving detection performance under adverse conditions; at the same time, this performance improvement is achieved without sacrificing model efficiency (model size and inference complexity remain unchanged), which shows that the method of the present application provides an effective way for deploying high-performance and robust target detection models on resource-constrained devices, especially for scenarios that need to handle low-light and other non-ideal imaging conditions.

[0085] Referring toFig. 2 and Fig. 3 A teacher tensor knowledge distillation driven anti-violation target detection system in the power safety monitoring scene, comprising: A student model for real-time detection of power violation behaviors (such as not wearing a safety helmet, operating beyond the boundary), volume lightweight compression, and adaptation to edge device deployment; A pre-trained and parameter frozen teacher model, which only provides knowledge in the training stage, and the parameters remain frozen to ensure that it only serves as a knowledge source; A processor responsible for executing training and inference tasks, supporting hardware acceleration (such as the GPU of NVIDIA Jetson Nano); A memory for storing programs, and the processor executes the programs to implement the teacher tensor knowledge distillation driven anti-violation target detection method in the power safety monitoring scene.

[0086] The program instructions in the memory are executed by the processor to perform the following steps: 1. Input the power work site image data to the student model and the teacher model; 2. The student model outputs the predicted tensor (including the bounding box coordinates and class confidence); 3. The teacher model generates the original output tensor through the bottom forward() method; 4. Fusion standard detection loss (based on real labels) and knowledge distillation loss (based on teacher-student model output); 5. Update the student model parameters based on the total loss, and the teacher model parameters remain unchanged.

[0087] Based on the above, through joint training of the teacher model and the student model, the student model can learn the high-order feature representation of the teacher model, improve the robustness to device occlusion and strong electromagnetic field interference, and combine dynamic temperature factor T and gradient direction coordinator (GDC) to enhance the detection accuracy of the student model in low light conditions.

[0088] As an optional embodiment, the trainer is configured to automatically set the distillation loss to zero and record logs when the gradient is abnormal, and the gradient direction coordinator analyzes the gradient direction consistency of the two types of losses through cosine similarity: If the consistency is lower than the threshold, the gradient of the distillation loss is projected and corrected.

[0089] Wherein, the gradient abnormality processing is in the back propagation stage, and the abnormality is judged by gradient amplitude threshold or NaN / Inf detection.

[0090] Further, the student model is implemented in the self-built power violation data set.

[0091] It should be noted that the above modules can be implemented by software or hardware, and the hardware can be implemented in the following manner, but is not limited thereto: all the modules are located in the same processor; or the modules are located in different processors in any combination.

[0092] According to another aspect of the embodiments of the present application, an electronic device is also provided, which includes a memory and a processor; the memory is configured to store a program; and the processor is configured to execute the program to implement the method of any one of the preceding aspects.

[0093] According to another aspect of the embodiments of the present application, a computer readable storage medium is also provided, which stores a computer program; the computer program is configured to be executed by a processor to implement the method of any one of the preceding aspects.

[0094] According to another aspect of the embodiments of the present application, a computer program product is also provided, which includes a computer program; the computer program is configured to be executed by a processor to implement the method of any one of the preceding aspects.

[0095] The above description is only the preferred embodiments of the present application, and it should be pointed out that, for those skilled in the art, without departing from the principles of the present application, a number of improvements and refinements can be made, and these improvements and refinements should be considered as the protection scope of the present application. The structures, devices and operation methods not specifically described and explained in the present application are implemented according to the conventional means in the art, unless otherwise specified and limited.

Claims

1. A teacher tensor knowledge distillation driven anti-illegal target detection method in the power safety scene, characterized in that, The method comprises the following steps: inputting image data to a student model and a pre-trained teacher model; forward propagating the student model to obtain a student prediction tensor; calculating a standard detection loss based on the difference between the student prediction tensor and the real label associated with the input image data; obtaining an original teacher output tensor by calling a bottom forward propagation method of the teacher model using the same input image data; calculating a knowledge distillation loss based on the difference between the student prediction tensor and the original teacher output tensor; combining the standard detection loss and the knowledge distillation loss to form a total loss; updating the parameters of the student model based on the total loss by back propagation to complete model training and real-time target detection and detection method optimization, wherein the parameters of the teacher model remain unchanged.

2. The teacher tensor knowledge distillation driven anti-illegal target detection method in the power safety scene according to claim 1, characterized in that: The knowledge distillation loss comprises a KL divergence loss of classification logits and an L2 loss of bounding box regression parameters, and the knowledge distillation loss is calculated using KL divergence between probability distributions derived from classification logits existing in the student prediction tensor and the original teacher output tensor; wherein a temperature factor T is used to soften the classification logits when calculating the KL divergence, and T>1, and the temperature factor T is dynamically adjusted according to the light intensity of the input image.

3. The teacher tensor knowledge distillation driven anti-violation target detection method in the power safety scene according to claim 1, characterized in that: The student prediction tensor comprises bounding box coordinates, class confidence and multi-scale feature maps. The standard detection loss comprises a bounding box regression loss and a classification cross-entropy loss.

4. The teacher tensor knowledge distillation driven anti-violation target detection method in the power safety scene according to claim 1, characterized in that: The original teacher output tensor is obtained by calling the bottom forward method of the teacher model inside the trainer class, and the teacher model is in evaluation mode and the parameters are frozen, and the distillation loss is automatically set to zero and logs are recorded when the gradient is abnormal.

5. The teacher tensor knowledge distillation driven anti-violation target detection method in the power safety scene according to claim 4, characterized in that: The trainer class is used to run the teacher tensor knowledge distillation driven illegal target detection method, and the trainer class inherits from the basic detection trainer class in the machine learning framework.

6. The teacher tensor knowledge distillation driven anti-violation target detection method in the power safety scene according to claim 1, characterized in that: The student model and the teacher model are constructed based on the visual target detection algorithm YOLO architecture, the student model is a lightweight yolov10n, the teacher model is yolov10x, and the channel number of the teacher model output tensor is dynamically compressed to the student model dimension through a dynamic channel alignment module (DCAM) in the knowledge distillation process, and the compression parameter is adaptively adjusted according to the light intensity of the input image during training.

7. A teacher tensor knowledge distillation driven anti-illegal target detection system in the power safety scene, executes the method of any one of claims 1-6. It comprises: a student model; a pre-trained and parameter-frozen teacher model; a processor; a memory for storing a program, wherein the processor executes the program to implement the teacher tensor knowledge distillation driven illegal target detection method in the power safety monitoring scene.

8. The teacher tensor knowledge distillation driven anti-illegal target detection system in the power safety scene according to claim 7, wherein: The trainer is configured to automatically set the distillation loss to zero and record logs when the gradient is abnormal, and the gradient direction coordinator analyzes the gradient direction consistency of the two types of losses by cosine similarity: If the consistency is lower than a threshold, the gradient of the distillation loss is projected and corrected.

9. A real-time violation detection system on an edge device, characterized in that, The student model trained by the method of any one of claims 1-6 is deployed in a self-built power violation data set.

10. A computer program product comprising a computer program, characterized in that, The computer program is executed by the processor to implement the method of any one of claims 1 to 6.

Images