Real-time target intelligent detection method and device based on double-flow perception and endogenous knowledge distillation

Abstract

The invention discloses a real-time target intelligent detection method and device based on double-flow perception and endogenous knowledge distillation, and provides a target detection model named as SSD-DSKD. According to the method, a double-flow sensing module and a knowledge distillation technology are combined, the double-flow sensing module processes continuous video frames, captures space static characteristics and time dynamic change characteristics respectively, fuses the space static characteristics and the time dynamic change characteristics and then inputs the fused space static characteristics and time dynamic change characteristics as an SSD target detection model for training; the SSD target detection model serves as a teacher model, and a soft label of the teacher model is used for training a student model. In the training process of the student model, the output smoothness of the teacher model is adjusted through temperature attenuation, an attention mechanism is introduced, and the student model is optimized. Compared with the prior art, the method can effectively improve the accuracy and efficiency of target detection, enhances the robustness of the model in different scenes, and achieves a higher automation level.

Description

Technical Field

[0001] The present invention belongs to the field of video processing and machine vision technology, and in particular relates to a real-time target intelligent detection method and device based on dual-stream perception and endogenous knowledge distillation. Background Art

[0002] With widespread application in fields such as video surveillance, autonomous driving, and intelligent manufacturing, object detection technology has gradually become a key technology. Existing object detection methods primarily rely on deep learning models for training. However, due to the complexity of target scenes, especially the interweaving of dynamic and static information, traditional single detection models struggle to provide stable and efficient detection performance in complex environments. Effectively integrating different types of feature information to improve the accuracy and robustness of object detection models has become a key research topic.

[0003] In recent years, dual-stream perception technology has garnered widespread attention in the video and video processing fields. The dual-stream perception module simultaneously extracts static and dynamic features to comprehensively capture all aspects of a target, improving target detection performance. By organically combining static and dynamic streams, the dual-stream perception module can obtain richer feature information when processing targets in complex scenes, enhancing the model's recognition capabilities. However, in practical applications, further optimizing the dual-stream feature fusion and improving model training efficiency remain pressing technical challenges.

[0004] Knowledge distillation, as a method for improving the performance of deep learning models, is widely used in various computer vision tasks. In object detection, knowledge is transferred from a teacher model to a student model, effectively improving the student's performance and reducing computing resource consumption. Knowledge distillation, combined with the application of a two-stream perception module, further enhances the student model's ability to recognize complex objects, particularly its robustness in detecting small objects and in high-noise environments. Combining two-stream perception with knowledge distillation, a real-time intelligent object detection method has been proposed, which offers significant technical advantages and effectively improves the accuracy and efficiency of object detection. Summary of the Invention

[0005] Purpose of the invention: In response to the problems in the background technology, the present invention discloses a real-time intelligent target detection method and device based on dual-stream perception and endogenous knowledge distillation. By comprehensively applying the dual-stream perception module and endogenous knowledge distillation technology, the feature representation of the video is optimized to achieve high-precision detection of the detection target.

[0006] Technical Solution: This invention proposes a real-time intelligent target detection method based on dual-stream perception and endogenous knowledge distillation, which includes the following steps:

[0007] Step 1: Obtain video frame data in the target scene, perform data preprocessing, enhancement, and annotation, and divide it into video dataset G, verification dataset G_val, and test dataset G_test in a ratio of 8:1:1;

[0008] Step 2: Use the dual-stream perception module to extract the fusion feature map F conv , used as the input of the SSD detection model for training, and the performance is evaluated on the validation dataset. The verified SSD detection model is used as the teacher model; the dual-stream perception module consists of a static stream and a dynamic stream, which respectively capture spatial static features and temporal dynamic change features and fuse the spatial static features and temporal dynamic change features;

[0009] Step 3: Based on the teacher model, the student model is trained using the knowledge distillation method of temperature decay. Soft labels are generated for the logits output by the teacher model to fit the teacher model's prediction distribution. An attention alignment mechanism is introduced to narrow the differences in feature focus areas between the two models. A weighted total loss function is constructed using a combination of cross-entropy distillation loss and attention loss. The temperature parameter is dynamically adjusted based on the convergence rate during training. Finally, the student model is determined through performance verification.

[0010] Step 4: Use the student model to evaluate the test dataset, analyze the misclassification and model attention areas through the confusion matrix and Grad-CAM, deploy the student model on the server and generate performance evaluation reports regularly.

[0011] Furthermore, the specific method of step 1 is:

[0012] Step 1.1: Collect dynamic video frames of various targets in the target scene. Ensure that the collected data covers different weather and lighting conditions, and that the video frames contain a variety of target categories and cover different behavior patterns from static to fast-moving.

[0013] Step 1.2: Normalize the captured original video frames to a uniform video resolution of 1080p and normalize the pixel value range to [0, 1].

[0014] Step 1.3: Apply inter-frame data augmentation technology to rotate each video frame at random angles ranging from -30 degrees to 30 degrees to simulate different viewing angles;

[0015] Step 1.4: Use Gaussian filtering to remove noise and artifacts in the video.

[0016] Step 1.5: Use the video annotation tool CVAT to manually annotate key frames and automatically annotate non-key frames for dynamic objects using interpolation technology;

[0017] Step 1.6: Divide the processed dataset into training set G, validation set G_val, and test set G_test in a ratio of 8:1:1.

[0018] Furthermore, the specific method of the dual-stream perception module in step 2 is:

[0019] First, the pre-trained ResNet-50 network is used to train the continuous frames F t and F t-1 Extract static features P t and P t-1 , and calculate the dynamic characteristics D by difference t =P t -P t-1 , threshold the dynamic feature amplitude and filter out frames with significant dynamic events; for significant frames, P t With D t Perform splicing and fusion to generate the original fusion feature map F concat , and then through the 1×1 convolution operation, the semantically optimized fusion feature map F is output refined , and then obtain the regularized feature map F through Batch Normalization conv .

[0020] Furthermore, the step 2 performs a model performance test on the SSD detection model on the validation set G_val. If the accuracy Acc DFP ≥90% and F1 DFP If the performance reaches ≥90%, the performance meets the expected standard, and the SSD candidate model is used as the teacher model for subsequent knowledge distillation. Otherwise, the number of output channels is first increased to 512 channels, and the model is re-optimized. If the performance still does not meet the standard after optimization, the number of output channels is reduced to 128 channels, and the model is optimized again. If the performance still does not meet the standard after two optimizations, the adjustment is stopped, and the one with the best performance on the validation set is selected from all trained SSD detection models as the teacher model to enter the subsequent knowledge distillation stage.

[0021] Furthermore, in step 3, both the student model and the teacher model are based on the SSD detection model. The structure of the student model is consistent with that of the teacher model, but its parameters are generated by knowledge distillation training, as follows:

[0022] Step 3.1: Set the initial temperature T initial , ranging from 1 to 5, initially set to 1, and introduce the temperature control variable T current , used to control the smoothness of the teacher model output, T current Initialized to T initial , dynamically adjusted with epoch during training, setting the change rate threshold η and loss threshold u;

[0023] Step 3.2: During the training process, gradually reduce the temperature T using the exponential decay formula current ,Dynamically adjust the temperature to optimize the output smoothness of the teacher model.,Exponential decay formula: Where T initial is the initial temperature, λ T is the decay rate, epoch is the current training round;

[0024] Step 3.3: Use the temperature parameter T obtained in step 3.2 current , generate soft labels for the teacher model formula: in, is the logits output of the teacher model, Sum the indices of all categories for normalization to ensure that the sum of the probabilities of all categories is 1;

[0025] Step 3.4: Use the temperature parameter T obtained in step 3.2 current , generates the predicted probability distribution of the student model currently being trained formula: in, is the logits output of the student model currently being trained, Sum the indices for all categories;

[0026] Step 3.5: Calculate the cross entropy distillation loss between the teacher model and the student model currently in training: C is the total number of target detection categories;

[0027] Step 3.6: Calculate the rate of change R of training loss every 10 epochs. Formula: Among them, L current is the average loss of the current 10 rounds, L previous is the average loss of the first 10 rounds;

[0028] Step 3.7: If the rate of change R is less than or equal to the threshold η, the training is stable and we proceed to step 3.8. Otherwise, the training is still converging and we proceed to step 3.6 for the next 10 epochs of training and loss calculation.

[0029] Step 3.8: Introduce the attention mechanism into the student model currently being trained and calculate the attention loss: Among them, A teacher (x,y) is the attention value of the teacher model at the feature map position (x,y), A student (x, y) is the attention value of the student model in the current training at the feature map position (x, y), H and W are the height and width of the feature map;

[0030] Step 3.9: Define the weighted total loss function: L total =L KD +ρ·L attention , where L KD is the cross entropy distillation loss in step 3.5, ρ is the hyperparameter that weighs the attention loss, and L attention is the attention loss in step 3.8, if L total If the loss is less than or equal to the loss threshold u, the training is completed and step 3.10 is executed. Otherwise, step 3.2 is executed again and the training optimization is performed.

[0031] Furthermore, when evaluating the performance of the student model in the current training on the validation set G_val, if the accuracy Acc stu ≥90% and F1 stu ≥90%, the verification is completed and the student model currently being trained is used as the final student model; otherwise, the initial temperature T is adjusted. initial , retrain, and cycle optimization training for up to 3 times. If the optimization still fails to meet the standard after three times, stop adjusting and select the one with the best performance on the validation set from all trained student models as the final student model for subsequent deployment and further evaluation.

[0032] Furthermore, the specific method of step 4 is:

[0033] Step 4.1: Use the student model to predict the test data set and calculate the evaluation indicators: Accuracy, F1 Score, and Loss.

[0034] Step 4.2: Use the confusion matrix to identify highly misclassified categories and calculate the recall rate of each category. Formula: Among them, TP i is the number of correctly predicted positive samples in the i-th category, FN i is the actual number of positive samples predicted as negative in the i-th category;

[0035] Step 4.3: Use Grad-CAM to visualize the model's attention area and calculate the Intersection over Union (IoU) ratio between the highlighted area in the Grad-CAM heatmap and the target ground truth area. If the IoU is high, it means that the model is paying attention to the correct area. Otherwise, it means that the model is paying attention to other areas incorrectly. The formula is as follows: The Grad-CAM focus area is the highlighted area of ​​the heat map generated by Grad-CAM, and the real area is the bounding box of the real object;

[0036] Step 4.4: Deploy the student model to the cloud server and expose it as a REST API, allowing other applications to perform inference through network requests;

[0037] Step 4.5: Use the performance monitoring tool Prometheus to track the model's inference speed, latency, and resource consumption.

[0038] Step 4.6: Generate student model performance evaluation reports regularly.

[0039] The present invention also discloses a real-time target intelligent detection method and device based on dual-stream perception and endogenous knowledge distillation, including a memory, a processor, and a computer program stored in the memory and runnable on the processor. When the computer program is loaded into the processor, the real-time target intelligent detection method and device based on dual-stream perception and endogenous knowledge distillation are implemented.

[0040] Beneficial effects:

[0041] 1. This invention combines a dual-stream perception module with endogenous knowledge distillation technology to optimize the representation and processing of video features, improving the precision and accuracy of real-time object detection. By fusing static and dynamic streams, the model can more accurately extract target feature information, achieving high-precision defect identification and classification in complex environments. Endogenous knowledge distillation technology improves the performance of the student model, enabling it to effectively learn the knowledge of the teacher model, enhancing detection results and the model's generalization ability.

[0042] 2. This invention gradually optimizes the temperature decay strategy during training, making the soft labels output by the teacher model smoother, helping the student model acquire richer knowledge during training. This improvement improves target detection accuracy, particularly when processing complex backgrounds and low-contrast videos, reducing false detections and missed detections. By introducing an attention mechanism, the student model can focus on the target area during learning, improving its target recognition ability, especially for small target detection and complex scenes.

[0043] 3. This invention can be widely applied to industrial automated quality inspection systems, suitable for real-time defect detection in high-speed, large-scale production environments. By utilizing dual-stream sensing and endogenous knowledge distillation technology, this invention improves the automation and accuracy of quality control in the production process. This technological innovation can help manufacturers reduce the economic losses caused by defects, improve product quality reliability, and enhance consumer trust in their products. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 It is the overall flow chart of the method of the present invention;

[0045] Figure 2 This is a flowchart of product video acquisition and preprocessing;

[0046] Figure 3 This is the processing flow chart of the dual-stream perception module;

[0047] Figure 4 This is the flow chart for endogenous knowledge distillation;

[0048] Figure 5 Provide a flowchart for model evaluation and deployment;

[0049] Figure 6 This is the SSD-DSKD model architecture diagram;

[0050] Figure 7 This is an example of a data set in an embodiment of the present invention. DETAILED DESCRIPTION

[0051] The present invention is further illustrated below with reference to specific examples. It should be understood that these examples are only used to illustrate the present invention and are not used to limit the scope of the present invention. After reading the present invention, modifications of various equivalent forms of the present invention made by those skilled in the art all fall within the scope defined by the claims attached to this application.

[0052] The present invention discloses a real-time target intelligent detection method and device based on dual-stream perception and endogenous knowledge distillation. The real-time target intelligent detection method based on dual-stream perception and endogenous knowledge distillation includes the following steps:

[0053] Step 1: Obtain the video frame data in the target scene, perform data preprocessing, enhancement and annotation, and divide it into video dataset G, verification dataset G_val and test dataset G_test in the ratio of 8:1:1. Figure 2 As shown:

[0054] Step 1.1: Select a suitable HD camera as the data acquisition device and ensure that the device supports acquisition under different lighting and viewing angle conditions;

[0055] Step 1.2: Collect dynamic video frames of various objects (pedestrians, vehicles, and machinery) in the target scene. Ensure that the collected data covers different weather and lighting conditions, and that the video frames contain a variety of object categories, covering different behavior patterns from static to fast-moving.

[0056] Step 1.3: Normalize the captured original video frames to a uniform video resolution of 1080p and normalize the pixel value range to [0, 1].

[0057] Step 1.4: Apply inter-frame data augmentation technology to rotate each video frame by a random angle ranging from -30 degrees to 30 degrees to simulate different viewing angles and enhance the model's adaptability to changing conditions.

[0058] Step 1.5: Use Gaussian filtering to remove noise and artifacts in the video, improve video quality, and reduce errors during training.

[0059] Step 1.6: Manually annotate the key frames using the video annotation tool CVAT, ensuring that the trajectory and category of each target are marked. For dynamic targets, use interpolation technology to automatically annotate non-key frames;

[0060] Step 1.7: Divide the processed dataset into training set G, validation set G_val, and test set G_test in a ratio of 8:1:1.

[0061] Step 2: Use the dual-stream perception module to extract the fusion feature map F conv , as the input of the SSD detection model. The dual-stream perception module consists of a static stream and a dynamic stream, which capture the spatial static features and the temporal dynamic change features respectively. The specific execution process of the dual-stream perception module is as follows: First, the pre-trained ResNet-50 network is used to analyze the continuous frames F t and F t-1 Extract static features P t and P t-1 , and calculate the dynamic characteristics D by difference t =P t -P t-1 The dynamic feature amplitude is thresholded and the frames with significant dynamic events are screened out. For significant frames, their P t With D t Perform splicing and fusion to generate the original fusion feature map F concat , and then through the 1×1 convolution operation, the semantically optimized fusion feature map F is output refined , and then obtain the regularized feature map F through Batch Normalization conv .

[0062] Finally, F conv Input into the SSD detection model, train it, and evaluate its performance on the validation set. The validated model is used as the teacher model. Figure 3 As shown:

[0063] Step 2.1: Select the pre-trained convolutional neural network ResNet-50 as the feature extractor, set the feature change threshold θ, set the initial frame index t to 1, and the total number of frames in the training set G to N.

[0064] Step 2.2: Read two consecutive frames of video F from the dataset G in sequence t and F t-1 , each time a frame is read, t increases by 1, where F t is the current frame, F t-1For the previous frame.

[0065] Step 2.3: Use the pre-trained convolutional neural network ResNet-50 and a 3x3 convolution kernel with a stride of 1 to extract the convolution kernel from F. t and F t-1 Extract static features from the static feature P t and P t-1 , where P t is the static feature of the current frame, P t-1 is the static feature of the previous frame.

[0066] Step 2.4: For the static feature P t and P t-1 Perform the difference and calculate the current frame F t Dynamic characteristics of D t , the formula is: D t =P t -P t-1 .

[0067] Step 2.5: If the absolute value of the dynamic characteristic |D t If | exceeds the threshold θ, proceed to step 2.6; otherwise, proceed to step 2.7.

[0068] Step 2.6: Set the current frame F t Mark as containing significant dynamic events and record the dynamic features of the current frame D t and static characteristics P t .

[0069] Step 2.7: If t≤N-1, execute step 2.2 and continue reading the next pair of consecutive frames; otherwise, execute step 2.8.

[0070] Step 2.8: For the frames marked as containing significant dynamic events in step 2.6, extract the recorded static features P t With dynamic characteristics D t , spliced ​​according to the channel dimension to generate the original fusion feature map F concat , the formula is: F concat =Concat(P t ,D t ).

[0071] Step 2.9: F concat Apply a 1×1 convolution operation to compress the channel dimension and output the semantically optimized fused feature map F refined The initial number of channels is set to 256 to improve the compactness of feature expression.

[0072] Step 2.10: F refinedApply the Batch Normalization operation to obtain the regularized feature map F conv .

[0073] Step 2.11: F conv This is fed into the SSD detection model as input and trained end-to-end. The training results in an SSD candidate model with detection capabilities.

[0074] Step 2.12: Test the performance of the SSD candidate model on the validation set G_val. If the accuracy Acc DFP ≥90% and F1 DFP If the performance reaches ≥90%, the performance meets the expected standard, and the SSD candidate model is used as the teacher model for subsequent knowledge distillation. Otherwise, execute step 2.9, increase the number of output channels to 512 channels, and re-optimize the model. If the performance still does not meet the standard after optimization, reduce the number of output channels to 128 channels and optimize the model again. If the performance still does not meet the standard after two optimizations, stop adjusting, select the one with the best performance on the validation set from all trained SSD candidate models, and use it as the teacher model to enter the subsequent knowledge distillation stage.

[0075] Step 3: Based on the teacher model, the knowledge distillation method of temperature decay is used to train the student model. By generating soft labels for the logits output by the teacher model, the prediction distribution of the teacher model is fitted, and the attention alignment mechanism is introduced to narrow the difference between the two in the feature focus area. The cross entropy distillation loss and attention loss are used to construct a weighted total loss function. The temperature parameter is dynamically adjusted based on the convergence rate during training. Finally, the student model is determined through performance verification for subsequent deployment. Figure 4 As shown:

[0076] Step 3.1: Set the initial temperature T initial (Range 1 to 5, initially set to 1), introduce temperature control variable T current , used to control the smoothness of the teacher model output, T current Initialized to T initial , which is dynamically adjusted with each epoch during training. Set the change rate threshold η and loss threshold u.

[0077] Step 3.2: During the training process, gradually reduce the temperature T using the exponential decay formula current ,Dynamically adjust the temperature to optimize the output smoothness of the teacher model.,Exponential decay formula: Where T initial is the initial temperature, λ T is the decay rate, and epoch is the current training round.

[0078] Step 3.3: Use the temperature parameter T obtained in step 3.2 current , generate soft labels for the teacher model formula: in, is the logits output of the teacher model, Sum the indices of all categories for normalization to ensure that the sum of the probabilities of all categories is 1.

[0079] Step 3.4: Use the temperature parameter T obtained in step 3.2 current , generates the predicted probability distribution of the student model currently being trained formula: in, is the logits output of the student model currently being trained, Sum the indices for all categories.

[0080] Step 3.5: Calculate the cross entropy distillation loss between the teacher model and the student model currently in training: C is the total number of target detection categories.

[0081] Step 3.6: Calculate the rate of change R of training loss every 10 epochs. Formula: Among them, L current is the average loss of the current 10 rounds, L previous is the average loss of the first 10 rounds.

[0082] Step 3.7: If the rate of change R is less than or equal to the threshold η, the training is stabilizing and we proceed to step 3.8. Otherwise, the training is still converging and we proceed to step 3.6 for the next 10 epochs of training and loss calculation.

[0083] Step 3.8: Introduce the attention mechanism into the student model currently being trained and calculate the attention loss: Among them, A teacher (x,y) is the attention value of the teacher model at the feature map position (x,y), A student (x, y) is the attention value of the student model currently being trained at the feature map position (x, y), and H and W are the height and width of the feature map.

[0084] Step 3.9: Define the weighted total loss function: L total =L KD +ρ·L attention . Where L KD is the cross entropy distillation loss in step 3.5, ρ is the hyperparameter that weighs the attention loss, and L attention is the attention loss in step 3.8, if L totalIf the loss is less than or equal to the loss threshold u, the training is completed and step 3.10 is executed. Otherwise, step 3.2 is executed again and the training optimization is performed.

[0085] Step 3.10: Evaluate the performance of the student model in the current training on the validation set G_val. If the accuracy Acc stu ≥90% and F1 stu ≥90%, the verification is completed and the student model currently being trained is used as the final student model; otherwise, return to step 3.1 to adjust the initial temperature T initial , retrain, and cycle optimization training for up to 3 times. If the optimization still fails to meet the standard after three times, stop adjusting and select the one with the best performance on the validation set from all trained student models as the final student model for subsequent deployment and further evaluation.

[0086] Step 4: Use the student model to evaluate the test set, calculate various performance indicators, analyze misclassification and model focus areas through confusion matrix and Grad-CAM, deploy the student model on the server and generate performance evaluation reports regularly. Figure 5 As shown:

[0087] Step 4.1: Use the student model to predict the test set and calculate the evaluation indicators: Accuracy, F1 Score, and Loss.

[0088] Step 4.2: Use the confusion matrix to identify highly misclassified categories and calculate the recall rate of each category. Formula: Among them, TP i is the number of correctly predicted positive samples in the i-th category, FN i is the actual number of positive samples predicted as negative in the i-th category.

[0089] Step 4.3: Use Grad-CAM to visualize the model's attention area and calculate the Intersection over Union (IoU) ratio between the highlighted area in the Grad-CAM heatmap and the target ground truth area. If the IoU is high, it means that the model is paying attention to the correct area. Otherwise, it means that the model is paying attention to other areas incorrectly. The formula is as follows: The Grad-CAM focus area is the highlighted area of ​​the heat map generated by Grad-CAM, and the real area is the bounding box of the real object.

[0090] Step 4.4: Deploy the student model to the cloud server and expose it as a REST API, allowing other applications to perform inference through network requests.

[0091] Step 4.5: Use the performance monitoring tool Prometheus to track the model’s inference speed, latency, and resource consumption.

[0092] Step 4.6: Generate student model performance evaluation reports regularly to summarize the performance of the student model in the production environment, identify potential problems and propose optimization directions.

[0093] The present invention can be combined with a computer system to form a real-time intelligent target detection device based on dual-stream perception and endogenous knowledge distillation. The device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. The computer program contains code that implements the aforementioned steps. When loaded into the processor, the program automatically executes the dual-stream perception module technology and endogenous knowledge distillation technology to process and optimize the video.

[0094] Experimental data:

[0095] The experimental dataset uses a self-made dataset, which contains 5 categories: helmet, safety rope, person, head, and truck. The dataset is collected from different scenes of the same factory and real-time monitoring under different weather conditions. There are 5000 key frame samples and 48472 labels. Among them, 4000 key frame samples are training set, 500 key frame samples are test set, and 500 key frame samples are validation set. Some images in the dataset are as follows Figure 7 shown.

[0096] Comparative experiment:

[0097] The study compared multiple object detection models, including MobileNetV3, GhostNet, ShuffleNetV2, YOLOv10, and the proposed SSD-DSKD model. (See Table 1.) The experimental results show that the SSD-DSKD model outperformed across various evaluation metrics. SSD-DSKD achieved high scores in precision, recall, and mAP@50%, reaching 91.6%, 88.6%, and 91.1%, respectively. Compared to other models, SSD-DSKD maintained a moderate number of parameters and GFLOPs, at 4,256,017 and 6.4, respectively, while maintaining high accuracy.

[0098] Table 1 Comparison of SSD-DSKD with other algorithms

[0099]

[0100] The above embodiments are intended only to illustrate the technical concepts and features of the present invention. Their purpose is to enable those skilled in the art to understand the contents of the present invention and implement them accordingly. They are not intended to limit the scope of protection of the present invention. Any equivalent changes or modifications made in accordance with the spirit of the present invention are intended to be covered by the scope of protection of the present invention.

Claims

1. A real-time intelligent target detection method based on dual-stream perception and endogenous knowledge distillation, characterized by: The steps include: Step 1: Obtain video frame data in the target scene, perform data preprocessing, enhancement, and annotation, and divide it into video dataset G, verification dataset G_val, and test dataset G_test in a ratio of 8:1:1; Step 2: Use the dual-stream perception module to extract the fusion feature map F conv , used as the input of the SSD detection model for training, and the performance is evaluated on the validation dataset. The verified SSD detection model is used as the teacher model; the dual-stream perception module consists of a static stream and a dynamic stream, which respectively capture spatial static features and temporal dynamic change features and fuse the spatial static features and temporal dynamic change features; Step 3: Based on the teacher model, the student model is trained using the knowledge distillation method of temperature decay. Soft labels are generated for the logits output by the teacher model to fit the teacher model's prediction distribution. An attention alignment mechanism is introduced to narrow the differences in feature focus areas between the two models. A weighted total loss function is constructed using a combination of cross-entropy distillation loss and attention loss. The temperature parameter is dynamically adjusted based on the convergence rate during training. Finally, the student model is determined through performance verification. Step 4: Use the student model to evaluate the test dataset, analyze the misclassification and model attention areas through the confusion matrix and Grad-CAM, deploy the student model on the server and generate performance evaluation reports regularly.

2. The real-time intelligent target detection method based on dual-stream perception and endogenous knowledge distillation according to claim 1 is characterized in that: The specific method of step 1 is: Step 1.1: Collect dynamic video frames of various targets in the target scene. Ensure that the collected data covers different weather and lighting conditions, and that the video frames contain a variety of target categories and cover different behavior patterns from static to fast-moving. Step 1.2: Normalize the captured original video frames to a uniform video resolution of 1080p and normalize the pixel value range to [0, 1]. Step 1.3: Apply inter-frame data augmentation technology to rotate each video frame at random angles ranging from -30 degrees to 30 degrees to simulate different viewing angles; Step 1.4: Use Gaussian filtering to remove noise and artifacts in the video. Step 1.5: Use the video annotation tool CVAT to manually annotate key frames and automatically annotate non-key frames for dynamic objects using interpolation technology; Step 1.6: Divide the processed dataset into training set G, validation set G_val, and test set G_test in a ratio of 8:1:

1.

3. The real-time intelligent target detection method based on dual-stream perception and endogenous knowledge distillation according to claim 1 is characterized in that: The specific method of the dual-stream perception module in step 2 is: First, the pre-trained ResNet-50 network is used to train the continuous frames F t and F t-1 Extract static features P t and P t-1 , and calculate the dynamic characteristics D by difference t =P t -P t-1 , threshold judgment is performed on the amplitude of dynamic features to filter out frames containing significant dynamic events; For salient frames, P t With D t Perform splicing and fusion to generate the original fusion feature map F concat , and then through the 1×1 convolution operation, the semantically optimized fusion feature map F is output refined , and then obtain the regularized feature map F through Batch Normalization conv .

4. The real-time intelligent target detection method based on dual-stream perception and endogenous knowledge distillation according to claim 3 is characterized in that: Step 2 performs a performance test on the SSD detection model on the validation set G_val. If the accuracy Acc DFP ≥90% and F1 DFP If the performance reaches ≥90%, the performance meets the expected standard, and the SSD candidate model is used as the teacher model for subsequent knowledge distillation. Otherwise, the number of output channels is first increased to 512 channels, and the model is re-optimized. If the performance still does not meet the standard after optimization, the number of output channels is reduced to 128 channels, and the model is optimized again. If the performance still does not meet the standard after two optimizations, the adjustment is stopped, and the one with the best performance on the validation set is selected from all trained SSD detection models as the teacher model to enter the subsequent knowledge distillation stage.

5. The real-time intelligent target detection method based on dual-stream perception and endogenous knowledge distillation according to claim 1 is characterized in that: In step 3, both the student model and the teacher model are based on the SSD detection model. The structure of the student model is consistent with that of the teacher model, but its parameters are generated by knowledge distillation training, as follows: Step 3.1: Set the initial temperature T initial , ranging from 1 to 5, initially set to 1, and introduce the temperature control variable T current , used to control the smoothness of the teacher model output, T current Initialized to T initial , dynamically adjusted with epoch during training, setting the change rate threshold η and loss threshold u; Step 3.2: During the training process, gradually reduce the temperature T using the exponential decay formula current ,Dynamically adjust the temperature to optimize the output smoothness of the teacher model.,Exponential decay formula: Where T initial is the initial temperature, λ T is the decay rate, epoch is the current training round; Step 3.3: Use the temperature parameter T obtained in step 3.2 current , generate soft labels for the teacher model formula: in, is the logits output of the teacher model, Sum the indices of all categories for normalization to ensure that the sum of the probabilities of all categories is 1; Step 3.4: Use the temperature parameter T obtained in step 3.2 current , generates the predicted probability distribution of the student model currently being trained formula: in, is the logits output of the student model currently being trained, Sum the indices for all categories; Step 3.5: Calculate the cross entropy distillation loss between the teacher model and the student model currently in training: C is the total number of target detection categories; Step 3.6: Calculate the rate of change R of training loss every 10 epochs. Formula: Among them, L current is the average loss of the current 10 rounds, L previous is the average loss of the first 10 rounds; Step 3.7: If the rate of change R is less than or equal to the threshold η, the training is stable and we proceed to step 3.

8. Otherwise, the training is still converging and we proceed to step 3.6 for the next 10 epochs of training and loss calculation. Step 3.8: Introduce the attention mechanism into the student model currently being trained and calculate the attention loss: Among them, A teacher (x,y) is the attention value of the teacher model at the feature map position (x,y), A student (x, y) is the attention value of the student model in the current training at the feature map position (x, y), H and W are the height and width of the feature map; Step 3.9: Define the weighted total loss function: L total =L KD +ρ·L attention , where L KD is the cross entropy distillation loss in step 3.5, ρ is the hyperparameter that weighs the attention loss, and L attention is the attention loss in step 3.8, if L total If the loss is less than or equal to the loss threshold u, the training is completed and step 3.10 is executed. Otherwise, step 3.2 is executed again and the training optimization is performed.

6. The real-time intelligent target detection method based on dual-stream perception and endogenous knowledge distillation according to claim 5 is characterized in that: When evaluating the performance of the student model in the current training on the validation set G_val, if the accuracy Acc stu ≥90% and F1 stu If the value is ≥90%, the verification is completed and the student model currently being trained is used as the final student model; Otherwise, adjust the initial temperature T initial , retrain, and cycle optimization training for up to 3 times. If the optimization still fails to meet the standard after three times, stop adjusting and select the one with the best performance on the validation set from all trained student models as the final student model for subsequent deployment and further evaluation.

7. The real-time intelligent target detection method based on dual-stream perception and endogenous knowledge distillation according to claim 1 is characterized in that: The specific method of step 4 is: Step 4.1: Use the student model to predict the test data set and calculate the evaluation indicators: Accuracy, F1 Score, and Loss. Step 4.2: Use the confusion matrix to identify highly misclassified categories and calculate the recall rate of each category. Formula: Among them, TP i is the number of correctly predicted positive samples in the i-th category, FN i is the actual number of positive samples predicted as negative in the i-th category; Step 4.3: Use Grad-CAM to visualize the model's attention area and calculate the Intersection over Union (IoU) ratio between the highlighted area in the Grad-CAM heatmap and the target ground truth area. If the IoU is high, it means that the model is paying attention to the correct area. Otherwise, it means that the model is paying attention to other areas incorrectly. The formula is as follows: The Grad-CAM focus area is the highlighted area of ​​the heat map generated by Grad-CAM, and the real area is the bounding box of the real object; Step 4.4: Deploy the student model to the cloud server and expose it as a REST API, allowing other applications to perform inference through network requests; Step 4.5: Use the performance monitoring tool Prometheus to track the model's inference speed, latency, and resource consumption. Step 4.6: Generate student model performance evaluation reports regularly.

8. A real-time intelligent target detection device based on dual-stream perception and endogenous knowledge distillation, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the computer program is loaded into a processor, the real-time intelligent target detection method based on dual-stream perception and endogenous knowledge distillation according to any one of claims 1 to 7 is implemented.

Images