A knowledge distillation based pedestrian and vehicle detection method and system

Abstract

The application provides a pedestrian and vehicle detection method and system based on knowledge distillation, comprising the following steps: constructing a real foggy traffic dataset; performing model optimization based on YOLOv13, introducing a double-domain feature interaction module in a backbone network, introducing a transverse information compensation module at the connection of the backbone network and a neck network, and replacing an up-sampling module of the neck network with an attention up-sampling module to obtain an improved YOLOv13 model; and adopting an MGD-Logit joint real distillation method, taking the improved YOLOv13L as a teacher model, taking a lightweight YOLOv13n as a student model, and performing distillation training through feature layer loss constraint and Logit constraint of result labels. The application can solve the technical problem that the precision, lightweight and real scene generalization capability of a target detection model in a foggy environment are difficult to be considered in the prior art.

Description

Technical Field

[0001] This invention relates to the field of autonomous driving environmental perception technology, specifically to a pedestrian and vehicle detection method and system based on knowledge distillation. Background Technology

[0002] In recent years, with the rapid development of autonomous driving and intelligent transportation systems, the ability to perceive complex environments has become crucial for ensuring driving safety. As the core of perception tasks, pedestrian and vehicle target detection faces significant challenges in low-visibility weather. Tiny particles suspended in the air during foggy weather scatter and absorb light, leading to a significant reduction in road image contrast, blurred target edge information, and even partial occlusion. This severely reduces the recognition accuracy of conventional target detection algorithms.

[0003] To address the low accuracy of fog detection, existing technologies typically focus on two aspects: network structure optimization and data augmentation. On one hand, more complex convolutional structures or feature enhancement modules are introduced into the detection network to improve the model's feature extraction capabilities in low-visibility environments. However, these methods often rely on deeper network layers and a larger number of parameters, leading to a sharp increase in computational complexity and making real-time deployment difficult on in-vehicle embedded devices or edge platforms with limited roadside computing resources. On the other hand, due to the high cost and difficulty of acquiring real foggy traffic images, and the limited number of publicly available datasets, some technologies use foggy data synthesized from clear images for training and validation. However, synthetic foggy images deviate from real foggy scenes in terms of imaging mechanisms and degradation distribution characteristics, resulting in significantly insufficient generalization ability of the trained models in real-world complex scenarios.

[0004] Furthermore, although knowledge distillation techniques have been gradually introduced to balance model accuracy and lightweight design, the application of existing distillation methods in foggy scenarios is still in the exploratory stage. They typically perform loose knowledge transfer only at a single feature layer or output layer, failing to fully consider the collaborative degradation of high-level semantics and low-level spatial details caused by foggy degradation, thus making it difficult to maximize the performance potential of lightweight student models.

[0005] Therefore, how to construct a target detection method that can both ensure the edge deployment efficiency and real-time performance of lightweight models, and significantly improve their detection accuracy and generalization ability in real foggy and normal weather environments, is an urgent problem to be solved in this field. Summary of the Invention

[0006] To address the shortcomings of existing technologies, this invention proposes a pedestrian and vehicle detection method and system based on knowledge distillation, in order to solve the technical problem that existing technologies struggle to balance the accuracy, lightweight design, and real-world generalization capabilities of target detection models in foggy environments.

[0007] The technical solution adopted in this invention is as follows: A traffic dataset containing real foggy images from multiple time periods is constructed and pedestrian and vehicle images are labeled; the YOLOv13L backbone and neck network are optimized in multiple dimensions to serve as the teacher model. The optimization process includes introducing a dual-domain feature interaction module into the backbone network to extract spatial details and global frequency domain features, introducing a multi-path lateral information compensation module at the connection between the backbone and neck networks, and introducing an attention upsampling module in the neck network; a joint distillation network is constructed using lightweight YOLOv13n as the student model. During the training phase, a distillation-constrained intermediate feature layer is generated using a mask, and a Logit divergence constraint output layer controlled by a temperature coefficient is combined with the output layer. The student model is trained using a joint loss function.

[0008] In a first feasible approach, a pedestrian and vehicle detection method based on knowledge distillation is provided, comprising the following steps: Construct traffic datasets for multiple time periods under foggy and normal weather conditions, and label pedestrian and vehicle targets within them; A joint distillation network was constructed using the optimized YOLOv13L model as the teacher model and the lightweight YOLOv13n model as the student model. The traffic dataset is input into the joint distillation network, and the student model is jointly trained using feature distillation loss and Logit distillation loss to obtain the trained student model. The trained student model is used to detect pedestrians and vehicles in the images to be detected.

[0009] Furthermore, the teacher model is optimized based on the YOLOv13L model, and the optimization steps include: A dual-domain feature interaction module is introduced into the backbone network of the teacher model, a lateral information compensation module is introduced at the connection between the backbone network and the neck network, and the upsampling module in the neck network of the teacher model is replaced with an attention upsampling module. A dual-domain feature interaction module is introduced into the backbone network of the teacher model, including: The overall input features F of the dual-domain interaction module are initially transformed through a convolutional layer, and the transformed features are divided into n branches according to the channel dimension. Use one of the branches as input features Multiple spatial domain-frequency domain feature interaction modules are sequentially input to obtain the first branch output features; The remaining n-1 branches are input into the corresponding spatial domain-frequency domain feature interaction modules for processing to obtain the branch output features; The first branch output feature is concatenated with the branch output feature to obtain the concatenated feature; The concatenated features are integrated through convolutional layers, and the processed dual-domain interactive features are output as input features for the next layer in the backbone network. Among them, the input features of each spatial domain-frequency domain feature interaction module are individual interaction module input features. This corresponds to the branch channel feature, or the output feature of the previous spatial domain-frequency domain feature interaction module in the serial link; Input features for the single interactive module The processing steps include: Input features for the single interactive module Normalization is performed to obtain the normalized feature representation. ; The spatial detail feature extraction branch utilizes the input features of the single interaction module. and normalized feature representation Extract and output spatial domain branch features. ; The frequency domain feature extraction branch is based on the normalized feature representation. Extract and output frequency domain features. ; The spatial domain branching features With the frequency domain output characteristics Perform channel splicing, and through Convolutional processes are used to perform interactive fusion, resulting in the output features of the spatial domain-frequency domain feature interaction module.

[0010] Furthermore, the spatial detail feature extraction branch is based on the input features of a single interaction module. and normalized feature representation Spatial domain branch features were extracted. Specifically, it includes: The first path directly transmits the input features of the single interactive module. ; The second pathway activates the normalized features through an activation function. Nonlinear processing is performed to obtain the global feature representation in the spatial domain. Simultaneously, spatial detail features of the image are extracted using a residual structure centered on bidirectional depthwise separable convolution (DEConv). ; Regarding the features After convolution processing, and with the features Pixel-by-pixel multiplication yields the fused features. The calculation formula is:

[0011] In the formula, Indicates fusion characteristics, express Convolution operation, This indicates pixel-by-pixel multiplication; The fusion features Input features with the single interactive module The pixels are added one by one, and the final output is the spatial domain branch feature. :

[0012] In the formula, Represents the branching characteristics of the spatial domain. This represents the input features of a single interactive module, where m represents the downsampling level of the module. This indicates pixel-by-pixel addition. This indicates the fusion feature.

[0013] Furthermore, the frequency domain feature extraction branch is based on the normalized feature representation. Extracting frequency domain output features Specifically, it includes: The first path will normalize the features. Features are obtained through activation function processing. ; The second path compensates the amplitude spectrum of the image in the frequency domain through a spectrum amplitude-phase integration module, and solves for the phase spectrum corresponding to the compensated amplitude spectrum to obtain the output features. ; The feature With the output features Pixel-by-pixel multiplication yields the frequency domain output features. :

[0014] The specific implementation of the spectrum amplitude-phase integration module includes: Features The frequency domain features are converted using Fourier transform and then decomposed into amplitude spectra. and phase spectrum ; pass The convolution process is used to recover the amplitude spectrum caused by fog degradation, resulting in the recovered amplitude spectrum. ; Calculate the amplitude spectrum residual between the restored amplitude spectrum and the original amplitude spectrum; An attention map is obtained by processing the amplitude spectrum residuals using an attention mechanism. The corresponding phase spectrum is then calculated based on the attention map and the original phase spectrum. ; The recovered amplitude spectrum and the solved phase spectrum are combined to express the real and imaginary parts, and then converted back to the spatial domain using inverse Fourier transform.

[0015] Furthermore, the lateral information compensation module includes: Utilizing feature maps from the backbone network The convolutional layer performs channel adjustment to align the feature dimensions and generate an initial feature representation. ; Represent the initial features The data is fed into the feature compensation branch for multi-path processing. Path 1: To use Convolutional layers increase the receptive field and output features. ; Path 2: For Convolution and normalization calculations are performed sequentially through two parallel and serial sub-paths, followed by pixel-by-pixel multiplication and convolution to output local correlation features. ; Path 3: Compression The size is determined, and channel-enhanced features are output through adaptive learning of channel weights. ; Path 4: Directly connect features ; The output features of path 1, path 2, path 3, and path 4 are fused based on adaptive Softmax normalization weights to obtain the output features of the feature compensation branch. .

[0016] Furthermore, the lateral information compensation module also introduces a residual fusion mechanism, including: The feature compensation branch outputs features Combined with residual fusion features calculated using the following formula The pixels are added one by one to obtain the final output lateral compensation feature. :

[0017]

[0018] In the formula, express Convolutional layer operations.

[0019] Furthermore, the attention upsampling module includes: Use 3 Convolution upsamples the input features of the attention module These are mapped to query Q, key K, and value V, respectively. Perform matrix transformations on Q, K, and V to obtain the transformed matrices. , and and to and Perform a dot product operation to obtain the aggregate weight. :

[0020] Using the Softmax function to aggregate weights Perform normalization and compare it with Perform dot product and matrix transformation to obtain feature Y containing rich semantic information; Perform on the feature Y Convolution operations expand the number of channels, which in turn expands the input features. The pixels are added one by one, and bilinear interpolation is used to obtain the final output features. :

[0021] In the formula, This represents the bilinear interpolation operation. It is a convolution operator.

[0022] Furthermore, the student model is jointly trained using feature distillation loss and Logit distillation loss, specifically including: Selecting the corresponding feature layer in the joint distillation network as the distillation layer, when the channel dimensions of the corresponding distillation layer are inconsistent between the teacher model and the student model, utilizes... The convolutional layer performs a linear mapping alignment on the feature map of the student model to make its channel number consistent with that of the teacher model; Mask-generated distillation is employed as a feature distillation strategy. This involves applying a random masking operation to the feature map of the student model to obscure a portion of the spatial region. The feature distillation loss function... as follows:

[0023] In the formula, This represents the feature map of the student model after channel alignment. This represents the feature map of the teacher model, where M is a randomly generated mask matrix. This indicates element-wise multiplication. This indicates the generation module used for feature recovery. This represents the weighting of distillation loss.

[0024] Furthermore, the method for constructing the Logit distillation loss and the overall loss function used in joint training includes: A temperature coefficient T is introduced to perform Softmax normalization on the output Logits of the teacher and student models. Knowledge transfer is achieved by minimizing the KL divergence between the predicted probability distributions of the two models, and a Logit distillation loss function is constructed. :

[0025] In the formula, This represents the predicted probability distribution of the teacher model. This represents the predicted probability distribution of the student model, where T is the distillation temperature coefficient. The overall loss function of joint optimization The structure is as follows:

[0026] In the formula, This represents the loss function for the original object detection task, including classification loss, bounding box regression loss, and object confidence loss. and These are the weighting coefficients for characteristic distillation loss and Logit distillation loss, respectively.

[0027] In conjunction with the first feasible approach, a second feasible approach provides a pedestrian and vehicle detection system based on knowledge distillation, comprising: The dataset construction and annotation module is used to construct traffic datasets for multiple time periods under foggy and normal weather conditions, and to annotate pedestrian and vehicle targets within them; The joint distillation network construction module is used to construct a joint distillation network using the optimized YOLOv13L model as the teacher model and the lightweight YOLOv13n model as the student model. The student model joint training module is used to input the traffic dataset into the joint distillation network and jointly train the student model through feature distillation loss and Logit distillation loss to obtain the trained student model. The object detection module is used to detect pedestrians and vehicles in the image to be detected using the trained student model.

[0028] As can be seen from the above technical solution, the beneficial technical effects of the present invention are as follows: 1. By deeply optimizing the teacher model, utilizing dual-domain feature interaction, lateral multi-path information compensation, and attention upsampling, the spatial details and frequency domain structure of the image are comprehensively reconstructed. In the knowledge transfer stage, the collaborative constraints of feature layer mask generation distillation and output layer Logit soft label distillation are used to significantly improve the detection performance of the lightweight student model for distant and small targets in foggy weather without increasing any additional parameters or computational overhead.

[0029] 2. A realistic foggy traffic dataset covering multiple time periods, including daytime, evening, and nighttime, was constructed, fundamentally overcoming the domain bias problem caused by using synthetic datasets. The trained student model maintains low computational load and meets the real-time deployment requirements of vehicle-mounted or vehicle-embedded devices, while exhibiting extremely high adaptability and robustness to the changing environments of real fog and normal weather. Attached Figure Description

[0030] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the accompanying drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. In all the drawings, similar elements or parts are generally identified by similar reference numerals. In the drawings, the elements or parts are not necessarily drawn to scale.

[0031] Figure 1 This is a flowchart of the method in Embodiment 1 of the present invention; Figure 2 This is a structural diagram of YOLOv13 according to Embodiment 1 of the present invention; Figure 3 This is a structural diagram of DDFIM according to Embodiment 1 of the present invention; Figure 4 This is a structural diagram of the SFFIM according to Embodiment 1 of the present invention; Figure 5 This is a structural diagram of the LICM according to Embodiment 1 of the present invention; Figure 6 This is a structural diagram of the LCUM in Embodiment 1 of the present invention; Figure 7 This is a structural diagram of the joint knowledge distillation of Embodiment 1 of the present invention; Figure 8 This is a system structure diagram of Embodiment 2 of the present invention; Figure label: 2- A pedestrian and vehicle detection system based on knowledge distillation, 21- Dataset construction and annotation module, 22- Joint distillation network construction module, 23- Student model joint training module, 24- Object detection module. Detailed Implementation

[0032] The embodiments of the technical solution of the present invention will now be described in detail with reference to the accompanying drawings. These embodiments are merely illustrative of the technical solution of the present invention and are therefore intended to limit the scope of protection of the present invention.

[0033] It should be noted that, unless otherwise stated, the technical or scientific terms used in this application should have the ordinary meaning as understood by one of ordinary skill in the art to which this invention pertains.

[0034] Example 1

[0035] This embodiment provides a pedestrian and vehicle detection method based on knowledge distillation. The working principle of Embodiment 1 is explained in detail below: This embodiment provides an end-to-end pedestrian and vehicle detection method based on knowledge distillation under foggy and normal weather conditions. The flowchart of this method is shown below. Figure 1 As shown, it includes the following steps: S1. To address the issue of insufficient real-world traffic scene detection datasets, in this embodiment, a real-world traffic scene dataset was first collected and constructed from road scenes in foggy conditions.

[0036] This dataset encompasses foggy images from three different time periods: daytime, dusk, and nighttime. It includes three categories of targets: pedestrians, motor vehicles, and non-motorized vehicles. Motor vehicles include cars and vans, while non-motorized vehicles include motorcycles and bicycles. The entire dataset contains 3000 images with 21492 labeled target instances. The construction of this real-world foggy traffic scene detection dataset provides a reliable experimental foundation for subsequent research and evaluation of detection algorithms.

[0037] S2. Model optimization based on YOLOv13.

[0038] The optimized overall structure of YOLOv13 is as follows: Figure 2 As shown, it mainly consists of a backbone network, a neck network, and a detection head. The core modules, operators, and operation definitions involved are as follows: Conv (Convolutional Module): A standard convolutional layer, consisting of two-dimensional convolutional layers, batch normalization layers, and the SiLU activation function, used to achieve basic local feature extraction, dimensionality reduction, and channel number conversion.

[0039] Concat (channel splicing): Channel-dimensional concatenation operation, used to splice and fuse feature maps from different network paths with the same resolution on the channel axis, realizing the integration of multi-scale features or cross-layer features.

[0040] DS-C3k2 (Depthly Separable / Two-Stage C3k2 Module): Employs a two-stage cascaded structure combined with a depthwise separable convolution feature extraction bottleneck to efficiently extract rich local texture and edge features while maintaining a lightweight model.

[0041] DDFI-C2f (Dual-Domain Interaction C2f Module): Based on the C2f (Coarse-to-Fine) structure, the dual-domain feature interaction module (DDFIM) of this embodiment is embedded at its bottleneck to realize deep bidirectional interaction and multi-scale transfer of spatial and frequency domain features in the backbone network.

[0042] DSConv (Depthwise Separable Convolution): Decomposes standard convolution into channel-wise convolution and pointwise convolution, greatly reducing the amount of computation and parameters with minimal reduction in accuracy, thus ensuring the deployment efficiency of edge devices.

[0043] A2C2f (Attention-Guided C2f Module): Introduces an attention mechanism into the traditional C2f structure, enabling the network to adaptively focus on foreground targets such as pedestrians and vehicles during feature flow, effectively suppressing background interference in foggy weather.

[0044] HyperACE (Hypergraph Attention Context Engine): A hypergraph relationship modeling mechanism that focuses on mining high-order correlations between multi-scale feature nodes globally, establishing semantic connections across spatial locations and scales, thereby capturing a wide range of long-distance dependencies.

[0045] Detect (Detection Head): The target detection output head adopts a decoupled head structure, which predicts the bounding box regression coordinates, confidence scores, and class probabilities separately, and outputs the final pedestrian and vehicle detection results.

[0046] This embodiment first introduces a dual-domain feature interaction module (DDFIM) into the backbone network, the structure of which is as follows: Figure 3 As shown.

[0047] The overall input feature F of the dual-domain interaction module is first transformed through a convolutional layer, and then the feature is divided into n branch channel features according to the channel dimension.

[0048] In this process, the feature of one branch channel is sequentially processed through multiple Spatial-Frequency Feature Interaction Modules (SFFIM) to enhance the feature representation and obtain the first branch output feature; the features of the other n-1 branch channels are processed by their respective individual SFFIMs to obtain the remaining n-1 branch output features.

[0049] Finally, the output features of the first branch are concatenated with the output features of the remaining n-1 branches to fuse the feature information of multiple branches; the concatenated fused features are then processed by a convolutional layer to complete feature integration, and the processed dual-domain interactive features are output as the input features of the next layer in the backbone network.

[0050] The core component of the dual-domain feature interaction module is SFFIM, with the following structure: Figure 4 As shown. This module first inputs features to a single interactive module. (The corresponding features are the branch channel features or the output features of the previous SFFIM in the serial link.) Normalization is performed to obtain the normalized feature representation. Next, the spatial domain features and frequency domain features of the image are extracted using the DPFFM branch and the FFEM branch, respectively.

[0051] Specifically, in the spatial domain feature extraction part, local detail features are extracted using DPFFM to recover complex textures and edge structures in the image, improving the model's sensitivity to object contours and directional features. The core of the frequency domain feature extraction part is to first use Fourier transform to extract the image's amplitude and phase spectra. The amplitude spectrum contains detailed information such as image contrast and texture, while the phase spectrum contains image structural information. The amplitude spectrum is then used to supplement the phase spectrum to obtain more accurate global structural features. Next, an inverse Fourier transform is used to transform the compensated frequency domain features back to the spatial domain. Finally, feature concatenation and... Convolutional processing interacts and fuses the features output by DPFFM and FFEM to obtain the output of SFFIM. SFFIM achieves sensitivity to spatial domain edge details and orientation, as well as full utilization of the position and arrangement of objects in the frequency domain, through the interaction of dual-domain features, and improves the generalization and robustness of feature representation.

[0052] DPFFM is primarily used for extracting spatial domain features from images. To achieve simultaneous extraction of global and local detail features in the image spatial domain, DPFFM employs a two-path structure.

[0053] The first path directly utilizes a single interactive module to input features. ; The second path outputs by adding two parts pixel by pixel. The first part uses the SiLU activation function to apply the normalized features. Nonlinear processing is performed to obtain the global feature representation in the spatial domain. The second part utilizes the residual structure centered on DEConv to extract spatial detail features from the image. ,pass After convolution, it is combined with the first part of the output. Pixel-by-pixel multiplication yields the fused features. .

[0054] Finally, the fusion features The features are added pixel-by-pixel to the original input features F_m, and the final output is the spatial domain branch features. .

[0055] The mathematical expression of this process is as follows:

[0056]

[0057] In the formula: m represents the m-th downsampling code of the coding layer, and its value is 1 or 2. This indicates pixel-by-pixel multiplication. This indicates that each pixel is added together.

[0058] The first path in the DPFFM module aims to directly pass the input features from the SFFIM module, alleviating the gradient vanishing problem in deep networks and ensuring that the original feature information can propagate forward. The second path focuses on learning detailed features, partly by extracting fine spatial details and texture information through a residual structure centered on DEConv, and then combining it with... Pixel-wise multiplication is performed, and the gating properties of SiLU are used to adaptively filter features. Finally, pixel-wise addition is performed to fuse the two paths, thereby achieving coordinated optimization of global and detailed information in the spatial domain.

[0059] FFEM is primarily used for extracting frequency domain features from images, and it also employs a two-path structure. The first path extracts the features... Features are obtained through the activation function SiLU. The second approach obtains the output characteristics through the Amplitude-Phase Integration Module (APIM). Then, the output features of the two paths are... and Pixel-by-pixel multiplication yields the output features in the frequency domain. The specific formula is shown in Formula 3:

[0060] Different spatial locations have varying degrees of dependence on frequency domain features; the two paths are multiplied pixel by pixel. The frequency domain features are locally modulated using the learned spatial weights.

[0061] As discussed earlier, image degradation caused by haze is mainly reflected in the amplitude spectrum, while the phase spectrum of hazy and hazy images shows little difference. Therefore, APIM primarily compensates for the amplitude spectrum and uses the compensated amplitude spectrum to solve for the corresponding phase spectrum, in order to obtain more accurate global structural features. The specific structure is as follows: Figure 3 As shown. APIM first normalizes the spatial domain features. Converted to frequency domain features via Fourier transform :

[0062]

[0063] In the formula, Represents the amplitude spectrum. The phase spectrum is represented. Since the amplitude spectrum in the frequency domain of a foggy image is severely degraded compared to a fog-free image, it needs to be compensated. However, to obtain more accurate global structural features in the frequency domain, it is still necessary to accurately solve for the phase spectrum corresponding to the compensated amplitude spectrum.

[0064] Specifically, firstly adopt Convolution recovery pairs The recovered amplitude spectrum is obtained through processing. :

[0065] Next, the recovered amplitude spectrum was analyzed. and the original amplitude spectrum By performing element-wise subtraction, the amplitude spectrum residuals are obtained. :

[0066] In order to extract and Significant differences, using The attention mechanism processes the amplitude residuals to obtain its attention map. And solve it based on this. Corresponding phase spectrum :

[0067]

[0068] In the formula, GAP represents global average pooling. After the amplitude spectrum and phase spectrum are recovered, they are then converted into their corresponding real and imaginary parts:

[0069]

[0070] Finally, the frequency domain features are transformed back to the spatial domain using an inverse Fourier transform to obtain the feature representation. :

[0071] Secondly, a Lateral Information Compensation Module (LICM) is introduced at the connection between the backbone network and the neck network, such as... Figure 5As shown, the core of this module is to introduce a multi-path feature compensation mechanism on top of traditional lateral connections, enabling deep semantic features and shallow spatial information to form a complementary relationship during the fusion process, thereby compensating for the information loss caused by fog degradation. Unlike traditional lateral connections that only perform feature alignment, LICM uses multi-scale convolutional transformations and residual fusion on the input features, enabling the network to re-enhance spatial structural information and semantic expression capabilities during lateral propagation.

[0072] The LICM module first performs channel adjustment on the feature maps from the backbone network, through... Convolution aligns feature dimensions and generates initial feature representations. Subsequently, The data is fed into the Feature Compensation Branch (FCB) for further processing, which expands the data into four different paths to enhance the final output features.

[0073] Path 1 features use Convolution aims to increase the receptive field (RF) and thus output features. ,in . The calculation formula is shown below. This path is highly advantageous for extracting deep semantic information:

[0074] In the formula, express Convolution operation.

[0075] The main purpose of path 2 is to capture the intrinsic correlation (IC) between local information in the features, and it has two sub-paths. Sub-path 1... application Convolution; Subpath 2 is then... proceed in sequence convolution, Convolution and Sigmoid weight normalization are performed. Then, the output features of the two sub-paths are multiplied element-wise, and finally... Convolution output features ,in . The calculation formula is as follows:

[0076] In the formula, This represents element-wise product.

[0077] Path 3 focuses on Channel Enhancement (CE). First, it incorporates features... Size compressed to Among them, the channel Then for Initialize the learning weights for each channel, and then combine them with the features. Perform multiplication and output features ,in The adaptive learning of weights in this feature is achieved through backpropagation of the network. The calculation formula is as follows:

[0078] In the formula, It is a scale compression operation implemented through global average pooling. This is the channel weight initialization operation.

[0079] Path 4 is primarily designed to prevent network degradation caused by deep semantic information extraction and channel information enhancement in other paths. This path performs no operations and directly connects to the features. And the final fusion port.

[0080] The feature compensation branch ultimately outputs features. This is a fusion feature based on the different weights of these four paths. These weights are determined by Softmax normalization to ensure that the contributions of each path are combined proportionally. The fusion and normalization formulas are as follows:

[0081]

[0082] In the formula, It initializes the weights. To normalize the weights, adaptive weighting and Softmax normalization can enhance the stability of dynamic fusion of multiple input features and accelerate the network convergence speed.

[0083] After obtaining the enhanced semantic features, the LICM module further introduces a residual fusion mechanism (RFM) to fuse the enhanced features with the original input features, as shown in Equation 18. This design can enhance the deep semantic expressive power while preserving the original spatial information, thereby avoiding the problem of excessive information compression during the deep feature extraction process.

[0084] Ultimately and The pixels are added one by one to obtain the final output lateral compensation feature. As shown in Formula 19:

[0085] In this way, the final output features of the module not only contain semantic information from deep layers but also retain texture and edge information from shallow features, making the complementary relationships between multi-scale features more apparent. From the perspective of the overall network structure, the LICM module and the HyperACE mechanism in YOLOv13 form a synergistic and complementary relationship. HyperACE focuses on mining high-order correlations between feature nodes globally, enabling the network to establish semantic connections across spatial locations and scales, while the LICM module is mainly responsible for compensating for local information and spatial structure during feature propagation, thus providing HyperACE with more stable and information-rich input features. In other words, by enhancing local detail information and multi-scale semantic expression, LICM enables HyperACE to obtain more accurate node representations when constructing high-order relationship graphs, while HyperACE further strengthens the semantic connections between LICM output features through global relationship modeling. The two form a complementary "local compensation-global association" mechanism at the feature expression level, thereby significantly improving the network's object detection capabilities in complex environments.

[0086] Finally, the upsampling module of the neck network is replaced with an attention upsampling module (Lightweight Channel-aware Upsampling Module, LCUM), as follows: Figure 6 As shown, this module combines an attention mechanism with the feature reconstruction process, enabling the network to adaptively adjust the importance of features while restoring spatial resolution, thereby reducing the propagation of irrelevant information and highlighting key semantic features.

[0087] First, using 3 Convolution transforms the input features of LCUM The mapping is divided into three features, which respectively represent the query Q, key K, and value V in the attention mechanism, where , and These represent the depth, height, and width of the input LCUM feature, respectively:

[0088]

[0089]

[0090] In the formula, , , , All are learnable mapping weights. Then, matrix transformations are performed on Q, K, and V to obtain:

[0091]

[0092]

[0093] right and Perform a dot product operation to obtain the aggregate weight. This represents the importance of each position in the feature:

[0094] Then, the Softmax function is used to aggregate the weights. Perform normalization and combine it with features The dot product operation is performed, followed by matrix transformation to obtain feature Y. Due to the attention mechanism, feature Y contains rich semantic information.

[0095] Finally, perform the following on feature Y: Convolution operations expand the number of channels, connecting them to the input features, i.e., skip-level connections. The pixels are added one by one, and bilinear interpolation is used to obtain the final output features. The calculation formula is as follows:

[0096]

[0097] in, This indicates a bilinear interpolation operation.

[0098] From an overall structural perspective, the LCUM module also effectively collaborates with the HyperACE mechanism. HyperACE obtains global semantic dependencies through hypergraph relationship modeling, while LCUM strengthens these key semantic features during the feature propagation stage, allowing them to be more fully expressed in the multi-scale feature fusion process. In other words, HyperACE is responsible for discovering important feature relationships, while LCUM is responsible for strengthening and propagating these relationships, ensuring consistency across feature maps at different scales. This collaborative mechanism effectively enhances the network's ability to represent target structural information in complex scenes, especially showing significant advantages in detecting distant and small targets in foggy environments.

[0099] S3. In this embodiment, YOLOv13n is selected as the student model, and the improved YOLOv13L in S2 is used as the teacher model for joint knowledge distillation. The distillation structure diagram is shown below. Figure 7 As shown.

[0100] The proposed framework combines feature distillation and Logit distillation, enabling the student model to learn not only the intermediate feature representation capabilities of the teacher model, but also the final decision information of the teacher model, thereby comprehensively improving the detection performance of the student model.

[0101] In terms of feature distillation, this embodiment establishes a feature alignment relationship between the feature pyramid network layers of the teacher model and the student model, and selects the P3, P4, and P5 feature layers in the detection network as distillation layers. These feature layers contain semantic information of targets at different scales. A loss function is used to constrain the feature layers, allowing the student model to continuously learn the feature representations of the teacher model in multi-scale target detection. However, due to differences in network size and channel number design between the teacher and student models, the channel dimensions of their corresponding feature layers are often inconsistent. Therefore, the channel number between the models needs to be aligned before feature distillation.

[0102] This embodiment adopts The convolutional layer linearly maps the feature map of the student model to maintain the same number of channels as the teacher model. Let the feature map of the teacher model's distillation layer be... The feature map corresponding to the student model is ,in and These represent the number of channels in the teacher model and the student model, respectively. When At that time, through a Convolution mapping function The mathematical expression for transforming student characteristics is shown in the following formula:

[0103] in, This is the channel-aligned student feature map. A mapping operation ensures that the student model features and the teacher model features remain consistent in the channel dimension, thus providing a foundation for subsequent feature distillation.

[0104] To further improve the feature distillation effect, this embodiment employs Masked Generative Distillation (MGD) as the feature distillation strategy. The MGD method applies a random masking operation to the student model's feature map, obscuring information in some spatial regions and guiding the student model to generate a complete feature representation of the teacher model using the remaining feature information, thereby achieving knowledge transfer. Compared with traditional direct feature alignment methods, this method enables the student model to learn more structured feature representations, thus improving the model's feature representation ability and generalization ability.

[0105] In addition to the distillation of the intermediate feature layers, this embodiment also introduces a Logit-based distillation method to constrain the model's output layer. During Logit distillation, the student model learns the soft labels output by the teacher model to obtain richer class distribution information. Compared to traditional hard labels, soft labels not only contain information about the correct class but also reflect the relative probability relationships between different classes. To smooth the probability distribution, a temperature coefficient T is introduced during distillation to perform Softmax normalization on the model's output Logits, as shown below:

[0106] in, This represents the Logit value of the model output, where T is the temperature parameter. When T > 1, the output probability distribution will be smoother, thus better reflecting the relative relationships between categories. During training, by minimizing the difference between the predicted probability distributions of the teacher model and the student model, the student model gradually learns the decision-making patterns of the teacher model, thereby improving detection performance.

[0107] The overall loss function consists of three parts: object detection loss, feature distillation loss, and Logit distillation loss, as shown in the following equation:

[0108] in, The original loss function for object detection mainly includes classification loss, bounding box regression loss, and object confidence loss. This represents the characteristic distillation loss based on MGD. This represents the Logit distillation loss based on the output layer probability distribution. and This is the distillation loss weighting coefficient, used to adjust the degree of influence between different loss items.

[0109] In the feature distillation part, the MGD method randomly masks a portion of the student model's feature map, requiring the student model to recover the complete feature representation of the teacher model using the remaining feature information. Its distillation loss function is shown below:

[0110] In the formula, This represents the feature map of the student model after channel alignment. This represents the feature map of the teacher model, where M is a randomly generated mask matrix. For element-wise multiplication, This indicates the generation module used for feature recovery. These are the weighting coefficients for the distillation loss. Using this loss function, the student model needs to generate a complete feature representation close to that of the teacher model based on the unoccluded feature information, thereby achieving effective knowledge transfer.

[0111] In the Logit distillation part, knowledge transfer is achieved by minimizing the KL divergence between the predicted probability distributions of the teacher model and the student model. The loss function is shown in the following equation:

[0112] in, This represents the predicted probability distribution of the teacher model. Let T represent the predicted probability distribution of the student model, and T be the distillation temperature coefficient. This loss function allows the student model's output distribution to gradually approximate the teacher model's, thus learning the teacher model's knowledge of category decision-making.

[0113] By jointly optimizing the aforementioned loss functions, the student model can not only learn supervised information from the object detection task itself during training, but also obtain multi-level knowledge guidance from the teacher model. Feature distillation primarily enhances the student model's feature representation ability, while Logit distillation strengthens its decision-making capabilities. Their synergistic effect allows the lightweight student model to achieve higher detection accuracy and better environmental adaptability while maintaining low computational complexity.

[0114] Example 2

[0115] In conjunction with the method provided in Embodiment 1, this embodiment provides a pedestrian and vehicle detection system 2 based on knowledge distillation, the system structure diagram of which is shown below. Figure 8 As shown, it includes: The dataset construction and annotation module 21 is used to construct traffic datasets for multiple time periods under foggy and normal weather conditions, and to annotate pedestrian and vehicle targets in the datasets. The joint distillation network construction module 22 is used to construct a joint distillation network using the optimized YOLOv13L model as the teacher model and the lightweight YOLOv13n model as the student model. The student model joint training module 23 is used to input the traffic dataset into the joint distillation network and jointly train the student model through feature distillation loss and Logit distillation loss to obtain the trained student model. The target detection module 24 is used to perform target detection of pedestrians and vehicles on the image to be detected using the trained student model.

[0116] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention, and they should all be covered within the scope of the claims and specification of the present invention.

Claims

1. A pedestrian and vehicle detection method based on knowledge distillation, characterized in that, Includes the following steps: Construct traffic datasets for multiple time periods under foggy and normal weather conditions, and label pedestrian and vehicle targets within them; A joint distillation network was constructed using the optimized YOLOv13L model as the teacher model and the lightweight YOLOv13n model as the student model. The traffic dataset is input into the joint distillation network, and the student model is jointly trained using feature distillation loss and Logit distillation loss to obtain the trained student model. The trained student model is used to detect pedestrians and vehicles in the images to be detected.

2. The pedestrian and vehicle detection method based on knowledge distillation according to claim 1, characterized in that, The teacher model is optimized based on the YOLOv13L model, and the optimization steps include: A dual-domain feature interaction module is introduced into the backbone network of the teacher model, a lateral information compensation module is introduced at the connection between the backbone network and the neck network, and the upsampling module in the neck network of the teacher model is replaced with an attention upsampling module. A dual-domain feature interaction module is introduced into the backbone network of the teacher model, including: The overall input features F of the dual-domain interaction module are initially transformed through a convolutional layer, and the transformed features are divided into n branches according to the channel dimension. Use one of the branches as input features Multiple spatial domain-frequency domain feature interaction modules are sequentially input to obtain the first branch output features; The remaining n-1 branches are input into the corresponding spatial domain-frequency domain feature interaction modules for processing to obtain the branch output features; The first branch output feature is concatenated with the branch output feature to obtain the concatenated feature; The concatenated features are integrated through convolutional layers, and the processed dual-domain interactive features are output as input features for the next layer in the backbone network. Among them, the input features of each spatial domain-frequency domain feature interaction module are individual interaction module input features. This corresponds to the branch channel feature, or the output feature of the previous spatial domain-frequency domain feature interaction module in the serial link; Input features for the single interactive module The processing steps include: Input features for the single interactive module Normalization is performed to obtain the normalized feature representation. ; The spatial detail feature extraction branch utilizes the input features of the single interaction module. and normalized feature representation Extract and output spatial domain branch features. ; The frequency domain feature extraction branch is based on the normalized feature representation. Extract and output frequency domain features. ; The spatial domain branching features With the frequency domain output characteristics Perform channel splicing, and through Convolutional processes are used to perform interactive fusion, resulting in the output features of the spatial domain-frequency domain feature interaction module.

3. The pedestrian and vehicle detection method based on knowledge distillation according to claim 2, characterized in that, The spatial detail feature extraction branch is based on features input from a single interactive module. and normalized feature representation Spatial domain branch features were extracted. Specifically, it includes: The first path directly transmits the input features of the single interactive module. ; The second pathway activates the normalized features through an activation function. Nonlinear processing is performed to obtain the global feature representation in the spatial domain. Simultaneously, spatial detail features of the image are extracted using a residual structure centered on bidirectional depthwise separable convolution (DEConv). ; Regarding the features After convolution processing, and with the features Pixel-by-pixel multiplication yields the fused features. The calculation formula is: In the formula, Indicates fusion features, express Convolution operation, This indicates pixel-by-pixel multiplication; The fusion features Input features with the single interactive module The pixels are added one by one, and the final output is the spatial domain branch feature. : In the formula, Represents the branching characteristics of the spatial domain. This represents the input features of a single interactive module, where m represents the downsampling level of the module. This indicates pixel-by-pixel addition. This indicates the fusion feature.

4. The pedestrian and vehicle detection method based on knowledge distillation according to claim 2, characterized in that, The frequency domain feature extraction branch is based on the normalized feature representation. Extracting frequency domain output features Specifically, it includes: The first path will normalize the features. Features are obtained through activation function processing. ; The second path compensates the amplitude spectrum of the image in the frequency domain through a spectrum amplitude-phase integration module, and solves for the phase spectrum corresponding to the compensated amplitude spectrum to obtain the output features. ; The feature With the output features Pixel-by-pixel multiplication yields the frequency domain output features. : The specific implementation of the spectrum amplitude-phase integration module includes: Features The frequency domain features are converted using Fourier transform and then decomposed into amplitude spectra. and phase spectrum ; pass The convolution process is used to recover the amplitude spectrum caused by fog degradation, resulting in the recovered amplitude spectrum. ; Calculate the amplitude spectrum residual between the restored amplitude spectrum and the original amplitude spectrum; An attention map is obtained by processing the amplitude spectrum residuals using an attention mechanism. The corresponding phase spectrum is then calculated based on the attention map and the original phase spectrum. ; The recovered amplitude spectrum and the solved phase spectrum are combined to express the real and imaginary parts, and then converted back to the spatial domain using inverse Fourier transform.

5. The pedestrian and vehicle detection method based on knowledge distillation according to claim 2, characterized in that, The lateral information compensation module includes: Utilizing feature maps from the backbone network The convolutional layer performs channel adjustment to align the feature dimensions and generate an initial feature representation. ; Represent the initial features The data is fed into the feature compensation branch for multi-path processing. Path 1: To use Convolutional layers increase the receptive field and output features. ; Path 2: For Convolution and normalization calculations are performed sequentially through two parallel and serial sub-paths, followed by pixel-by-pixel multiplication and convolution to output local correlation features. ; Path 3: Compression The size is determined, and channel-enhanced features are output through adaptive learning of channel weights. ; Path 4: Directly connect features ; The output features of path 1, path 2, path 3, and path 4 are fused based on adaptive Softmax normalization weights to obtain the output features of the feature compensation branch. .

6. The pedestrian and vehicle detection method based on knowledge distillation according to claim 5, characterized in that, The horizontal information compensation module also introduces a residual fusion mechanism, including: The feature compensation branch outputs features Combined with residual fusion features calculated using the following formula The pixels are added one by one to obtain the final output lateral compensation feature. : In the formula, express Convolutional layer operations.

7. The pedestrian and vehicle detection method based on knowledge distillation according to claim 2, characterized in that, The attention upsampling module includes: Use 3 Convolution upsamples the input features of the attention module These are mapped to query Q, key K, and value V, respectively. Perform matrix transformations on Q, K, and V to obtain the transformed matrices. , and and to and Perform a dot product operation to obtain the aggregate weight. : Using the Softmax function to aggregate weights Perform normalization and compare it with Perform dot product and matrix transformation to obtain feature Y containing rich semantic information; Perform on the feature Y Convolution operations expand the number of channels, which in turn expands the input features. The pixels are added one by one, and bilinear interpolation is used to obtain the final output features. : In the formula, This represents the bilinear interpolation operation. It is a convolution operator.

8. The pedestrian and vehicle detection method based on knowledge distillation according to claim 1, characterized in that, The student model is jointly trained using feature distillation loss and Logit distillation loss, specifically including: Selecting the corresponding feature layer in the joint distillation network as the distillation layer, when the channel dimensions of the corresponding distillation layer are inconsistent between the teacher model and the student model, utilizes... The convolutional layer performs a linear mapping alignment on the feature map of the student model to make its channel number consistent with that of the teacher model; Mask-generated distillation is employed as a feature distillation strategy. This involves applying a random masking operation to the feature map of the student model to obscure a portion of the spatial region. The feature distillation loss function... as follows: In the formula, This represents the feature map of the student model after channel alignment. This represents the feature map of the teacher model, where M is a randomly generated mask matrix. This indicates element-wise multiplication. This indicates the generation module used for feature recovery. This represents the weighting of distillation loss.

9. The pedestrian and vehicle detection method based on knowledge distillation according to claim 8, characterized in that, The methods for constructing the overall loss function used in the Logit distillation loss and joint training include: A temperature coefficient T is introduced to perform Softmax normalization on the output Logits of the teacher and student models. Knowledge transfer is achieved by minimizing the KL divergence between the predicted probability distributions of the two models, and a Logit distillation loss function is constructed. : In the formula, This represents the predicted probability distribution of the teacher model. This represents the predicted probability distribution of the student model, where T is the distillation temperature coefficient. The overall loss function of joint optimization The structure is as follows: In the formula, This represents the loss function for the original object detection task, including classification loss, bounding box regression loss, and object confidence loss. and These are the weighting coefficients for characteristic distillation loss and Logit distillation loss, respectively.

10. A pedestrian and vehicle detection system based on knowledge distillation, characterized in that, include: The dataset construction and annotation module is used to construct traffic datasets for multiple time periods under foggy and normal weather conditions, and to annotate pedestrian and vehicle targets within them; The joint distillation network construction module is used to construct a joint distillation network using the optimized YOLOv13L model as the teacher model and the lightweight YOLOv13n model as the student model. The student model joint training module is used to input the traffic dataset into the joint distillation network and jointly train the student model through feature distillation loss and Logit distillation loss to obtain the trained student model. The object detection module is used to detect pedestrians and vehicles in the image to be detected using the trained student model.

Images