Vehicle collision detection method based on double-teacher knowledge distillation

Abstract

The invention discloses a vehicle collision detection method based on double-teacher knowledge distillation. The method comprises the following steps: constructing a vehicle collision detection data set fusing real and virtual data; a vehicle collision detection network is constructed based on a double-teacher knowledge distillation framework, the vehicle collision detection network comprises a student network and two teacher networks, each network introduces a space-channel collaborative attention mechanism module and a feature alignment and comparison learning module, the student network learns from the two pre-trained teacher networks through knowledge distillation, and the feature expression and recognition precision is improved; the first teacher network pre-trains and learns normal vehicle features based on the public data set, and the second teacher network pre-trains and learns accident vehicle features based on the virtual traffic accident data; training a student network based on real traffic accident data, calculating feature alignment loss, comparing learning loss and knowledge distillation loss, and finally obtaining a student network detection model which is applied to accident vehicle detection in a complex traffic scene; the method has the advantage that the accuracy and robustness of accident vehicle detection can be remarkably improved.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to a vehicle collision detection method based on dual-teacher knowledge distillation. Background Art

[0002] In recent years, research on deep learning-based object detection technology in the field of road traffic has made significant progress both domestically and internationally. Internationally, many leading universities and research institutions, such as Stanford University, MIT, and Google, have actively explored the application of deep learning in traffic scenarios and developed highly effective object detection algorithms such as YOLO and Faster R-CNN. These algorithms not only improve the accuracy and real-time performance of object detection but are also widely used in scenarios such as autonomous driving and intelligent surveillance. In China, with the rapid development of artificial intelligence and autonomous driving technologies, research institutions and companies are also actively conducting related research. Universities such as Tsinghua University and Beijing Institute of Technology have achieved significant results in optimizing deep learning algorithms and constructing traffic scenario datasets. Furthermore, technology companies such as Baidu and Huawei have invested significant resources in the research and development of intelligent transportation systems, promoting the practical application of deep learning-based object detection technology.

[0003] The development of object detection technology has provided strong technical support for vehicle collision detection. As a key application in intelligent transportation systems, vehicle collision detection aims to reduce traffic accidents by detecting potential collision risks between vehicles and obstacles and issuing early warnings. However, current vehicle collision detection methods based on knowledge distillation still face the following challenges:

[0004] First, how to improve the authenticity and reliability of Metaverse edge scene datasets. While Metaverse-generated datasets offer significant advantages in expanding the scale of training samples and enriching traffic scenarios, their construction often requires significant manpower and time, and there are significant domain differences between virtual data and real accident scenarios. This discrepancy can lead to reduced generalization performance of trained models in real-world environments. Therefore, how to incorporate real-world accident perception data to verify and correct virtual samples, ensuring consistency and complementarity between virtual and real data, is key to improving data quality and model practicality.

[0005] Second, how to improve the robustness of the student network when learning from multi-source heterogeneous data? Fusion of multi-source heterogeneous data improves the model's ability to perceive diverse scenarios, but it also increases the learning complexity of the student network. Faced with the coexistence of normal and accident vehicle features, interference from unseen scenes, and the influence of data noise, the student network is prone to overfitting to specific scenarios or losing its discriminative ability. Therefore, it is urgent to establish an efficient feature fusion and decoupling mechanism to enhance the adaptability and robustness of the student network in complex traffic environments and ensure its stable performance in multi-task detection.

[0006] Third, how to improve the heterogeneous knowledge fusion effect in dual-teacher knowledge distillation. Although the dual-teacher knowledge distillation method can guide the student network to learn multi-source information from different perspectives, there may be knowledge redundancy or conflict between the teacher networks, resulting in mutual interference of the transmitted feature information. In addition, there is still a lack of effective mechanisms to dynamically adjust the contribution weights of the two teachers in different scenarios. How to design a collaborative knowledge fusion strategy to accurately extract the effective information of each teacher in their area of ​​strength, avoid redundancy or misleading, and improve the overall learning effect of the student network is one of the core challenges of current research. Summary of the Invention

[0007] The technical problem to be solved by the present invention is to provide a vehicle collision detection method based on dual-teacher knowledge distillation, which can significantly improve the accuracy and robustness of accident vehicle detection by fusing real-world data and virtual simulation data and combining the dual-teacher knowledge distillation framework.

[0008] The technical solution adopted by the present invention to solve the above technical problems is: a vehicle collision detection method based on dual-teacher knowledge distillation, characterized by comprising the following steps:

[0009] Step 1: Construct a vehicle collision detection dataset that integrates real-world data and virtual simulation data. The vehicle collision detection dataset includes a real car accident scene dataset and a virtual car accident scene dataset;

[0010] Step 2: Based on the dual-teacher knowledge distillation framework, a vehicle collision detection network is constructed, which includes a student network and two teacher networks, namely the first teacher network and the second teacher network. Both the student network and the two teacher networks introduce a spatial-channel collaborative attention mechanism module for jointly modeling the spatial salient regions and the importance of channel features in the feature map, and both introduce a feature alignment and contrastive learning module for modeling the differences between normal vehicle targets and accident vehicle targets in the feature space. The feature alignment and contrastive learning module includes a first feature extraction layer for feature alignment and a second feature extraction layer for contrastive learning. The student network learns knowledge from the two pre-trained teacher networks simultaneously through the knowledge distillation method, and transfers the knowledge from the two pre-trained teacher networks to its own network, thereby improving its feature expression ability and recognition accuracy in the accident vehicle detection task.

[0011] Step 3: Pre-train the first teacher network based on the public dataset to learn the features of normal vehicles; pre-train the second teacher network based on the virtual car accident scene dataset to learn the features of accident vehicles; after the first and second teacher networks are pre-trained, the student network is trained based on the real car accident scene dataset. The student network outputs the accident vehicle detection results corresponding to each real car accident scene image in the real car accident scene dataset. After each round of training, the feature alignment loss, contrastive learning loss, and knowledge distillation loss are calculated. Among them, the feature alignment loss is obtained by calculating the mean square error between the standardized data and the true label, the contrastive learning loss is obtained by calculating the cosine similarity between positive samples and the cosine similarity between negative samples, and the knowledge distillation loss is calculated by Mimic loss to calculate the knowledge difference between the student network and the two teacher networks; the training process ends after the student network has trained for a preset number of rounds to obtain the student network detection model;

[0012] Step 4: Apply the student network detection model to the accident vehicle detection task in complex traffic scenes.

[0013] In step 1, the construction process of the real car accident scene dataset is as follows: searching for at least 2,000 real car accident scene images and forming a first dataset; then annotating the accident vehicle target and the normal vehicle target in each real car accident scene image in the first dataset, and saving the annotation information to obtain the annotated first dataset, wherein the annotation information includes the coordinate information of the four vertices of the annotation box and the category information of the annotation box, and the category information includes the accident vehicle target and the normal vehicle target; then adjusting the size of all the real car accident scene images in the annotated first dataset to be consistent to obtain the real car accident scene dataset.

[0014] In step 1, the construction process of the virtual car accident scene dataset is as follows: the collision scene, weather, and lighting condition prompt words are set by the user to generate at least 10,000 virtual car accident scene images and form a second dataset; then the accident vehicle target and the normal vehicle target in each virtual car accident scene image in the second dataset are labeled, and the labeling information is saved to obtain the labeled second dataset, wherein the labeling information includes the coordinate information of the four vertices of the labeling box and the category information of the labeling box, and the category information includes the accident vehicle target and the normal vehicle target; then the size of all virtual car accident scene images in the labeled second dataset is adjusted to be consistent to obtain the virtual car accident scene dataset.

[0015] In step 2, the structures of the student network and the two teacher networks are the same, which are composed of a first convolution module, a second convolution module, a third convolution module, a fourth convolution module, a fifth convolution module, a sixth convolution module, a seventh convolution module, a first space-channel collaborative attention mechanism layer, a second space-channel collaborative attention mechanism layer, a third space-channel collaborative attention mechanism layer, a fourth space-channel collaborative attention mechanism layer, a fifth space-channel collaborative attention mechanism layer, a sixth space-channel collaborative attention mechanism layer, a seventh space-channel collaborative attention mechanism layer, a first feature extraction layer, and a second feature extraction layer. , a spatial pyramid pooling layer, a first upsampling layer, a second upsampling layer, a first connection layer, a second connection layer, a third connection layer, a fourth connection layer, a first detection output layer, a second detection output layer, and a third detection output layer; the first spatial-channel collaborative attention mechanism layer to the seventh spatial-channel collaborative attention mechanism layer constitute a spatial-channel collaborative attention mechanism module; the first feature extraction layer is used for feature alignment, and the second feature extraction layer is used for contrastive learning; the first feature extraction layer and the second feature extraction layer constitute a feature alignment and contrastive learning module; the spatial pyramid pooling layer is used to fuse feature maps of different scales;

[0016] The implementation process of the student network and the two teacher networks is the same, specifically: the image samples are sequentially passed through the first convolution module, the second convolution module, the first space-channel collaborative attention mechanism layer, the third convolution module, the second space-channel collaborative attention mechanism layer, the fourth convolution module, the first feature extraction layer, the fifth convolution module, the second feature extraction layer, the spatial pyramid pooling layer, the third space-channel collaborative attention mechanism layer, the second space-channel collaborative attention mechanism layer outputs the second space-channel joint feature map, the first feature extraction layer outputs the alignment feature map, and the third space-channel collaborative attention mechanism layer outputs the third space-channel joint feature map; the third space-channel joint feature map passes the first upsampling layer to obtain the first upsampling feature map; the alignment feature map and the first upsampling feature map are combined. Figure 1 The first connection feature map is input into the first connection layer, and the first connection feature map is obtained; the first connection feature map is sequentially passed through the fourth space-channel collaborative attention mechanism layer and the second upsampling layer, the fourth space-channel collaborative attention mechanism layer outputs the fourth space-channel joint feature map, and the second upsampling layer outputs the second upsampling feature map; the second space-channel joint feature map and the second upsampling feature map are connected. Figure 1 The second connection feature map is input into the second connection layer to obtain the second connection feature map; the second connection feature map is sequentially passed through the fifth space-channel collaborative attention mechanism layer and the sixth convolution module, and the fifth space-channel collaborative attention mechanism layer outputs the fifth space-channel joint feature map; the fourth space-channel joint feature map and the feature map output by the sixth convolution module are combined. Figure 1The third connection feature map is input into the third connection layer to obtain the third connection feature map; the third connection feature map is sequentially passed through the sixth space-channel collaborative attention mechanism layer and the seventh convolution module, and the sixth space-channel collaborative attention mechanism layer outputs the sixth space-channel joint feature map; the third space-channel joint feature map and the feature map output by the seventh convolution module are combined. Figure 1 The fourth connection feature map is input into the fourth connection layer to obtain the fourth connection feature map; the fourth connection feature map is passed through the seventh space-channel collaborative attention mechanism layer to obtain the seventh space-channel joint feature map; the fifth space-channel joint feature map is passed through the first detection output layer to obtain the first detection output map; the sixth space-channel joint feature map is passed through the second detection output layer to obtain the second detection output map; the seventh space-channel joint feature map is passed through the third detection output layer to obtain the third detection output map; the first detection output map, the second detection output map, and the third detection output map are mapped to the coordinate space of the image sample, and the three detection output maps obtained after mapping are added to obtain the accident vehicle detection result.

[0017] The number of channels of the image sample is 3 and the size is H×W; the convolution kernel size of the first convolution module is 3×3, the stride is 2, the padding is 1, the number of convolution kernels is 64, and the size of the output feature map is (H / 2)×(W / 2); the convolution kernel size of the second convolution module is 3×3, the stride is 2, the padding is 1, the number of convolution kernels is 128, and the size of the output feature map is (H / 4)×(W / 4); the padding of the first spatial-channel collaborative attention mechanism layer is 2, the number of convolution kernels is 128, and the size of the output first spatial channel joint feature map is (H / 4)×(W / 4); the convolution kernel size of the third convolution module is 3×3, the stride is 2, the padding is 1, the number of convolution kernels is 256, and the size of the output feature map is (H / 8)×(W / 8); the padding of the second spatial-channel collaborative attention mechanism layer is 2, the number of convolution kernels is 256, and the size of the output second spatial channel joint feature map is (H / 8)×(W / 8); the convolution kernel size of the fourth convolution module is 3×3, the stride is 2, the padding is 1, the number of convolution kernels is 512, and the size of the output feature map is (H / 16)×(W / 16); the padding of the first feature extraction layer is 2, the number of convolution kernels is 512, and the size of the output alignment feature map is (H / 16)×(W / 16); the convolution kernel size of the fifth convolution module is 3×3, the stride is 2, the padding is 1, the number of convolution kernels is 1024, and the size of the output feature map is (H / 32)×(W / 32); the second feature The padding of the extraction layer is 2, the number of convolution kernels is 1024, and the size of the output contrastive learning feature map is (H / 32)×(W / 32); the convolution kernel size of the spatial pyramid pooling layer is 5×5, the stride is 1, the padding is 2, the number of convolution kernels is 1024, and the size of the output fusion feature map is (H / 32)×(W / 32); the padding of the third space-channel collaborative attention mechanism layer is 2, the number of convolution kernels is 1024, and the size of the output third space-channel joint feature map is (H / 32)×(W / 32); the size of the first upsampling feature map output by the first upsampling layer is (H / 16)×(W / 16); the size of the first connection feature map output by the first connection layer is (H / 16)×(W / 16); The padding of the four spatial-channel collaborative attention mechanism layers is 2, the number of convolution kernels is 512, and the size of the output fourth spatial channel joint feature map is (H / 16)×(W / 16); the size of the second upsampling feature map output by the second upsampling layer is (H / 8)×(W / 8); the size of the second connection feature map output by the second connection layer is (H / 8)×(W / 8); the padding of the fifth spatial-channel collaborative attention mechanism layer is 2, the number of convolution kernels is 256, and the size of the output fifth spatial channel joint feature map is (H / 8)×(W / 8); the convolution kernel size of the sixth convolution module is 3×3, the stride is 2, the padding is 1, the number of convolution kernels is 256, and the size of the output feature map is (H / 16)×(W / 16);The size of the third connection feature map output by the third connection layer is (H / 16)×(W / 16); the padding of the sixth space-channel collaborative attention mechanism layer is 2, the number of convolution kernels is 512, and the size of the output sixth space-channel joint feature map is (H / 16)×(W / 16); the convolution kernel size of the seventh convolution module is 3×3, the stride is 2, the padding is 1, the number of convolution kernels is 512, and the size of the output feature map is (H / 32)×(W / 32); the size of the fourth connection feature map output by the fourth connection layer is (H / 32)×(W / 32); the padding of the seventh space-channel collaborative attention mechanism layer is 2, the number of convolution kernels is 512, and the size of the output feature map is (H / 32)×(W / 32). The number of convolution kernels in the first detection output layer is 2, the number of convolution kernels is 1024, and the size of the output seventh spatial channel joint feature map is (H / 32)×(W / 32); the number of convolution kernels in the first detection output layer is 256, and the size of the output first detection output map is (H / 8)×(W / 8); the number of convolution kernels in the second detection output layer is 512, and the size of the output second detection output map is (H / 16)×(W / 16); the number of convolution kernels in the third detection output layer is 1024, and the size of the output third detection output map is (H / 32)×(W / 32); the number of channels of the accident vehicle detection result is 3 and the size is H×W.

[0018] The structures of the first to seventh spatial-channel collaborative attention mechanism layers are the same, and are composed of the following modules:

[0019] The global pooling module takes as input the feature map received by the i-th spatial-channel collaborative attention mechanism layer, downsamples the received feature map, and then performs channel normalization on the downsampled feature map. Its output is the normalized feature map;

[0020] The QKV generation module takes the normalized feature map as input and consists of three parallel convolution branches that generate query, key, and value features respectively. Its output is query features, key features, and value features.

[0021] The spatial attention extraction module takes as input the feature map received by the i-th spatial-channel collaborative attention mechanism layer. It includes four parallel one-dimensional depthwise separable convolutional layers with kernel sizes of 3×3, 5×5, 7×7, and 9×9, respectively. It uses four one-dimensional depthwise separable convolutional layers to generate four local spatial feature maps of different scales. The four local spatial feature maps are then concatenated in the channel dimension to obtain a spatially enhanced feature map. The output is the spatially enhanced feature map.

[0022] The multi-head attention calculation module takes as input the query features, key features, value features, and spatial enhancement feature maps. It calculates the attention weights based on the query features, key features, and value features based on the scaled dot product method, and then multiplies the attention weights with the spatial enhancement feature map to obtain the attention feature map. The output is the attention feature map;

[0023] The channel attention extraction module takes the attention feature map as input, dynamically adjusts the channel weights using global average pooling and gating mechanisms, and then recalibrates the attention feature map in the channel dimension using the adjusted channel weights. The output is the recalibrated feature map.

[0024] The recalibrated feature map is multiplied channel by channel with the feature map received by the i-th spatial-channel collaborative attention mechanism layer, and the feature map obtained by the channel-by-channel multiplication is used as the feature map output by the i-th spatial-channel collaborative attention mechanism layer; where i belongs to {1, 2, 3, 4, 5, 6, 7}.

[0025] The first feature extraction layer is composed of an eighth convolution module, a ninth convolution module, a first feature alignment operation, a second feature alignment operation, a first Bottleneck layer, a second Bottleneck layer, a third Bottleneck layer, and a fifth connection layer; the implementation process of the first feature extraction layer is: the feature map received by the first feature extraction layer passes through the eighth convolution module and the first feature alignment operation in sequence; the feature map obtained by the first feature alignment operation is divided into two tensors of the same size according to the channel direction, one of the tensors is input into the first Bottleneck layer, the second Bottleneck layer, and the third Bottleneck layer respectively, and the other tensor is used as the first input and the second input of the fifth connection layer respectively; the feature maps output by the first Bottleneck layer, the second Bottleneck layer, and the third Bottleneck layer are respectively used as the third input, the fourth input, and the fifth input of the fifth connection layer, and the fifth connection layer splices the first input, the second input, the third input, the fourth input, and the fifth input in the channel dimension; the feature map output by the fifth connection layer passes through the ninth convolution module and the second feature alignment operation in sequence; the feature map obtained by the second feature alignment operation is used as the alignment feature map output by the first feature extraction layer;

[0026] The second feature extraction layer is composed of the tenth convolution module, the eleventh convolution module, the twelfth convolution module, the thirteenth convolution module, the Module List module, and the Projection Head module. The Module List module includes the fourth Bottleneck layer, the fifth Bottleneck layer, and the sixth Bottleneck layer connected in sequence. The head module includes a fourteenth convolution module, a fifteenth convolution module, a batch normalization layer, and a ReLu activation layer connected in sequence; the implementation process of the second feature extraction layer is: the feature map received by the second feature extraction layer is sequentially passed through the tenth convolution module, the eleventh convolution module, the twelfth convolution module, the thirteenth convolution module, the fourth Bottleneck layer, the fifth Bottleneck layer, and the sixth Bottleneck layer to obtain the main path output feature map; when the number of targets in the feature map received by the second feature extraction layer is not 0, the main path output feature map is sequentially passed through the fourteenth convolution module, the fifteenth convolution module, the batch normalization layer, and the ReLu activation layer, and the feature map output by the ReLu activation layer is used as the comparative learning feature map output by the second feature extraction layer; when the number of targets in the feature map received by the second feature extraction layer is 0, the main path output feature map is directly used as the comparative learning feature map output by the second feature extraction layer, wherein the targets include normal vehicle targets and accident vehicle targets;

[0027] The first connection layer to the fourth connection layer are all used to achieve splicing in the channel dimension.

[0028] In step 3, during the pre-training of the first teacher network and the second teacher network and the training of the student network, batch_size is set to 8, the round is set to 150 times, the weight decay is set to 0.0005, and the learning rate is set to 0.0001.

[0029] In step 3, the calculation formula of feature alignment loss is: Among them, Loss featurealignment represents feature alignment loss, b=1, 2, ..., batch_size, batch_size represents batch size, y b It represents the alignment feature map output by the first feature extraction layer after the bth feature map in each batch of the real car accident scene dataset is normalized and input into the student network. Represents the true label corresponding to the b-th feature map in each batch of the real car accident scene dataset. The normalization process of the b-th feature map in each batch of the real car accident scene dataset is: C represents the number of channels of the aligned feature map output by the first feature extraction layer in the student network, h = 1, 2, ..., H′, w = 1, 2, ..., W′, H′×W′ represents the size of the aligned feature map output by the first feature extraction layer in the student network, γ represents a learnable scaling parameter, x b，c，h，w Represents the spatial dimension pixel value with coordinates (h, w) in the c-th channel of the b-th feature map in each batch of the real car accident scene dataset, μ b，c Represents the average value of all spatial dimension pixel values ​​in the cth channel of the bth feature map in each batch of the real car accident scene dataset, Represents the variance of all spatial dimension pixel values ​​in the cth channel of the bth feature map in each batch of the real car accident scene dataset, β represents a learnable bias parameter, ∈ represents a small constant used to prevent the denominator from being zero, and z b，c,h，w Represents the normalized spatial dimension pixel value with coordinates (h, w) in the c-th channel of the b-th feature map in each batch of the real car accident scene dataset;

[0030] The formula for contrastive learning loss is: Among them, Loss contrastive Represents contrastive learning loss, batch_size represents batch size, positive represents the positive samples in each batch of the real car accident scene dataset, negative represents the negative samples in each batch of the real car accident scene dataset, the positive samples are accident vehicle targets, the negative samples are normal vehicle targets, p≠q, CS p，q represents the cosine similarity between the p-th positive sample and the q-th positive sample, Indicates the negative samples and The cosine similarity between negative samples;

[0031] The calculation formula of knowledge distillation loss is: Among them, Loss distillation represents the knowledge distillation loss, L=2, represents the first feature extraction layer, represents the second feature extraction layer, MSE(·) represents the mean square error function, hour represents the aligned feature map output by the first feature extraction layer in the student network, hour represents the contrastive learning feature map output by the second feature extraction layer in the student network, hour represents the aligned feature map output by the first feature extraction layer in the first teacher network, hour represents the contrastive learning feature map output by the second feature extraction layer in the first teacher network, hour represents the aligned feature map output by the first feature extraction layer in the second teacher network, hour Represents the contrastive learning feature map output of the second feature extraction layer in the second teacher network.

[0032] Compared with the prior art, the advantages of the present invention are:

[0033] 1) This method aims to leverage multi-sample real-world and virtual scene data to improve the network's feature extraction capabilities for small-sample edge cases. Specifically, the first teacher network, trained using the BDD100K public dataset, guides the student network in learning the characteristics of normal vehicles. The second teacher network, trained using a dataset of virtual crash scenes, guides the student network in learning the characteristics of accident vehicles. The student network, trained using a dataset of real crash scenes, ensures that it meets the detection requirements of small-sample crash scenes in practical applications. Combining knowledge distillation, contrastive learning, and feature alignment, this method effectively improves accident vehicle detection performance, achieving a 54.3% accuracy rate for accident vehicle detection in complex real-world environments, 4.3% higher than the YOLOv11 object detection network.

[0034] 2) Based on the dual-teacher knowledge distillation framework, a first feature extraction layer for feature alignment and a second feature extraction layer for contrastive learning are designed. For the second feature extraction layer, a contrastive learning mechanism based on cosine similarity is introduced to enable the student network to effectively learn the differences between accident vehicles and normal vehicles in the feature space. Positive and negative sample pairs are dynamically constructed (accident vehicles are positive samples and normal vehicles are negative samples), so that the student network forms clear inter-class boundaries during training, thereby enhancing the student network's clustering ability and discriminative expression of abnormal targets, and effectively reducing the false detection and missed detection rates. For the first feature extraction layer, the mean-variance normalization method is used to dynamically calibrate the feature distributions in different domains (such as real environments and virtual scenes), effectively alleviating the problem of inter-domain feature offset. The first feature extraction layer has the advantages of being lightweight, unsupervised and end-to-end trainable, while improving the cross-domain robustness of the student network and avoiding complex adversarial training design.

[0035] Through the fusion design of multi-scale convolutional structure and gating mechanism, the spatial-channel collaborative attention module can simultaneously model key areas of spatial and channel dimensions. This module not only enhances the response to local salient features, but also extracts rich contextual information under different receptive fields, achieving high-precision focusing and expression enhancement on the target area of ​​the accident vehicle, and improving the adaptability and robustness of the student network detection model in complex scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 This is a block diagram of the overall implementation of the method of the present invention;

[0037] Figure 2 A schematic diagram of the structure of a student network and a teacher network in a vehicle collision detection network constructed in the method of the present invention;

[0038] Figure 3 for Figure 2 Schematic diagram of the composition structure of the spatial-channel collaborative attention mechanism layer in . DETAILED DESCRIPTION

[0039] The present invention will be described in further detail below with reference to the accompanying drawings and embodiments.

[0040] The present invention proposes a vehicle collision detection method based on dual-teacher knowledge distillation, and its overall implementation block diagram is as follows: Figure 1 As shown, it includes the following steps:

[0041] Step 1: Construct a vehicle collision detection dataset that integrates real-world data and virtual simulation data. The vehicle collision detection dataset includes a real car accident scene dataset and a virtual car accident scene dataset. By integrating the real car accident scene dataset and the virtual car accident scene dataset, the overall authenticity and reliability of the vehicle collision detection dataset are improved, and the domain difference between virtual simulation data and real accident scenes is reduced.

[0042] In this embodiment, in step 1, the construction process of the real car accident scene dataset is as follows: at least 2000 real car accident scene images are searched through the network to form a first dataset, and these real car accident scene images should cover a variety of accident types, vehicle types, collision angles and environmental conditions to ensure the diversity and representativeness of the first dataset; then the Label Img annotation tool is used to annotate the accident vehicle target and the normal vehicle target in each real car accident scene image in the first dataset, and the annotation information is saved to obtain the annotated first dataset, wherein the annotation information includes the coordinate information of the four vertices of the annotation box (used to determine the position of the vehicle in the real car accident scene image) and the category information of the annotation box, and the category information includes the accident vehicle target and the normal vehicle target; then the size of all real car accident scene images in the annotated first dataset is adjusted to be consistent (such as adjusted to 640×640) to obtain the real car accident scene dataset. In the actual construction process, the annotated first dataset can also be quality checked to ensure the accuracy and consistency of the annotation, and the real car accident scene images with inaccurate or incomplete annotations are re-annotated or removed.

[0043] In this embodiment, in step 1, the construction process of the virtual car accident scene dataset is as follows: set the collision scene, weather, and lighting condition prompt words by yourself, use Midjourney or other image generation tools to generate at least 10,000 virtual car accident scene images, and form a second dataset. These virtual car accident scene images should cover a variety of virtual accident scenes to ensure the diversity and richness of the second dataset; then use the LabelImg annotation tool to annotate the accident vehicle target and normal vehicle target in each virtual car accident scene image in the second dataset, and save the annotation information to obtain the annotated second dataset, wherein the annotation information includes the coordinate information of the four vertices of the annotation box (used to determine the position of the vehicle in the virtual car accident scene image) and the category information of the annotation box, the category information includes the accident vehicle target and the normal vehicle target; then resize all virtual car accident scene images in the annotated second dataset to be consistent (e.g., adjusted to 640×640), to obtain the virtual car accident scene dataset. In the actual construction process, the annotated second dataset can also be quality checked to ensure the accuracy and consistency of the annotation, and virtual car accident scene images with inaccurate or incomplete annotations are re-annotated or removed.

[0044] Step 2: Based on the dual-teacher knowledge distillation framework, a vehicle collision detection network is constructed, which includes a student network constructed based on the YOLOv11 target detection network and two teacher networks, namely the first teacher network and the second teacher network. Both the student network and the two teacher networks introduce a spatial-channel collaborative attention mechanism module for jointly modeling the spatial significance area and channel feature importance in the feature map to adaptively optimize the feature extraction process of the key area, and both introduce a feature alignment and contrastive learning module for modeling the difference between normal vehicle targets and accident vehicle targets in the feature space to improve the vehicle collision detection network's ability to discriminate accident vehicle targets. The feature alignment and contrastive learning module includes a first feature extraction layer for feature alignment and a second feature extraction layer for contrastive learning; the student network learns knowledge from the two pre-trained teacher networks at the same time through the knowledge distillation method, and transfers the knowledge in the two pre-trained teacher networks to its own network, thereby improving its feature expression ability and recognition accuracy in the accident vehicle detection task.

[0045] In this embodiment, in step 2, the structures of the student network and the two teacher networks are the same, such as Figure 2 As shown, they are composed of the first convolution module (Conv), the second convolution module, the third convolution module, the fourth convolution module, the fifth convolution module, the sixth convolution module, the seventh convolution module, the first space-channel collaborative attention mechanism layer (SCSA), the second space-channel collaborative attention mechanism layer, the third space-channel collaborative attention mechanism layer, the fourth space-channel collaborative attention mechanism layer, the fifth space-channel collaborative attention mechanism layer, the sixth space-channel collaborative attention mechanism layer, the seventh space-channel collaborative attention mechanism layer, the first feature extraction layer (FAL), the second feature extraction layer (CFE), the spatial pyramid pool The first upsampling layer, the second upsampling layer, the first connection layer (Concat), the second connection layer, the third connection layer, the fourth connection layer, the first detection output layer, the second detection output layer, and the third detection output layer are composed of the first spatial-channel collaborative attention mechanism layer to the seventh spatial-channel collaborative attention mechanism layer. The spatial-channel collaborative attention mechanism module is composed of the first feature extraction layer for feature alignment, and the second feature extraction layer for contrastive learning. The first feature extraction layer and the second feature extraction layer constitute a feature alignment and contrastive learning module. The spatial pyramid pooling layer is used to fuse feature maps of different scales.

[0046] In this embodiment, the implementation process of the student network and the two teacher networks is the same, specifically: the image samples are sequentially passed through the first convolution module, the second convolution module, the first space-channel collaborative attention mechanism layer, the third convolution module, the second space-channel collaborative attention mechanism layer, the fourth convolution module, the first feature extraction layer, the fifth convolution module, the second feature extraction layer, the spatial pyramid pooling layer, the third space-channel collaborative attention mechanism layer, the second space-channel collaborative attention mechanism layer outputs the second space-channel joint feature map, the first feature extraction layer outputs the alignment feature map, and the third space-channel collaborative attention mechanism layer outputs the third space-channel joint feature map; the third space-channel joint feature map passes the first upsampling layer to obtain the first upsampling feature map; the alignment feature map and the first upsampling feature map are combined. Figure 1 The first connection feature map is input into the first connection layer, and the first connection feature map is obtained; the first connection feature map is sequentially passed through the fourth space-channel collaborative attention mechanism layer and the second upsampling layer, the fourth space-channel collaborative attention mechanism layer outputs the fourth space-channel joint feature map, and the second upsampling layer outputs the second upsampling feature map; the second space-channel joint feature map and the second upsampling feature map are connected. Figure 1 The second connection feature map is input into the second connection layer to obtain the second connection feature map; the second connection feature map is sequentially passed through the fifth space-channel collaborative attention mechanism layer and the sixth convolution module, and the fifth space-channel collaborative attention mechanism layer outputs the fifth space-channel joint feature map; the fourth space-channel joint feature map and the feature map output by the sixth convolution module are combined. Figure 1 The third connection feature map is input into the third connection layer to obtain the third connection feature map; the third connection feature map is sequentially passed through the sixth space-channel collaborative attention mechanism layer and the seventh convolution module, and the sixth space-channel collaborative attention mechanism layer outputs the sixth space-channel joint feature map; the third space-channel joint feature map and the feature map output by the seventh convolution module are combined. Figure 1The fourth connection feature map is input into the fourth connection layer to obtain the fourth connection feature map; the fourth connection feature map is passed through the seventh space-channel collaborative attention mechanism layer to obtain the seventh space-channel joint feature map; the fifth space-channel joint feature map is passed through the first detection output layer to obtain the first detection output map; the sixth space-channel joint feature map is passed through the second detection output layer to obtain the second detection output map; the seventh space-channel joint feature map is passed through the third detection output layer to obtain the third detection output map; the first detection output map, the second detection output map, and the third detection output map are mapped to the coordinate space of the image sample, and the three detection output maps obtained after mapping are added to obtain the accident vehicle detection result. Since the student network and the two teacher networks are built based on the YOLOv11 target detection network, the student network and the teacher network will remap the center point and width and height of the bounding box of the first detection output image to the coordinate space of the image sample according to the proportional relationship (scaling factor) between the first detection output image and the image sample; similarly, the center point and width and height of the bounding box of the second detection output image will be remapped to the coordinate space of the image sample according to the proportional relationship (scaling factor) between the second detection output image and the image sample; the center point and width and height of the bounding box of the third detection output image will be remapped to the coordinate space of the image sample according to the proportional relationship (scaling factor) between the third detection output image and the image sample.

[0047] In this embodiment, the number of channels of the image sample is 3 and the size is H×W (640×640); the convolution kernel size of the first convolution module is 3×3, the stride is 2, the padding is 1, the number of convolution kernels is 64, and the size of the output feature map is (H / 2)×(W / 2) (i.e., 320×320); the convolution kernel size of the second convolution module is 3×3, the stride is 2, the padding is 1, the number of convolution kernels is 128, and the size of the output feature map is (H / 4)×(W / 4) (i.e., 160×160); the padding of the first spatial-channel collaborative attention mechanism layer is 2, the number of convolution kernels is 128, and the size of the output first spatial channel joint feature map is (H / 4)×(W / 4) (i.e., 160×160); The convolution kernel size of the three convolution modules is 3×3, the stride is 2, the padding is 1, the number of convolution kernels is 256, and the size of the output feature map is (H / 8)×(W / 8) (i.e. 80×80); the padding of the second spatial-channel collaborative attention mechanism layer is 2, the number of convolution kernels is 256, and the size of the output second spatial channel joint feature map is (H / 8)×(W / 8) (i.e. 80×80); the convolution kernel size of the fourth convolution module is 3×3, the stride is 2, the padding is 1, the number of convolution kernels is 512, and the size of the output feature map is (H / 16)×(W / 16) (i.e. 40×40); the padding of the first feature extraction layer is 2, the number of convolution kernels is 512, and the size of the output alignment feature map is (H / 1 6)×(W / 16) (i.e. 40×40); the convolution kernel size of the fifth convolution module is 3×3, the stride is 2, the padding is 1, the number of convolution kernels is 1024, and the size of the output feature map is (H / 32)×(W / 32) (i.e. 20×20); the padding of the second feature extraction layer is 2, the number of convolution kernels is 1024, and the size of the output contrastive learning feature map is (H / 32)×(W / 32) (i.e. 20×20); the convolution kernel size of the spatial pyramid pooling layer is 5×5, the stride is 1, the padding is 2, the number of convolution kernels is 1024, and the size of the output fusion feature map is (H / 32)×(W / 32) (i.e. 20×20); the padding of the third spatial-channel collaborative attention mechanism layer is 2, the number of convolution kernels is 1024, and the size of the output fusion feature map is (H / 32)×(W / 32) (i.e. 20×20). The number of convolution kernels is 1024, and the size of the output third spatial channel joint feature map is (H / 32)×(W / 32) (i.e., 20×20); the size of the first up-sampled feature map output by the first upsampling layer is (H / 16)×(W / 16) (i.e., 40×40); the size of the first connection feature map output by the first connection layer is (H / 16)×(W / 16) (i.e., 40×40); the padding of the fourth spatial-channel collaborative attention mechanism layer is 2, the number of convolution kernels is 512, and the size of the output fourth spatial channel joint feature map is (H / 16)×(W / 16) (i.e., 40×40); the size of the second up-sampled feature map output by the second upsampling layer is (H / 8)×(W / 8) (i.e., 80×80);The size of the second connection feature map output by the second connection layer is (H / 8)×(W / 8) (i.e., 80×80); the padding of the fifth spatial-channel collaborative attention mechanism layer is 2, the number of convolution kernels is 256, and the size of the output fifth spatial channel joint feature map is (H / 8)×(W / 8) (i.e., 80×80); the convolution kernel size of the sixth convolution module is 3×3, the stride is 2, the padding is 1, the number of convolution kernels is 256, and the size of the output feature map is (H / 16)×(W / 1 6) (i.e. 40×40); the size of the third connection feature map output by the third connection layer is (H / 16)×(W / 16) (i.e. 40×40); the padding of the sixth spatial-channel collaborative attention mechanism layer is 2, the number of convolution kernels is 512, and the size of the sixth spatial channel joint feature map output is (H / 16)×(W / 16) (i.e. 40×40); the convolution kernel size of the seventh convolution module is 3×3, the stride is 2, the padding is 1, the number of convolution kernels is 512, and the output feature map is The size of the fourth connection feature map output by the fourth connection layer is (H / 32)×(W / 32) (i.e., 20×20); the padding of the seventh spatial-channel collaborative attention mechanism layer is 2, the number of convolution kernels is 1024, and the size of the output seventh spatial channel joint feature map is (H / 32)×(W / 32) (i.e., 20×20); the number of convolution kernels of the first detection output layer is 256, and the output of the first detection output is The image size is (H / 8)×(W / 8) (i.e., 80×80); the number of convolution kernels in the second detection output layer is 512, and the size of the output second detection output image is (H / 16)×(W / 16) (i.e., 40×40); the number of convolution kernels in the third detection output layer is 1024, and the size of the output third detection output image is (H / 32)×(W / 32) (i.e., 20×20); the number of channels in the accident vehicle detection result is 3, and the size is H×W (i.e., 640×640).

[0048] In this embodiment, the structures of the first space-channel collaborative attention mechanism layer to the seventh space-channel collaborative attention mechanism layer are the same, such as Figure 3As shown in the figure, it is composed of the following modules: a global pooling module, which is used to perform scale compression and feature normalization on the feature map, and its input is the feature map received by the i-th spatial-channel collaborative attention mechanism layer. It downsamples the received feature map and then performs channel normalization on the downsampled feature map, and its output is the normalized feature map; a QKV generation module, whose input is the normalized feature map, which includes three parallel convolution branches that generate query, key and value features respectively, and its output is query features, key features and value features; a spatial attention extraction module, whose input is the feature map received by the i-th spatial-channel collaborative attention mechanism layer, which includes four parallel one-dimensional depth-separable convolution layers to extract multi-scale local spatial information for enhancing the spatial responsiveness of the query features. Their convolution kernel sizes are 3×3, 5×5, 7×7 and 9×9 respectively. It uses four one-dimensional depth-separable convolution layers to generate four local spatial feature maps of different scales, and then The four local spatial feature maps are spliced ​​in the channel dimension to obtain a spatial enhancement feature map, and its output is a spatial enhancement feature map; the multi-head attention calculation module, whose input is the query feature, key feature, value feature and spatial enhancement feature map, calculates the attention weight based on the scaled dot product method and according to the query feature, key feature and value feature, and then multiplies the attention weight with the spatial enhancement feature map to obtain the attention feature map, and its output is the attention feature map; the channel attention extraction module, whose input is the attention feature map, dynamically adjusts the channel weight using the global average pooling operation and the gating mechanism, and then recalibrates the channel dimension of the attention feature map using the adjusted channel weight, and its output is the recalibrated feature map; the recalibrated feature map is multiplied channel by channel with the feature map received by the i-th spatial-channel collaborative attention mechanism layer to achieve feature enhancement and residual preservation, and the feature map obtained by channel-by-channel multiplication is used as the feature map output by the i-th spatial-channel collaborative attention mechanism layer; where i belongs to {one, two, three, four, five, six, seven}.

[0049] As mentioned above, the downsampling operation can be implemented using average pooling; the channel normalization operation can be implemented using batch normalization; the convolution kernel size of the convolution branch is 1×1 and the stride is 1; the stride of the one-dimensional depth-separable convolution layer is 1 and the padding mode is "same"; in the channel attention extraction module, a global average pooling operation is performed on the attention feature map to obtain the global features of each channel; a fully connected layer is used to map the global features to channel weights, and then the Sigmoid activation function is used to limit the channel weights to the [0, 1] interval; the channel weights are multiplied by the attention feature map channel by channel to achieve channel dimension recalibration of the attention feature map.

[0050] In this embodiment, the first feature extraction layer (FAL) is composed of an eighth convolution module, a ninth convolution module, a first feature alignment operation, a second feature alignment operation, a first Bottleneck layer, a second Bottleneck layer, a third Bottleneck layer, and a fifth connection layer; the implementation process of the first feature extraction layer is as follows: the feature map received by the first feature extraction layer is sequentially passed through the eighth convolution module and the first feature alignment (Feature Alignment) operation; the feature map obtained by the first feature alignment operation is divided into two tensors of the same size in the channel direction, one of the tensors is input into the first Bottleneck layer, the second Bottleneck layer, and the third Bottleneck layer for feature extraction, and the other tensor is used as the first input and the second input of the fifth connection layer; the feature maps output by the first Bottleneck layer, the second Bottleneck layer, and the third Bottleneck layer are used as the third input, the fourth input, and the fifth input of the fifth connection layer, and the fifth connection layer splices the first input, the second input, the third input, the fourth input, and the fifth input in the channel dimension; the feature map output by the fifth connection layer passes through the ninth convolution module and the second feature alignment operation in sequence; the feature map obtained by the second feature alignment operation is used as the alignment feature map output by the first feature extraction layer.

[0051] In this embodiment, the second feature extraction layer is composed of the tenth convolution module, the eleventh convolution module for feature channel compression, the twelfth convolution module for dynamic construction to keep the data type consistent, the thirteenth convolution module, the ModuleList module, and the Projection Head module. The Module List module includes the fourth Bottleneck layer, the fifth Bottleneck layer, and the sixth Bottleneck layer connected in sequence. The head module includes a fourteenth convolution module, a fifteenth convolution module, a batch normalization layer, and a ReLu activation layer connected in sequence; the implementation process of the second feature extraction layer is: the feature map received by the second feature extraction layer passes through the tenth convolution module, the eleventh convolution module, the twelfth convolution module, the thirteenth convolution module, the fourth Bottleneck layer, the fifth Bottleneck layer, and the sixth Bottleneck layer in sequence to obtain the main path output feature map; when the number of targets in the feature map received by the second feature extraction layer is not 0, the main path output feature map passes through the fourteenth convolution module, the fifteenth convolution module, the batch normalization layer, and the ReLu activation layer in sequence, and the feature map output by the ReLu activation layer is used as the comparative learning feature map output by the second feature extraction layer; when the number of targets in the feature map received by the second feature extraction layer is 0, the main path output feature map is directly used as the comparative learning feature map output by the second feature extraction layer, wherein the targets include normal vehicle targets and accident vehicle targets.

[0052] In this embodiment, the first to fourth connection layers are all used to achieve splicing in the channel dimension.

[0053] In this embodiment, the first detection output layer to the third detection output layer are all convolution operations.

[0054] It is further defined that, in this embodiment, the structures of the first convolution module to the fifteenth convolution module are the same, and all include a two-dimensional convolution (Conv2d), a batch normalization layer (BatchNormalization), and a SiLU activation layer connected in sequence.

[0055] Step 3: Pre-train the first teacher network based on the BDD100K public dataset to achieve normal vehicle feature learning; pre-train the second teacher network based on the virtual car accident scene dataset to achieve accident vehicle feature learning; after the first and second teacher networks are pre-trained, the student network is trained based on the real car accident scene dataset. The student network outputs the accident vehicle detection results corresponding to each real car accident scene image in the real car accident scene dataset. After each round of training, the feature alignment loss, contrastive learning loss, and knowledge distillation loss are calculated. Among them, the feature alignment loss is obtained by calculating the mean square error between the standardized data and the true label, the contrastive learning loss is obtained by calculating the cosine similarity between positive samples and the cosine similarity between negative samples, and the knowledge distillation loss is calculated by Mimic loss to calculate the knowledge difference between the student network and the two teacher networks; the training process ends after the student network has trained for a preset number of rounds to obtain the student network detection model.

[0056] In this embodiment, in step 3, during pre-training of the first and second teacher networks and training of the student network, batch_size is set to 8, rounds are set to 150, weight decay (weight_decay) is set to 0.0005, learning rate (Learning Rate) is set to 0.0001, and workers is set to 8. Both the pre-training of the first and second teacher networks and the training of the student network adopt existing technologies.

[0057] In this embodiment, in step 3, the calculation formula of the feature alignment loss is: Among them, Loss featurealignment represents feature alignment loss, b=1, 2, ..., batch_size, batch_size represents batch size, y b It represents the alignment feature map output by the first feature extraction layer after the bth feature map in each batch of the real car accident scene dataset is normalized and input into the student network. Represents the true label corresponding to the b-th feature map in each batch of the real car accident scene dataset. The normalization process of the b-th feature map in each batch of the real car accident scene dataset is: C represents the number of channels of the aligned feature map output by the first feature extraction layer in the student network, h = 1, 2, ..., H′, w = 1, 2, ..., W′, H′×W′ represents the size of the aligned feature map output by the first feature extraction layer in the student network, γ represents a learnable scaling parameter that can achieve separate scaling adjustment of each channel feature, and x b，c,h,w Represents the spatial dimension pixel value with coordinates (h, w) in the c-th channel of the b-th feature map in each batch of the real car accident scene dataset, μb，c Represents the average value of all spatial dimension pixel values ​​in the cth channel of the bth feature map in each batch of the real car accident scene dataset, Represents the variance of all spatial dimension pixel values ​​in the cth channel of the bth feature map in each batch of the real car accident scene dataset, β represents a learnable offset parameter, initialized to a full zero tensor, which can achieve separate offset adjustment for each channel feature, ∈ represents a small constant used to prevent the denominator from being zero, z b，c,h，w Represents the normalized spatial dimension pixel value with coordinates (h, w) in the c-th channel of the b-th feature map in each batch of the real car accident scene dataset.

[0058] In this embodiment, the calculation formula of contrastive learning loss is: Among them, Loss contrastive Represents contrastive learning loss, batch_size represents batch size, positive represents the positive samples in each batch of the real car accident scene dataset, negative represents the negative samples in each batch of the real car accident scene dataset, the positive samples are accident vehicle targets, the negative samples are normal vehicle targets, p≠q, CS p，q represents the cosine similarity between the p-th positive sample and the q-th positive sample, Indicates the negative samples and The contrastive learning loss encourages maximizing the cosine similarity between positive samples (decreasing the loss value) and minimizing the cosine similarity between negative samples (increasing the loss value).

[0059] In this embodiment, the calculation formula of knowledge distillation loss is: Among them, Loss distillation represents the knowledge distillation loss, L=2, represents the first feature extraction layer, represents the second feature extraction layer, MSE(·) represents the mean square error function, hour represents the aligned feature map output by the first feature extraction layer in the student network, hour represents the contrastive learning feature map output by the second feature extraction layer in the student network, hour represents the aligned feature map output by the first feature extraction layer in the first teacher network, hour represents the contrastive learning feature map output by the second feature extraction layer in the first teacher network, hour represents the aligned feature map output by the first feature extraction layer in the second teacher network, hour Represents the contrastive learning feature map output by the second feature extraction layer in the second teacher network. Knowledge distillation is used to guide the student network to simultaneously imitate the intermediate feature expressions of the two teacher networks, thereby improving the generalization ability in the accident vehicle detection scenario.

[0060] Step 4: Apply the student network detection model to the accident vehicle detection task in complex traffic scenes.

[0061] In order to test the performance of the method of the present invention, a comparative experiment was carried out.

[0062] The experimental hardware configuration is: CPU is IAMD Ryzen 75800X, RAM is 64G, GPU is iGame GeForce RTX3090Neptune OC; the environment is: Pytorch framework, CUDA11.8.

[0063] A pre-built dataset of real-world crash scenes includes accident vehicles and normal vehicles in various scenes, sizes, and lighting intensities to ensure the applicability and robustness of the vehicle collision detection network in real-world scenarios, thus meeting training requirements. The dataset consists of 2,400 images of 1920×1080 resolution, which are uniformly resized to 640×640 and divided into training and test sets in a 7:3 ratio. The batch_size is set to 8 during training.

[0064] A pre-built dataset of virtual car accident scenes includes accident vehicles and normal vehicles in different scenes, sizes, and lighting intensities, effectively simulating complex and diverse accident scenarios. The dataset consists of 10,000 images of 1920×1080 pixels, which are then resized to 640×640 pixels and divided into training and test sets in a 7:3 ratio. The batch_size is set to 8 during training.

[0065] The first teacher network was pre-trained using the BDD100K public dataset, and the second teacher network was pre-trained using a training set of a virtual car accident scene dataset. The weight parameters obtained from the pre-training of the first and second teacher networks were then used to guide the student network. The student network was then trained using a training set of a real car accident scene dataset to obtain the student network detection model.

[0066] The performance of the proposed method was compared with a comparative detection method based on the existing YOLOv11 architecture. Evaluation metrics included precision, recall, and mean average precision (mAP@0.5), where mAP@0.5 represents the average detection accuracy at an intersection-over-union (IoU) threshold of 0.5.

[0067] The method of the present invention and the comparative detection method based on the existing YOLOv11 structure were used to predict the test set of the real car accident scene dataset. The performance comparison of the predicted accident vehicle detection results is shown in Table 1. The method of the present invention and the comparative detection method based on the existing YOLOv11 structure were used to predict the test set of the virtual car accident scene dataset. The performance comparison of the predicted accident vehicle detection results is shown in Table 2.

[0068] Table 1 Performance comparison on the test set of the real car accident scene dataset

[0069] Table 2 Performance comparison on the test set of the virtual car accident scene dataset

[0070]

[0071] As can be seen from Table 1, on the test set of the real car accident scene dataset, the mAP@0.5 index of the proposed method reached 54.3%, which is 4.3 percentage points higher than the 50.0% of the comparison detection method based on the existing YOLOv11 structure, demonstrating the superior detection performance and accident vehicle recognition ability of the proposed method.

[0072] As can be seen from Table 2, on the test set of the virtual car accident scene dataset, the mAP@0.5 index of the proposed method reaches 88.8%, which is slightly higher than the 88.5% of the comparative detection method based on the existing YOLOv11 structure, indicating that the proposed method has good adaptability and robustness.

Claims

1. A vehicle collision detection method based on dual-teacher knowledge distillation, characterized by The following steps are involved: Step 1: Construct a vehicle collision detection dataset that integrates real-world data and virtual simulation data. The vehicle collision detection dataset includes a real car accident scene dataset and a virtual car accident scene dataset; Step 2: Based on the dual-teacher knowledge distillation framework, a vehicle collision detection network is constructed, which includes a student network and two teacher networks, namely the first teacher network and the second teacher network. Both the student network and the two teacher networks introduce a spatial-channel collaborative attention mechanism module for jointly modeling the spatial salient regions and the importance of channel features in the feature map, and both introduce a feature alignment and contrastive learning module for modeling the differences between normal vehicle targets and accident vehicle targets in the feature space. The feature alignment and contrastive learning module includes a first feature extraction layer for feature alignment and a second feature extraction layer for contrastive learning. The student network learns knowledge from the two pre-trained teacher networks simultaneously through the knowledge distillation method, and transfers the knowledge from the two pre-trained teacher networks to its own network, thereby improving its feature expression ability and recognition accuracy in the accident vehicle detection task. Step 3: Pre-train the first teacher network based on the public dataset to learn normal vehicle features; The second teacher network is pre-trained based on a virtual car accident scene dataset to learn the characteristics of the accident vehicles; After pre-training the first and second teacher networks, the student network is trained based on a dataset of real car accident scenes. The student network outputs the accident vehicle detection results corresponding to each real car accident scene image in the dataset. After each round of training, the feature alignment loss, contrastive learning loss, and knowledge distillation loss are calculated. The feature alignment loss is obtained by calculating the mean square error between the standardized data and the true label, the contrastive learning loss is obtained by calculating the cosine similarity between positive samples and the cosine similarity between negative samples, and the knowledge distillation loss is calculated by using Mimic loss to calculate the knowledge difference between the student network and the two teacher networks. The training process ends after the student network has trained for a preset number of rounds, and the student network detection model is obtained. Step 4: Apply the student network detection model to the accident vehicle detection task in complex traffic scenes.

2. A vehicle collision detection method based on dual-teacher knowledge distillation according to claim 1, characterized in that In step 1, the construction process of the real car accident scene dataset is as follows: searching for at least 2,000 real car accident scene images and forming a first dataset; then annotating the accident vehicle target and the normal vehicle target in each real car accident scene image in the first dataset, and saving the annotation information to obtain the annotated first dataset, wherein the annotation information includes the coordinate information of the four vertices of the annotation box and the category information of the annotation box, and the category information includes the accident vehicle target and the normal vehicle target; then adjusting the size of all the real car accident scene images in the annotated first dataset to be consistent to obtain the real car accident scene dataset.

3. A vehicle collision detection method based on dual-teacher knowledge distillation according to claim 1, characterized in that In step 1, the construction process of the virtual car accident scene dataset is as follows: the collision scene, weather, and lighting condition prompt words are set by the user to generate at least 10,000 virtual car accident scene images and form a second dataset; then the accident vehicle target and the normal vehicle target in each virtual car accident scene image in the second dataset are labeled, and the labeling information is saved to obtain the labeled second dataset, wherein the labeling information includes the coordinate information of the four vertices of the labeling box and the category information of the labeling box, and the category information includes the accident vehicle target and the normal vehicle target; then the size of all virtual car accident scene images in the labeled second dataset is adjusted to be consistent to obtain the virtual car accident scene dataset.

4. A vehicle collision detection method based on dual-teacher knowledge distillation according to claim 1, characterized in that In step 2, the structures of the student network and the two teacher networks are the same, which are composed of a first convolution module, a second convolution module, a third convolution module, a fourth convolution module, a fifth convolution module, a sixth convolution module, a seventh convolution module, a first space-channel collaborative attention mechanism layer, a second space-channel collaborative attention mechanism layer, a third space-channel collaborative attention mechanism layer, a fourth space-channel collaborative attention mechanism layer, a fifth space-channel collaborative attention mechanism layer, a sixth space-channel collaborative attention mechanism layer, a seventh space-channel collaborative attention mechanism layer, a first feature extraction layer, and a second feature extraction layer. , a spatial pyramid pooling layer, a first upsampling layer, a second upsampling layer, a first connection layer, a second connection layer, a third connection layer, a fourth connection layer, a first detection output layer, a second detection output layer, and a third detection output layer; the first spatial-channel collaborative attention mechanism layer to the seventh spatial-channel collaborative attention mechanism layer constitute a spatial-channel collaborative attention mechanism module; the first feature extraction layer is used for feature alignment, and the second feature extraction layer is used for contrastive learning; the first feature extraction layer and the second feature extraction layer constitute a feature alignment and contrastive learning module; the spatial pyramid pooling layer is used to fuse feature maps of different scales; The implementation process of the student network and the two teacher networks is the same, specifically: the image samples are sequentially passed through the first convolution module, the second convolution module, the first space-channel collaborative attention mechanism layer, the third convolution module, the second space-channel collaborative attention mechanism layer, the fourth convolution module, the first feature extraction layer, the fifth convolution module, the second feature extraction layer, the spatial pyramid pooling layer, the third space-channel collaborative attention mechanism layer, the second space-channel collaborative attention mechanism layer outputs the second space-channel joint feature map, the first feature extraction layer outputs the alignment feature map, and the third space-channel collaborative attention mechanism layer outputs the third space-channel joint feature map; Passing the third spatial-channel joint feature map through the first upsampling layer to obtain a first upsampling feature map; inputting the aligned feature map and the first upsampling feature map together into the first connection layer to obtain a first connection feature map; passing the first connection feature map sequentially through the fourth spatial-channel collaborative attention mechanism layer and the second upsampling layer; the fourth spatial-channel collaborative attention mechanism layer outputs the fourth spatial-channel joint feature map, and the second upsampling layer outputs the second upsampling feature map; The second spatial channel joint feature map and the second up-sampled feature map are input into the second connection layer together to obtain a second connection feature map; the second connection feature map is sequentially passed through the fifth spatial-channel collaborative attention mechanism layer and the sixth convolution module, and the fifth spatial-channel collaborative attention mechanism layer outputs the fifth spatial channel joint feature map; the fourth spatial channel joint feature map and the feature map output by the sixth convolution module are input into the third connection layer together to obtain a third connection feature map; the third connection feature map is sequentially passed through the sixth spatial-channel collaborative attention mechanism layer and the seventh convolution module, and the sixth spatial-channel collaborative attention mechanism layer outputs the sixth spatial channel joint feature map; the third spatial channel joint feature map and the feature map output by the seventh convolution module are input into the fourth connection layer together to obtain a fourth connection feature map; the fourth connection feature map is passed through the seventh spatial-channel collaborative attention mechanism layer to obtain a seventh spatial channel joint feature map; Passing the fifth spatial channel joint feature map through the first detection output layer to obtain a first detection output map; Passing the sixth spatial channel joint feature map through the second detection output layer to obtain a second detection output map; The joint feature map of the seventh spatial channel is passed through the third detection output layer to obtain the third detection output map; the first detection output map, the second detection output map, and the third detection output map are mapped to the coordinate space of the image sample, and the three detection output maps obtained after mapping are added together to obtain the accident vehicle detection result.

5. A vehicle collision detection method based on dual-teacher knowledge distillation according to claim 4, characterized in that The number of channels of the image sample is 3 and the size is H×W; the convolution kernel size of the first convolution module is 3×3, the stride is 2, the padding is 1, the number of convolution kernels is 64, and the size of the output feature map is (H / 2)×(W / 2); the convolution kernel size of the second convolution module is 3×3, the stride is 2, the padding is 1, the number of convolution kernels is 128, and the size of the output feature map is (H / 4)×(W / 4); the padding of the first spatial-channel collaborative attention mechanism layer is 2, the number of convolution kernels is 128, and the size of the output first spatial-channel joint feature map is (H / 4 )×(W / 4); the convolution kernel size of the third convolution module is 3×3, the stride is 2, the padding is 1, the number of convolution kernels is 256, and the size of the output feature map is (H / 8)×(W / 8); the padding of the second spatial-channel collaborative attention mechanism layer is 2, the number of convolution kernels is 256, and the size of the output second spatial-channel joint feature map is (H / 8)×(W / 8); the convolution kernel size of the fourth convolution module is 3×3, the stride is 2, the padding is 1, the number of convolution kernels is 512, and the size of the output feature map is (H / 16)×(W / 16); The padding of the first feature extraction layer is 2, the number of convolution kernels is 512, and the size of the output alignment feature map is (H / 16)×(W / 16); the convolution kernel size of the fifth convolution module is 3×3, the stride is 2, the padding is 1, the number of convolution kernels is 1024, and the size of the output feature map is (H / 32)×(W / 32); The padding of the second feature extraction layer is 2, the number of convolution kernels is 1024, and the size of the output contrastive learning feature map is (H / 32)×(W / 32); the convolution kernel size of the spatial pyramid pooling layer is 5×5, the stride is 1, the padding is 2, the number of convolution kernels is 1024, and the size of the output fusion feature map is (H / 32)×(W / 32); the padding of the third space-channel collaborative attention mechanism layer is 2, the number of convolution kernels is 1024, and the size of the output third space-channel joint feature map is (H / 3 2)×(W / 32); the size of the first up-sampled feature map output by the first upsampling layer is (H / 16)×(W / 16); the size of the first connected feature map output by the first connection layer is (H / 16)×(W / 16); the padding of the fourth spatial-channel collaborative attention mechanism layer is 2, the number of convolution kernels is 512, and the size of the fourth spatial-channel joint feature map output is (H / 16)×(W / 16); the size of the second up-sampled feature map output by the second upsampling layer is (H / 8)×(W / 8); The size of the second connection feature map output by the second connection layer is (H / 8)×(W / 8); The padding of the fifth spatial-channel collaborative attention mechanism layer is 2, the number of convolution kernels is 256, and the size of the output fifth spatial channel joint feature map is (H / 8)×(W / 8); the convolution kernel size of the sixth convolution module is 3×3, the stride is 2, the padding is 1, the number of convolution kernels is 256, and the size of the output feature map is (H / 16)×(W / 16); the size of the third connection feature map output by the third connection layer is (H / 16)×(W / 16); the padding of the sixth spatial-channel collaborative attention mechanism layer is 2, the number of convolution kernels is 512, and the size of the output sixth spatial channel joint feature map is (H / 16)×(W / 16); the convolution kernel size of the seventh convolution module is 3×3, the stride is 2, the padding is 1, the number of convolution kernels is 512, and the size of the output feature map is (H / 32)×(W / 32); the size of the fourth connection feature map output by the fourth connection layer is (H / 32)×(W / 32); the padding of the seventh space-channel collaborative attention mechanism layer is 2, the number of convolution kernels is 1024, and the size of the output seventh space-channel joint feature map is (H / 32)×(W / 32); the number of convolution kernels of the first detection output layer is 256, and the size of the output first detection output map is (H / 8)×(W / 8); the number of convolution kernels of the second detection output layer is 512, and the size of the output second detection output map is (H / 16)×(W / 16); the number of convolution kernels of the third detection output layer is 1024, and the size of the output third detection output map is (H / 32)×(W / 32); the number of channels of the accident vehicle detection result is 3 and the size is H×W.

6. A vehicle collision detection method based on dual-teacher knowledge distillation according to claim 4 or 5, characterized in that The structures of the first to seventh spatial-channel collaborative attention mechanism layers are the same, and are composed of the following modules: The global pooling module takes as input the feature map received by the i-th spatial-channel collaborative attention mechanism layer, downsamples the received feature map, and then performs channel normalization on the downsampled feature map. Its output is the normalized feature map; The QKV generation module takes the normalized feature map as input and consists of three parallel convolution branches that generate query, key, and value features respectively. Its output is query features, key features, and value features. The spatial attention extraction module takes as input the feature map received by the i-th spatial-channel collaborative attention mechanism layer. It includes four parallel one-dimensional depthwise separable convolutional layers with kernel sizes of 3×3, 5×5, 7×7, and 9×9, respectively. It uses four one-dimensional depthwise separable convolutional layers to generate four local spatial feature maps of different scales. The four local spatial feature maps are then concatenated in the channel dimension to obtain a spatially enhanced feature map. The output is the spatially enhanced feature map. The multi-head attention calculation module takes as input the query features, key features, value features, and spatial enhancement feature maps. It calculates the attention weights based on the query features, key features, and value features based on the scaled dot product method, and then multiplies the attention weights with the spatial enhancement feature map to obtain the attention feature map. The output is the attention feature map; The channel attention extraction module takes the attention feature map as input, dynamically adjusts the channel weights using global average pooling and gating mechanisms, and then recalibrates the attention feature map in the channel dimension using the adjusted channel weights. The output is the recalibrated feature map. The recalibrated feature map is multiplied channel by channel with the feature map received by the i-th spatial-channel collaborative attention mechanism layer, and the feature map obtained by the channel-by-channel multiplication is used as the feature map output by the i-th spatial-channel collaborative attention mechanism layer; where i belongs to {1, 2, 3, 4, 5, 6, 7}.

7. A vehicle collision detection method based on dual-teacher knowledge distillation according to claim 4 or 5, characterized in that The first feature extraction layer is composed of an eighth convolution module, a ninth convolution module, a first feature alignment operation, a second feature alignment operation, a first Bottleneck layer, a second Bottleneck layer, a third Bottleneck layer, and a fifth connection layer; the implementation process of the first feature extraction layer is: the feature map received by the first feature extraction layer passes through the eighth convolution module and the first feature alignment operation in sequence; the feature map obtained by the first feature alignment operation is divided into two tensors of the same size according to the channel direction, one of the tensors is input into the first Bottleneck layer, the second Bottleneck layer, and the third Bottleneck layer respectively, and the other tensor is used as the first input and the second input of the fifth connection layer respectively; the feature maps output by the first Bottleneck layer, the second Bottleneck layer, and the third Bottleneck layer are respectively used as the third input, the fourth input, and the fifth input of the fifth connection layer, and the fifth connection layer splices the first input, the second input, the third input, the fourth input, and the fifth input in the channel dimension; the feature map output by the fifth connection layer passes through the ninth convolution module and the second feature alignment operation in sequence; the feature map obtained by the second feature alignment operation is used as the alignment feature map output by the first feature extraction layer; The second feature extraction layer is composed of the tenth convolution module, the eleventh convolution module, the twelfth convolution module, the thirteenth convolution module, the Module List module, and the Projection Head module. The Module List module includes the fourth Bottleneck layer, the fifth Bottleneck layer, and the sixth Bottleneck layer connected in sequence. The head module includes a fourteenth convolution module, a fifteenth convolution module, a batch normalization layer, and a ReLu activation layer connected in sequence; the implementation process of the second feature extraction layer is: the feature map received by the second feature extraction layer is sequentially passed through the tenth convolution module, the eleventh convolution module, the twelfth convolution module, the thirteenth convolution module, the fourth Bottleneck layer, the fifth Bottleneck layer, and the sixth Bottleneck layer to obtain the main path output feature map; when the number of targets in the feature map received by the second feature extraction layer is not 0, the main path output feature map is sequentially passed through the fourteenth convolution module, the fifteenth convolution module, the batch normalization layer, and the ReLu activation layer, and the feature map output by the ReLu activation layer is used as the comparative learning feature map output by the second feature extraction layer; when the number of targets in the feature map received by the second feature extraction layer is 0, the main path output feature map is directly used as the comparative learning feature map output by the second feature extraction layer, wherein the targets include normal vehicle targets and accident vehicle targets; The first connection layer to the fourth connection layer are all used to achieve splicing in the channel dimension.

8. The vehicle collision detection method based on dual-teacher knowledge distillation according to claim 1 is characterized in that In step 3, during the pre-training of the first teacher network and the second teacher network and the training of the student network, the batch size is set to 8, the rounds are set to 150 times, the weight decay is set to 0.0005, and the learning rate is set to 0.0001.

9. The vehicle collision detection method based on dual-teacher knowledge distillation according to claim 1 is characterized in that In step 3, the calculation formula of feature alignment loss is: Among them, Loss featurealignment represents feature alignment loss, b=1, 2, ..., batch_size, batch_size represents batch size, y b It represents the alignment feature map output by the first feature extraction layer after the bth feature map in each batch of the real car accident scene dataset is normalized and input into the student network. Represents the true label corresponding to the b-th feature map in each batch of the real car accident scene dataset. The normalization process of the b-th feature map in each batch of the real car accident scene dataset is: c = 1, 2, ..., C, where C represents the number of channels of the aligned feature map output by the first feature extraction layer in the student network, h = 1, 2, ..., H′, w = 1, 2, ..., W′, where H′×W′ represents the size of the aligned feature map output by the first feature extraction layer in the student network, γ represents a learnable scaling parameter, and x b，c，h，w Represents the spatial dimension pixel value with coordinates (h, w) in the c-th channel of the b-th feature map in each batch of the real car accident scene dataset, μ b，c Represents the average value of all spatial dimension pixel values ​​in the cth channel of the bth feature map in each batch of the real car accident scene dataset, Represents the variance of all spatial dimension pixel values ​​in the cth channel of the bth feature map in each batch of the real car accident scene dataset, β represents a learnable bias parameter, ∈ represents a small constant used to prevent the denominator from being zero, and z b，c，h，w Represents the normalized spatial dimension pixel value with coordinates (h, w) in the c-th channel of the b-th feature map in each batch of the real car accident scene dataset; The formula for contrastive learning loss is: Among them, Loss contrastive Represents contrastive learning loss, batch_size represents batch size, positive represents the positive samples in each batch of the real car accident scene dataset, negative represents the negative samples in each batch of the real car accident scene dataset, the positive samples are accident vehicle targets, the negative samples are normal vehicle targets, p≠q, CS p，q represents the cosine similarity between the p-th positive sample and the q-th positive sample, Indicates the negative samples and The cosine similarity between negative samples; The calculation formula of knowledge distillation loss is: Among them, Loss distillation represents the knowledge distillation loss, L = 2, l = 1 represents the first feature extraction layer, l = 2 represents the second feature extraction layer, MSE (·) represents the mean square error function, l = 1 S l represents the aligned feature map output by the first feature extraction layer in the student network, and S is the output of the first feature extraction layer in the student network. l Represents the contrastive learning feature map output by the second feature extraction layer in the student network, when l=1, T 1，l represents the aligned feature map output by the first feature extraction layer in the first teacher network, when l=2, T 1，l represents the contrastive learning feature map output by the second feature extraction layer in the first teacher network, when l=1, T 2，l represents the aligned feature map output by the first feature extraction layer in the second teacher network, when l=2, T 2，l Represents the contrastive learning feature map output of the second feature extraction layer in the second teacher network.

Images